Athena HIVE_METASTORE_ERROR when working with map<string, string> columns in parquet file

0

I have plenty of other Athena tables using parquet files, so the setup is okay, but this particular table has a column (metadata) that is of type map<string, string>. The parquet file - when loaded with pyarrow from the CLI correctly shows the table schema for that column:

study_id: string
subject_id: string
sample_id: string
collection_date: date32[day]
genotyping: string
metadata: map<string, string>
  child 0, entries: struct<key: string not null, value: string> not null
      child 0, key: string not null
      child 1, value: string

And I load load the parquet file just in back into Pandas as well. However, when I attempt to query the table with Athena I get this:

HIVE_METASTORE_ERROR: Error: type expected at the position 11 of 'map<string, string>' but ' ' is found.

I've been fighting this for several days now trying different things to no avail. How can I properly save this parquet file with the metadata column so that Athena can read it?

質問済み 2年前146ビュー
回答なし

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ