Athena HIVE_METASTORE_ERROR when working with map<string, string> columns in parquet file

0

I have plenty of other Athena tables using parquet files, so the setup is okay, but this particular table has a column (metadata) that is of type map<string, string>. The parquet file - when loaded with pyarrow from the CLI correctly shows the table schema for that column:

study_id: string
subject_id: string
sample_id: string
collection_date: date32[day]
genotyping: string
metadata: map<string, string>
  child 0, entries: struct<key: string not null, value: string> not null
      child 0, key: string not null
      child 1, value: string

And I load load the parquet file just in back into Pandas as well. However, when I attempt to query the table with Athena I get this:

HIVE_METASTORE_ERROR: Error: type expected at the position 11 of 'map<string, string>' but ' ' is found.

I've been fighting this for several days now trying different things to no avail. How can I properly save this parquet file with the metadata column so that Athena can read it?

已提問 2 年前檢視次數 152 次
沒有答案

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南