Getting an error message when querying an external parquet table with a nested structure: HIVE_CANNOT_OPEN_SPLIT: Error opening Hive split ... required group stats {{parquet schema}} is not primitive

1

The parquet has a nested structure where a map contains a struct which is itself nested. i'm not sure if athen'as serDe can handle it, but i would like to find a way to read it with athena. The table creation worked fine, and the structure i described matches the parquet.

CREATE EXTERNAL TABLE amitworkloads14( name string, .... containers map<string,struct< name:string, ...... stats:struct< lastCpuReq:double, .... oomEventsTimed:map<string,double>, cpuThrottleTimed:map<string,struct< avgThrottle:double, ... >>, ... >

, maxInstances bigint, ... runningMinutesByNodeGroup map<string,bigint> ) STORED AS PARQUET LOCATION 's3://dev-customers-data/test/new-pqt-lib/workloads/' tblproperties ("parquet.compression"="SNAPPY") The full error message: HIVE_CANNOT_OPEN_SPLIT: Error opening Hive split s3://dev-customers-data/test/new-pqt-lib/workloads/new_workloads.parquet (offset=0, length=2227521): required group stats { ..................... } is not primitive This query ran against the "new-pqt-lib" database, unless qualified by the query. Please post the error message on our forum or contact customer support with Query Id: 4a72c1ed-fd99-4149-9f57-af240ccbef6b

gefragt vor einem Jahr270 Aufrufe
Keine Antworten

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen