Getting an error message when querying an external parquet table with a nested structure: HIVE_CANNOT_OPEN_SPLIT: Error opening Hive split ... required group stats {{parquet schema}} is not primitive

1

The parquet has a nested structure where a map contains a struct which is itself nested. i'm not sure if athen'as serDe can handle it, but i would like to find a way to read it with athena. The table creation worked fine, and the structure i described matches the parquet.

CREATE EXTERNAL TABLE amitworkloads14( name string, .... containers map<string,struct< name:string, ...... stats:struct< lastCpuReq:double, .... oomEventsTimed:map<string,double>, cpuThrottleTimed:map<string,struct< avgThrottle:double, ... >>, ... >

, maxInstances bigint, ... runningMinutesByNodeGroup map<string,bigint> ) STORED AS PARQUET LOCATION 's3://dev-customers-data/test/new-pqt-lib/workloads/' tblproperties ("parquet.compression"="SNAPPY") The full error message: HIVE_CANNOT_OPEN_SPLIT: Error opening Hive split s3://dev-customers-data/test/new-pqt-lib/workloads/new_workloads.parquet (offset=0, length=2227521): required group stats { ..................... } is not primitive This query ran against the "new-pqt-lib" database, unless qualified by the query. Please post the error message on our forum or contact customer support with Query Id: 4a72c1ed-fd99-4149-9f57-af240ccbef6b

asked a year ago268 views
No Answers

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions