By using AWS re:Post, you agree to the Terms of Use
/Athena : serDe should not be accessed from a null StorageFormat/

Athena : serDe should not be accessed from a null StorageFormat

0

I'm stroing cloudtrail logs into the S3 and through glue crwaler trying to crawl the S3(cloudtrail logs) and creating a glue table. While I'm running query against that table in Athena , getting below error

HIVE_UNKNOWN_ERROR: serDe should not be accessed from a null StorageFormat

Can someone please help in fixing the issue or tell the workaround. I'm creating all the resources through terraform.

1 Answers
0

Check the table parameters for the SerdeInfo of the StorageDescriptor. The crawlers don't always detect and configure the InputFormat, OutputFormat and SerializationLibrary parameters, so they can be NULL. Ex:

$ aws glue get-table --region <region> --profile <profile> --database-name <db name> --name <table name> --query '*[].StorageDescriptor.[InputFormat,OutputFormat,SerdeInfo.[SerializationLibrary]]'

[ [ null, null, null ] ]

If you manually edit the table (and recreate any partitions), then you'll be cleared to query. Ex: (for a .csv file)

[ [ "org.apache.hadoop.mapred.TextInputFormat", "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat", "org.apache.hadoop.hive.serde2.OpenCSVSerde" ] ]

answered 2 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions