Getting the below error for athena query.

Question

Hi Team,

Im working on with a project on integrating the logs from waf and pushing the log to S3 using kinesis delivery stream which is working fine. 
Now from aws glue crawler im trying to pull the data into its own inbuilt database and that is also happening. But when the query is being run im getting the below error.

**HIVE_PARTITION_SCHEMA_MISMATCH: There is a mismatch between the table and partition schemas. The types are incompatible and cannot be coerced. The column 'requestbodysize' in table 'default.jsonwaf_log_trafficlog' is declared as type 'int', but partition 'partition_0=2023/partition_1=05/partition_2=19/partition_3=21' declared column 'labels' as type 'array>'.
This query ran against the "" database, unless qualified by the query. Please post the error message on our forum 
or contact customer support 
with Query Id: f684c53c-d74f-45d0-b690-bd221bb762ac**

when this error comes i delete three field from the table requestbodysize and labels and requestbodyinspectedbywaf. Post deleting these three fields. im getting the query run successfully.

Each and everytime the crawler pulls the data from S3 the table is getting updated and the above deleted columns pop up again and getting the same error. What could be the solution for this. could this be any bug or supporting issue between the table and athena.

Need your solution on the team. Awaiting updates from you all.

Regards

Answer

Assuming the files have a consistent schema, is possible that the sample the crawler takes confuses it and is not able to resolve the conflict.  
A simple solution that comes to mind is setting the right schema in the table and then configure the crawler to just add new partitions without touching the schema.

Getting the below error for athena query.

Relevant content