- Newest
- Most votes
- Most comments
There are several versions of the HIVE_BAD_DATA error. One reason might be because The data type defined in the table definition doesn't match the actual source data and another reason might be a single field contains different types of data (for example, a boolean value for one record and a decimal value for another record).
In your case, you need to change the type in the schema to the appropriate data type (in your case it's binary).
I would suggest you format your data in AWS Glue (ETL Programming) then you can load into Athena Via Glue Data Catalog or directly from query.
Ref Links:
https://aws.amazon.com/premiumsupport/knowledge-center/athena-hive-bad-data-parsing-field-value/
https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-format.html
As per my understanding you have some files where the column is typed as binary and some where it is typed as boolean. When you type the column of the table as boolean Athena will eventually read a file where the corresponding column is boolean and throw this error, and vice versa.The solution is to make sure your files all have the same schema.
Relevant content
- asked 2 years ago
- asked 8 months ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated 7 months ago