By using AWS re:Post, you agree to the Terms of Use
/HIVE_PARTITION_SCHEMA_MISMATCH - Athena error on S3 Parquet file/

HIVE_PARTITION_SCHEMA_MISMATCH - Athena error on S3 Parquet file


Getting below error while querying S3 parquet file from Athena. The schema has changed from timestamp to string for one of the columns, is there any workaround to have the latest schema changed to string (Alter table and msck repair table didn't work).

HIVE_PARTITION_SCHEMA_MISMATCH: There is a mismatch between the table and partition schemas. The types are incompatible and cannot be coerced. The column 'birth_year' in table 'curated_ndb_datapipeline.registered_nurse' is declared as type 'timestamp', but partition 'data_year=2021/prov_terr_reg=959' declared column 'birth_year' as type 'string'

asked a month ago7 views
1 Answers

If all partitions have the same schema with the column defined as string and only the table has the column defined as timestamp you could try to:

  1. run a Glue crawler and see if it correctly updates the table
  2. use the Glue APis to update the column data type where needed (if it is only at table level you could consider to do it in the console).

hope this helps.

answered a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions