Data in partitions on S3 bucket, getting below error in preview data query:

0

HIVE_PARTITION_SCHEMA_MISMATCH: There is a mismatch between the table and partition schemas. The types are incompatible and cannot be coerced. The column 'site_rank' in table 'luci-data.luci-newslinkdata' is declared as type 'double', but partition 'partition_0=2011-12-30' declared column 'site_rank' as type 'bigint'. then, I followed the steps mentioned in: repost.aws/questions/QU1vPk1069Q5qg4iyuQWTk6Q/while-importing-s3-data-into-quicksight-from-glue-database-getting-the-following-error#ANMuJykKnKSD-jKoZk2tbvqw after updating in glue crawler: Update all new and existing partitions with metadata from the table Now the error is as follows: HIVE_BAD_DATA: Field site_rank's type INT64 in parquet file s3://rnvd-s3-luci-poc/NewsLinkData/2011-06-26/output_4404.parquet is incompatible with type double defined in table schema

demandé il y a un an265 vues
1 réponse
0

What you are trying to do is not allowed, you cannot just change the types in the metadata if they don't match (or are compatible) with the actual data files.
If you don't want to update the data files, you can read as bigint and then cast in your query.
If you have mixture of parquet files with both types, the table is broken, replace/delete one of the set of files.

profile pictureAWS
EXPERT
répondu il y a un an

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions