while importing s3 data into quicksight from glue database getting the following error
HIVE_PARTITION_SCHEMA_MISMATCH: There is a mismatch between the table and partition schemas. The types are incompatible and cannot be coerced. The column 'attributes' in table 'clickstream-ctr-database.care_events_dev1' is declared as type 'structaid:string,kvstriggerlambdaresult:string,sak:string,savecallrecording:string,sst:string,startstreamingaudiostatus:string,transcribecall:string,getlang:string,getaeroplanno:string,getemergency:string,getholiday:string,getlanguage:string,getlanguageselected:string,getpin:string,getpriority:string,language:string,userlanguage:string', but partition 'partition_0=ctr/partition_1=2022/partition_2=04/partition_3=04/partition_4=12' declared column 'attributes' as type 'structgetaeroplanno:string,getemergency:string,getholiday:string,getlanguage:string,getlanguageselected:string,getpin:string,getpriority:string'.
Hi,
I expect that you have used a crawler to add the catalog information, is that correct?
The issue that the logs is mentioning is that while for the table the crawler had discovered a the schema with a certain number of nested values in the field attribute that is a struct , at least one of the partition has a different set of valued in the same struct.
to avoid that you could try to delete the table and re-run the crawler modifying it to prevent changes to the schema as described here, specifically you could try to configure it to Update all new and existing partitions with metadata from the table .
Also, I am not sure QuickSight will support the data type struct, so you might want to modify the data type to string in the console or via APIs.
hope this helps,
Relevant questions
HIVE_PARTITION_SCHEMA_MISMATCH - Athena error on S3 Parquet file
asked 2 months agoPartition schema mismatch in Glue Table
asked a month agoAutomating data updates on Quicksight
asked 4 months agoAWS Glue API get-partitions can't seem to cope with the partition column name "key" in the `expression` filter
asked 4 months agoGetting error on AWS Athena while setting up Agent event data stream to S3 using AWS Glue
asked 5 days agoHow to insert S3 data into Aurora table via glue transform?
asked 3 years agowhile importing s3 data into quicksight from glue database getting the following error
Accepted Answerasked 3 months agoBest way to overcome HIVE_PARTITION_SCHEMA_MISMATCH error in Athena while preserving structure of structs?
asked 3 months agoCan Glue crawler be configured to include only the most recent partition in a table?
asked a month agoGlue Crawler error: Folder partition keys do not match table partition keys
asked 4 months ago
Thank you @Fabrizio@AWS. It's resolved my issue