while importing s3 data into quicksight from glue database getting the following error
HIVE_PARTITION_SCHEMA_MISMATCH: There is a mismatch between the table and partition schemas. The types are incompatible and cannot be coerced. The column 'attributes' in table 'clickstream-ctr-database.care_events_dev1' is declared as type 'structaid:string,kvstriggerlambdaresult:string,sak:string,savecallrecording:string,sst:string,startstreamingaudiostatus:string,transcribecall:string,getlang:string,getaeroplanno:string,getemergency:string,getholiday:string,getlanguage:string,getlanguageselected:string,getpin:string,getpriority:string,language:string,userlanguage:string', but partition 'partition_0=ctr/partition_1=2022/partition_2=04/partition_3=04/partition_4=12' declared column 'attributes' as type 'structgetaeroplanno:string,getemergency:string,getholiday:string,getlanguage:string,getlanguageselected:string,getpin:string,getpriority:string'.
I expect that you have used a crawler to add the catalog information, is that correct?
The issue that the logs is mentioning is that while for the table the crawler had discovered a the schema with a certain number of nested values in the field attribute that is a struct , at least one of the partition has a different set of valued in the same struct.
to avoid that you could try to delete the table and re-run the crawler modifying it to prevent changes to the schema as described here, specifically you could try to configure it to
Update all new and existing partitions with metadata from the table .
Also, I am not sure QuickSight will support the data type struct, so you might want to modify the data type to string in the console or via APIs.
hope this helps,
Thank you @Fabrizio@AWS. It's resolved my issue
HIVE_PARTITION_SCHEMA_MISMATCH - Athena error on S3 Parquet fileasked 2 months ago
Partition schema mismatch in Glue Tableasked a month ago
Automating data updates on Quicksightasked 4 months ago
AWS Glue API get-partitions can't seem to cope with the partition column name "key" in the `expression` filterasked 4 months ago
Getting error on AWS Athena while setting up Agent event data stream to S3 using AWS Glueasked 5 days ago
How to insert S3 data into Aurora table via glue transform?asked 3 years ago
while importing s3 data into quicksight from glue database getting the following errorAccepted Answerasked 3 months ago
Best way to overcome HIVE_PARTITION_SCHEMA_MISMATCH error in Athena while preserving structure of structs?asked 3 months ago
Can Glue crawler be configured to include only the most recent partition in a table?asked a month ago
Glue Crawler error: Folder partition keys do not match table partition keysasked 4 months ago