while importing s3 data into quicksight from glue database getting the following error

0

HIVE_PARTITION_SCHEMA_MISMATCH: There is a mismatch between the table and partition schemas. The types are incompatible and cannot be coerced. The column 'attributes' in table 'clickstream-ctr-database.care_events_dev1' is declared as type 'structaid:string,kvstriggerlambdaresult:string,sak:string,savecallrecording:string,sst:string,startstreamingaudiostatus:string,transcribecall:string,getlang:string,getaeroplanno:string,getemergency:string,getholiday:string,getlanguage:string,getlanguageselected:string,getpin:string,getpriority:string,language:string,userlanguage:string', but partition 'partition_0=ctr/partition_1=2022/partition_2=04/partition_3=04/partition_4=12' declared column 'attributes' as type 'structgetaeroplanno:string,getemergency:string,getholiday:string,getlanguage:string,getlanguageselected:string,getpin:string,getpriority:string'.

asked 2 years ago595 views
1 Answer
1
Accepted Answer

Hi,

I expect that you have used a crawler to add the catalog information, is that correct?

The issue that the logs is mentioning is that while for the table the crawler had discovered a the schema with a certain number of nested values in the field attribute that is a struct , at least one of the partition has a different set of valued in the same struct.

to avoid that you could try to delete the table and re-run the crawler modifying it to prevent changes to the schema as described here, specifically you could try to configure it to Update all new and existing partitions with metadata from the table .

Also, I am not sure QuickSight will support the data type struct, so you might want to modify the data type to string in the console or via APIs.

hope this helps,

AWS
EXPERT
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions