while importing s3 data into quicksight from glue database getting the following error

Question

HIVE_PARTITION_SCHEMA_MISMATCH: There is a mismatch between the table and partition schemas. The types are incompatible and cannot be coerced. The column 'attributes' in table 'clickstream-ctr-database.care_events_dev1' is declared as type 'struct', but partition 'partition_0=ctr/partition_1=2022/partition_2=04/partition_3=04/partition_4=12' declared column 'attributes' as type 'struct'.

Accepted Answer

Hi,

I expect that you have used a crawler to add the catalog information, is that correct?

The issue that the logs is mentioning is that while for the table the crawler had discovered a the schema with a certain number of nested values in the field attribute that is a struct , at least one of the partition has a different set of valued in the same struct.

to avoid that you could try to delete the table and re-run the crawler modifying it to prevent changes to the schema as[ described here](https://docs.aws.amazon.com/glue/latest/dg/crawler-configuration.html), specifically you could try to configure it to `Update all new and existing partitions with metadata from the table ` .

Also, I am not sure QuickSight will support the data type struct, so you might want to modify the data type to string in the console or via APIs.

hope this helps,

while importing s3 data into quicksight from glue database getting the following error

相關內容