s3 parquet partitions load to redshift using COPY command

0

Hello,

I have folder called s3://bucket/redshift-backup-test/folder_fact_ads_spend/fact_ads_spend_test.parquet/ inside this there partitions like Market=AU,Market=US,Market=SG (all data in parquet format)

columns: date,market,price,quantity

I am trying to load this using COPY command into my redshift database where I have same columns .

But I am not able to copy the data and error is columns not matching ( file having 3 and db have 4)

I have tried to remove the "Market" field from db and it worked.

Now my question is how I can load "Market" field which is partition column to db also.

gefragt vor 2 Jahren2220 Aufrufe
1 Antwort
0

Hi, you can try using the FILLRECORD paramater - FORMAT PARQUET FILLRECORD; see this blog for more details - https://aws.amazon.com/blogs/big-data/part-1-introducing-new-features-for-amazon-redshift-copy/

profile pictureAWS
beantwortet vor 2 Jahren

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen