By using AWS re:Post, you agree to the Terms of Use

Copy command is ignoring extra columns in case of BLANK values

0

Hi there,

We came across the below issue while copying data from CSV file in S3 to Redshift.

Copy command is ignoring extra columns in case of BLANK values in the last columns of the records.

Example -

Redshift table has 5 columns and CSV file in S3 contains 7 columns(last two columns are NULL/BLANK).

When we tried to use COPY command to load data from file to Redshift, COPY command didn't fail but loaded data from first 5 columns in file to Redshift table.

Please post your views on this, if you experienced this issue before.

Thanks,
Mahesh

asked 4 years ago213 views
3 Answers
0

In the COPY command, are you specifying the column names?

answered 4 years ago
0

No, we are not mentioning the column names in the COPY command.

Does it cause an issue?

Thanks,
Mahesh

answered 4 years ago
0

The extra columns are allowed by design.

Users often need to add new columns to extracts but they may not wish to add those columns to the Redshift table.

For the reverse case, the FILLRECORD parameter allows files with columns missing at the end each row to be loaded.
https://docs.aws.amazon.com/redshift/latest/dg/copy-parameters-data-conversion.html#copy-fillrecord

answered 4 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions