Copy command is ignoring extra columns in case of BLANK values

Question

Hi there,  
  
We came across the below issue while copying data from CSV file in S3 to Redshift.  
  
Copy command is ignoring extra columns in case of BLANK values in the last columns of the records.  
  
Example -   
  
Redshift table has 5 columns and CSV file in S3 contains 7 columns(last two columns are NULL/BLANK).   
  
When we tried to use COPY command to load data from file to Redshift, COPY command didn't fail but loaded data from first 5 columns in file to Redshift table.  
  
Please post your views on this, if you experienced this issue before.  
  
Thanks,  
Mahesh

Answer

The extra columns are allowed by design.   
  
Users often need to add new columns to extracts but they may not wish to add those columns to the Redshift table.   
  
For the reverse case, the FILLRECORD parameter allows files with columns missing at the end each row to be loaded.  
https://docs.aws.amazon.com/redshift/latest/dg/copy-parameters-data-conversion.html#copy-fillrecord

Answer

No, we are not mentioning the column names in the COPY command.   
  
Does it cause an issue?  
  
Thanks,  
Mahesh

Answer

In the COPY command, are you specifying the column names?

Copy command is ignoring extra columns in case of BLANK values

Contenuto pertinente