Copy from S3 parquet to Redshift table

0

Hi!
I tried to copy parquet files from S3 to Redshift table but instead I got an error:

  Invalid operation: COPY from this file format only accepts IAM_ROLE credentials

I provide User credentials what should work according to that document: https://docs.aws.amazon.com/redshift/latest/dg/copy-usage_notes-copy-from-columnar.html
"COPY command credentials must be supplied using an AWS Identity and Access Management (IAM) role as an argument for the IAM_ROLE parameter or the CREDENTIALS parameter."

My code:

COPY {table_name}
FROM {path_to_s3}
CREDENTIALS 'aws_access_key_id=...;aws_secret_access_key=...'
FORMAT PARQUET
;

Is it possible to load parquet data using user credentials ?

asked 5 years ago2602 views
1 Answer
0

"COPY command credentials must be supplied using an AWS Identity and Access Management (IAM) role as an argument for the IAM_ROLE parameter or the CREDENTIALS parameter."
You must use a role but you can declare the role to either the IAM_ROLE or CREDENTIALS parameters.

CREDENTIALS 'aws_iam_role=arn:aws:iam::<aws-account-id>:role/<role-name>'
IAM_ROLE 'arn:aws:iam::<aws-account-id>:role/<role-name'

I recommend that you transition to using IAM roles for access control in general, rather than embedding keys in your processes.

answered 5 years ago
  • I found the problem in our upload code, the target table has a TIME column which is not supported.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions