By using AWS re:Post, you agree to the Terms of Use

PARQUET argument is not supported when loading from file system

0

In the AWS documentation is states I should be able to COPY parquet files into Redshift. (https://aws.amazon.com/about-aws/whats-new/2018/06/amazon-redshift-can-now-copy-from-parquet-and-orc-file-formats/) I am having a dog of a time making this work.

After creating a role with S3ReadOnlyAccess, AWSGlueFullAccess, and AWSAthenaFullAccess, I trusted the role to the user account of Redshift. I then added the IAM role to the Redshift Cluster.

However after running the COPY command I received the error:
PARQUET argument is not supported when loading from file system

I am attempting to load it from an S3 location. Any idea why this error is being thrown? Is PARQUET not supported as an Importable file into Redshift from S3, or am I not giving the role the correct access to something?

Any help is appreciated.

Thanks,
Stephen.

Edited by: roland23571113 on Jun 8, 2018 6:47 PM

Edited by: roland23571113 on Jun 11, 2018 1:32 PM
I have attached a pointer to the documentation I read online. This does not work on my US-WEST-2 redshift cluster. https://aws.amazon.com/about-aws/whats-new/2018/06/amazon-redshift-can-now-copy-from-parquet-and-orc-file-formats/

Edited by: roland23571113 on Jun 12, 2018 10:50 AM
After discussing with the aws architect it appears the role settings are correct:
AWSS3ReadOnly
AWSGlueFullControl
AWSAthenaFullControl

Assigning trust of the Redshift user account to the role was done correctly.

Adding the role to the AWS Cluster was done correctly.

The copy command was the only thing done incorrectly.

The trick is that the S3 bucket path must point to a folder containing the .parquet file, not the file itself. Any file found in the folder is assumed to be uploaded as a parquet file.

copy <table_name>
from "<S3 Folder Path>
IAM_ROLE '<role string>'
FORMAT AS PARQUET

asked 4 years ago342 views
1 Answer
0

Answered by AWS Architect Review.

answered 4 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions