PARQUET argument is not supported when loading from file system

0

In the AWS documentation is states I should be able to COPY parquet files into Redshift. (https://aws.amazon.com/about-aws/whats-new/2018/06/amazon-redshift-can-now-copy-from-parquet-and-orc-file-formats/) I am having a dog of a time making this work.

After creating a role with S3ReadOnlyAccess, AWSGlueFullAccess, and AWSAthenaFullAccess, I trusted the role to the user account of Redshift. I then added the IAM role to the Redshift Cluster.

However after running the COPY command I received the error:
PARQUET argument is not supported when loading from file system

I am attempting to load it from an S3 location. Any idea why this error is being thrown? Is PARQUET not supported as an Importable file into Redshift from S3, or am I not giving the role the correct access to something?

Any help is appreciated.

Thanks,
Stephen.

Edited by: roland23571113 on Jun 8, 2018 6:47 PM

Edited by: roland23571113 on Jun 11, 2018 1:32 PM
I have attached a pointer to the documentation I read online. This does not work on my US-WEST-2 redshift cluster. https://aws.amazon.com/about-aws/whats-new/2018/06/amazon-redshift-can-now-copy-from-parquet-and-orc-file-formats/

Edited by: roland23571113 on Jun 12, 2018 10:50 AM
After discussing with the aws architect it appears the role settings are correct:
AWSS3ReadOnly
AWSGlueFullControl
AWSAthenaFullControl

Assigning trust of the Redshift user account to the role was done correctly.

Adding the role to the AWS Cluster was done correctly.

The copy command was the only thing done incorrectly.

The trick is that the S3 bucket path must point to a folder containing the .parquet file, not the file itself. Any file found in the folder is assumed to be uploaded as a parquet file.

copy <table_name>
from "<S3 Folder Path>
IAM_ROLE '<role string>'
FORMAT AS PARQUET

gefragt vor 6 Jahren1828 Aufrufe
1 Antwort
0

Answered by AWS Architect Review.

beantwortet vor 6 Jahren

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen