PARQUET argument is not supported when loading from file system

0

In the AWS documentation is states I should be able to COPY parquet files into Redshift. (https://aws.amazon.com/about-aws/whats-new/2018/06/amazon-redshift-can-now-copy-from-parquet-and-orc-file-formats/) I am having a dog of a time making this work.

After creating a role with S3ReadOnlyAccess, AWSGlueFullAccess, and AWSAthenaFullAccess, I trusted the role to the user account of Redshift. I then added the IAM role to the Redshift Cluster.

However after running the COPY command I received the error:
PARQUET argument is not supported when loading from file system

I am attempting to load it from an S3 location. Any idea why this error is being thrown? Is PARQUET not supported as an Importable file into Redshift from S3, or am I not giving the role the correct access to something?

Any help is appreciated.

Thanks,
Stephen.

Edited by: roland23571113 on Jun 8, 2018 6:47 PM

Edited by: roland23571113 on Jun 11, 2018 1:32 PM
I have attached a pointer to the documentation I read online. This does not work on my US-WEST-2 redshift cluster. https://aws.amazon.com/about-aws/whats-new/2018/06/amazon-redshift-can-now-copy-from-parquet-and-orc-file-formats/

Edited by: roland23571113 on Jun 12, 2018 10:50 AM
After discussing with the aws architect it appears the role settings are correct:
AWSS3ReadOnly
AWSGlueFullControl
AWSAthenaFullControl

Assigning trust of the Redshift user account to the role was done correctly.

Adding the role to the AWS Cluster was done correctly.

The copy command was the only thing done incorrectly.

The trick is that the S3 bucket path must point to a folder containing the .parquet file, not the file itself. Any file found in the folder is assumed to be uploaded as a parquet file.

copy <table_name>
from "<S3 Folder Path>
IAM_ROLE '<role string>'
FORMAT AS PARQUET

preguntada hace 6 años1829 visualizaciones
1 Respuesta
0

Answered by AWS Architect Review.

respondido hace 6 años

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas