Glue ETL job to load data from S3 to Redshift failed

Question

I am running a Glue ETL job to load data from parquet file from S3 into Redshift table.
![Enter image description here](/media/postImages/original/IMrrvRKNfNQ0604h-FuKYtFQ)

I am able to get data preview while configuring connections but job fails with below error.

`Error Category: UNCLASSIFIED_ERROR; Failed Line Number: 31; An error occurred while calling o112.pyWriteDynamicFrame. Exception thrown in awaitResult:`

Please let me issue and provide solution.

Accepted Answer

I would recommend you switch to the script and edit it to add a few things:

1. print out the schema of the dynamic frame of the source. See if that matches your Redshift schema.
2. Look for nested fields/structures - you might need a Resolve choice
3. use an Apply mapping transform if your schema doesn't match Redshift
4. check permissions of the IAM role of your job

Answer

The data preview won't really insert into Redshift, which is what is really failing, likely a permission issue.  
Look in the log for a cause, also looking into the Redshift error table might help (if it got that far): https://docs.aws.amazon.com/redshift/latest/dg/r_STL_LOAD_ERRORS.html

Glue ETL job to load data from S3 to Redshift failed

相关内容