Error Category: UNCLASSIFIED_ERROR; An error occurred while calling o99.getJDBCSink. The connection attempt failed.

Question

I got the error when write glue dynamic data frame to redshift table .
The code I have used :-
my_conn_op={
    "dbtable":"public.xxxx",
    "database":"dev"
}

redshift_results=glueContext.write_dynamic_frame.from_jdbc_conf(
    frame=dyn1,
    catalog_connection="Redshift connection",
    connection_options=my_conn_op,
    redshift_tmp_dir="s3://XXXXXXXXXX/temp_redshift/"
    )

The error I got

Error Category: UNCLASSIFIED_ERROR; An error occurred while calling o99.getJDBCSink. The connection attempt failed.

Elena Madrazo · Answer

The error you're encountering is likely due to a connection issue between AWS Glue and your Amazon Redshift cluster. Based on the error you posted, here are a few things you can check and try to resolve the issue:

* Redshift Cluster Connectivity:Ensure that your Redshift cluster is accessible from the AWS Glue job's VPC and subnet. The Glue job and Redshift cluster should be in the same VPC or have a VPC peering connection established.
Check if the Redshift cluster has a publicly accessible endpoint or if it's configured for enhanced VPC routing. If not, you may need to set up a Glue job bookmark or use a Glue job in a private subnet with a NAT gateway.

* IAM Permissions: Verify that the IAM role used by your Glue job has the necessary permissions to access the Redshift cluster. The role should have the redshift:GetClusterCredentials and redshift:DescribeClusters permissions.
If you're using a Redshift cluster in a different AWS account, ensure that the IAM role has the sts:AssumeRole permission and the appropriate trust relationship is set up.

* Redshift Cluster Configuration: Check if the Redshift cluster has a security group or VPC security group that allows inbound connections from the Glue job's VPC and subnet.

* Connection Options: Double-check the connection options you're passing to the write_dynamic_frame.from_jdbc_conf method. Make sure the dbtable and database values are correct.
Try using the fully qualified table name in the dbtable option (e.g. "dbtable":"dev.public.xxxx").

* Glue Job Logs: Check the Glue job logs for any additional error messages or clues that could help diagnose the issue.

Lastly, please find below further links to help you with your error:
https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-connect-redshift-home.html
https://docs.aws.amazon.com/glue/latest/dg/setup-vpc-for-glue-access.html

Error Category: UNCLASSIFIED_ERROR; An error occurred while calling o99.getJDBCSink. The connection attempt failed.

Add your answer

Relevant content