Unable to connect AWS RDS to Glue using custom JDBC driver

0

Hi Folks, I have a postgres RDS connection that I wish to connect with my AWS Glue. The version is PostgreSQL 12.14 on x86_64-pc-linux-gnu. I require to use a custom JDBC driver which I have uploaded to my S3 bucket whose version is postgresql-42.2.18.jar. While running crawler, I encounter "ERROR : Internal Service Exception".

Could it be because of incorrect VPC setting or be an incorrect JDBC driver issue? Do mention if you require any further specificity.

Many in advance.

1 Answer
0

As you are using JDBC as a source for crawler, firstly Test the connection and see if its working fine. [+] https://docs.aws.amazon.com/glue/latest/dg/console-test-connections.html

Crawler internal service exceptions can be caused by transient issues. Before you start to troubleshoot, run the crawler again. If you still get an internal service exception, then check for the following common issues. [+] https://repost.aws/knowledge-center/glue-crawler-internal-service-exception

Validate that you comply with the following requirements:

If you're crawling a JDBC data source that's encrypted with AWS KMS, then check the subnet that you're using for the connection. The subnet's route table must have a route to the AWS KMS endpoint. This route can go through an AWS KMS supported virtual private cloud (VPC) endpoint or a NAT gateway.

Be sure that you're using the correct Include path syntax. For more information, see Defining crawlers. (https://docs.aws.amazon.com/glue/latest/dg/add-crawler.html)

If you're crawling a JDBC data store, then confirm that the SSL connection is configured correctly. If you're not using an SSL connection, then be sure that Require SSL connection isn't selected when you configure the crawler.

Confirm that the database name in the AWS Glue connection matches the database name in the crawler's Include path. Also, be sure that you enter the Include path correctly. For more information, see Include and exclude patterns. (https://docs.aws.amazon.com/glue/latest/dg/define-crawler.html#crawler-data-stores-exclude) 

Be sure that the subnet that you're using is in an Availability Zone that's supported by AWS Glue.

Be sure that the subnet that you're using has enough available private IP addresses.

Confirm that the JDBC data source is supported with the built-in AWS Glue JDBC driver. (https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-connect.html#aws-glue-programming-etl-connect-jdbc)
AWS
SUPPORT ENGINEER
answered 5 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions