Unable to connect AWS RDS to Glue using custom JDBC driver

0

Hi Folks, I have a postgres RDS connection that I wish to connect with my AWS Glue. The version is PostgreSQL 12.14 on x86_64-pc-linux-gnu. I require to use a custom JDBC driver which I have uploaded to my S3 bucket whose version is postgresql-42.2.18.jar. While running crawler, I encounter "ERROR : Internal Service Exception".

Could it be because of incorrect VPC setting or be an incorrect JDBC driver issue? Do mention if you require any further specificity.

Many in advance.

rshah
已提问 5 个月前297 查看次数
1 回答
0

As you are using JDBC as a source for crawler, firstly Test the connection and see if its working fine. [+] https://docs.aws.amazon.com/glue/latest/dg/console-test-connections.html

Crawler internal service exceptions can be caused by transient issues. Before you start to troubleshoot, run the crawler again. If you still get an internal service exception, then check for the following common issues. [+] https://repost.aws/knowledge-center/glue-crawler-internal-service-exception

Validate that you comply with the following requirements:

If you're crawling a JDBC data source that's encrypted with AWS KMS, then check the subnet that you're using for the connection. The subnet's route table must have a route to the AWS KMS endpoint. This route can go through an AWS KMS supported virtual private cloud (VPC) endpoint or a NAT gateway.

Be sure that you're using the correct Include path syntax. For more information, see Defining crawlers. (https://docs.aws.amazon.com/glue/latest/dg/add-crawler.html)

If you're crawling a JDBC data store, then confirm that the SSL connection is configured correctly. If you're not using an SSL connection, then be sure that Require SSL connection isn't selected when you configure the crawler.

Confirm that the database name in the AWS Glue connection matches the database name in the crawler's Include path. Also, be sure that you enter the Include path correctly. For more information, see Include and exclude patterns. (https://docs.aws.amazon.com/glue/latest/dg/define-crawler.html#crawler-data-stores-exclude) 

Be sure that the subnet that you're using is in an Availability Zone that's supported by AWS Glue.

Be sure that the subnet that you're using has enough available private IP addresses.

Confirm that the JDBC data source is supported with the built-in AWS Glue JDBC driver. (https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-connect.html#aws-glue-programming-etl-connect-jdbc)
AWS
支持工程师
已回答 5 个月前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则