Failing to run AWS Glue workers with AWS RDS PostgresSQL14.7

0

I am trying to load data from my AWS RDS PostgreSQL v14.7 instance hosted on ECS, and I kept getting the following error whenever I test my connection in the AWS Glue connectors:

Check that your connection definition references your JDBC database with correct URL syntax, username, and password. Full SSL connection to data store using host matching failed. The authentication type 10 is not supported. Check that you have configured the pg_hba.conf file to include the client's IP address or subnet, and that it is using an authentication scheme supported by the driver.

I then uploaded a JDBC driver v42.6.0 on a dedicated S3 bucket and specified the path in the crawler connection as the following (I cannot afford to downgrade the Postgres instance version): Enter image description here

Now the test connection is giving a non-supported error for custom uploaded JDBC drivers, so I immediately tried to run my crawlers on the connection. No matter the size of specified schema, the crawler takes 15 minutes and then fails with the following error message in CloudWatch (I would appreciate it if I can find more informative error messages in some other way): ERROR : Internal Service Exception

My database/table/column names are all snake_case. I confirmed that the data source is within the same VPC. I configured an unresitrcted outbound security group rule from the RDS instance, and yet nothing has worked out for me

asked a year ago67 views
No Answers

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions