Connect to RDS postgres database in GLUE script
I would like to connect to the RDS postgres database with Glue script written in scala, I have a postgres driver located in S3 and added in script in Dependent JARs path. After running the script it fails with the message Glue ETL Marketplace - Retrieved no ETL connector jars, this may be due to no marketplace/custom connection attached to the job or failure of downloading them, please scroll back to the previous logs to find out the root cause. I cannot find any related logs helping me to understand why there is a problem and I receive a Compilation error.
Hi,
any specific reason why you are not using the native Glue connection which supports RDS PostgreSQL?
Glue 2.0 uses the driver 42.1.0 and Glue 3.0 uses 42.2.18 (see in this section of the documentation or in this other one).
If you really need to bring your own connector, have you followed the documentation or this blog post as an example?
can you share an anonymised version of the code snippet that contains the setup for the customJDBCDriverJsonOptions ?
hope this helps
Relevant questions
Custom script cannot be saved in Glue
asked 9 days agoCannot connect to MongoDB tables in catalog
asked 4 months agoglue connection not working to crawl DB table
asked 2 months agoException in User Class: org.postgresql.util.PSQLException : ERROR: column "id" does not exist
asked 4 months agoQuickSight Unable to Connect to RDS(postgres)
asked a month agoFATAL: password authentication failed for user "postgres
asked 4 years agoConnect to RDS postgres database in GLUE script
asked a month ago60K updates to RDS Postgres - performance challenge
Accepted Answerasked 2 years agoHeterogeneous database links on AWS RDS
Accepted Answerasked a year agoUse RDS Postgres Replicas as a cluster
Accepted Answerasked 4 months ago
What I want to achieve is this: https://repost.aws/questions/QUQE0EE8VmQuau0uRhSzN0gw#ANTI7-sqAZSsqUJvy1Borvzw
I am following the Medium article that was suggested to me in the responses.