By using AWS re:Post, you agree to the Terms of Use

glue connection not working to crawl DB table

0

Hi team,

I have 2 glue crawlers :

  • 1 in dev account that crawls database tables from aurora serverless MYSQL
  • 1 in stg account that crawls database tables from RDS aurora MYSQL 8 version: AuroraMysqlEngineVersion.of("8.0.mysql_aurora.3.01.0", "8.0")

I created a glue connection to connect to my RDS aurora MySQL and crawl my DB tables. The connection in dev (serverless aurora) works fine and tables are crawled, but it doesn't work in stg (MYSQL 8) :

Check that your connection definition references your JDBC database with correct URL syntax, username, and password. Could not create connection to database server.
10:34
Exiting with error code 30

and when I run the crawler it fails telling that: fails to connect to the database

when creating the connection in stg I tried both types : JDBC and amazon Mysql Aurora non of them want connect to the DB

my JDBC URL looks like this in both accounts dev and stg :

mysql://databasestack-databaseb2sdgfbb-128ewdfsdsi3ew.cluster-csdggdsf1qdsdgdfc.region.rds.amazonaws.com:3306/myaccountdb?verifyServerCertificate=false&useSSL=true

not sure what is the issue here in stg account, how can I make my crawler/connection work two be able to crawl my tables on the database ?

Thank you!!!

2 Answers
0

Hello,

Unfortunately Glue connections to MySQL 8.0 are currently not supported. As of now, the driver we use in Glue does not support MySQL 8. Please refer this link as to know what JDBC drivers are supported by Glue - https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-connect.html#aws-glue-programming-etl-connect-jdbc

There is currently no way to specify your own JDBC driver for crawlers. However, there's a workaround available with Glue ETL jobs, where you can use your own JDBC drivers by uploading a MySQL 8 compatible driver to S3 - https://aws.amazon.com/about-aws/whats-new/2019/11/aws-glue-now-enables-you-to-bring-your-own-jdbc-drivers-to-your-glue-spark-etl-jobs/

Please open a support ticket with Glue team for any further questions and for us to troubleshoot the issue further.

answered 6 months ago
EXPERT
reviewed 6 months ago
0

Additionally Glue 3.0 ETL job has Mysql 8 driver inbuilt in environment , So using AWS Glue 3.0 version you can read and write data to MYSQL 8 directly without providing any driver manually.

Appendix B: JDBC driver upgrades : https://docs.aws.amazon.com/glue/latest/dg/migrating-version-30.html

answered 6 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions