glue connection not working to crawl DB table

0

Hi team,

I have 2 glue crawlers :

  • 1 in dev account that crawls database tables from aurora serverless MYSQL
  • 1 in stg account that crawls database tables from RDS aurora MYSQL 8 version: AuroraMysqlEngineVersion.of("8.0.mysql_aurora.3.01.0", "8.0")

I created a glue connection to connect to my RDS aurora MySQL and crawl my DB tables. The connection in dev (serverless aurora) works fine and tables are crawled, but it doesn't work in stg (MYSQL 8) :

Check that your connection definition references your JDBC database with correct URL syntax, username, and password. Could not create connection to database server.
10:34
Exiting with error code 30

and when I run the crawler it fails telling that: fails to connect to the database

when creating the connection in stg I tried both types : JDBC and amazon Mysql Aurora non of them want connect to the DB

my JDBC URL looks like this in both accounts dev and stg :

mysql://databasestack-databaseb2sdgfbb-128ewdfsdsi3ew.cluster-csdggdsf1qdsdgdfc.region.rds.amazonaws.com:3306/myaccountdb?verifyServerCertificate=false&useSSL=true

not sure what is the issue here in stg account, how can I make my crawler/connection work two be able to crawl my tables on the database ?

Thank you!!!

2 回答
0

Hello,

Unfortunately Glue connections to MySQL 8.0 are currently not supported. As of now, the driver we use in Glue does not support MySQL 8. Please refer this link as to know what JDBC drivers are supported by Glue - https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-connect.html#aws-glue-programming-etl-connect-jdbc

There is currently no way to specify your own JDBC driver for crawlers. However, there's a workaround available with Glue ETL jobs, where you can use your own JDBC drivers by uploading a MySQL 8 compatible driver to S3 - https://aws.amazon.com/about-aws/whats-new/2019/11/aws-glue-now-enables-you-to-bring-your-own-jdbc-drivers-to-your-glue-spark-etl-jobs/

Please open a support ticket with Glue team for any further questions and for us to troubleshoot the issue further.

AWS
已回答 2 年前
AWS
专家
已审核 2 年前
0

Additionally Glue 3.0 ETL job has Mysql 8 driver inbuilt in environment , So using AWS Glue 3.0 version you can read and write data to MYSQL 8 directly without providing any driver manually.

Appendix B: JDBC driver upgrades : https://docs.aws.amazon.com/glue/latest/dg/migrating-version-30.html

AWS
已回答 2 年前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则