glue connection not working to crawl DB table

0

Hi team,

I have 2 glue crawlers :

  • 1 in dev account that crawls database tables from aurora serverless MYSQL
  • 1 in stg account that crawls database tables from RDS aurora MYSQL 8 version: AuroraMysqlEngineVersion.of("8.0.mysql_aurora.3.01.0", "8.0")

I created a glue connection to connect to my RDS aurora MySQL and crawl my DB tables. The connection in dev (serverless aurora) works fine and tables are crawled, but it doesn't work in stg (MYSQL 8) :

Check that your connection definition references your JDBC database with correct URL syntax, username, and password. Could not create connection to database server.
10:34
Exiting with error code 30

and when I run the crawler it fails telling that: fails to connect to the database

when creating the connection in stg I tried both types : JDBC and amazon Mysql Aurora non of them want connect to the DB

my JDBC URL looks like this in both accounts dev and stg :

mysql://databasestack-databaseb2sdgfbb-128ewdfsdsi3ew.cluster-csdggdsf1qdsdgdfc.region.rds.amazonaws.com:3306/myaccountdb?verifyServerCertificate=false&useSSL=true

not sure what is the issue here in stg account, how can I make my crawler/connection work two be able to crawl my tables on the database ?

Thank you!!!

2 個答案
0

Hello,

Unfortunately Glue connections to MySQL 8.0 are currently not supported. As of now, the driver we use in Glue does not support MySQL 8. Please refer this link as to know what JDBC drivers are supported by Glue - https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-connect.html#aws-glue-programming-etl-connect-jdbc

There is currently no way to specify your own JDBC driver for crawlers. However, there's a workaround available with Glue ETL jobs, where you can use your own JDBC drivers by uploading a MySQL 8 compatible driver to S3 - https://aws.amazon.com/about-aws/whats-new/2019/11/aws-glue-now-enables-you-to-bring-your-own-jdbc-drivers-to-your-glue-spark-etl-jobs/

Please open a support ticket with Glue team for any further questions and for us to troubleshoot the issue further.

AWS
已回答 2 年前
AWS
專家
已審閱 2 年前
0

Additionally Glue 3.0 ETL job has Mysql 8 driver inbuilt in environment , So using AWS Glue 3.0 version you can read and write data to MYSQL 8 directly without providing any driver manually.

Appendix B: JDBC driver upgrades : https://docs.aws.amazon.com/glue/latest/dg/migrating-version-30.html

AWS
已回答 2 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南