Glue container does not recognize mysql driver

0

I need to read a dynamic frame from catalog of a table from mysql RDS. I tryed many ways to install mysql driver, but I had no success. Anyone can help me?

The ways I allready tryed were: method 1: %%configure -f { "conf": { "spark.jars.packages": "mysql:mysql-connector-java:8.0.29" } }

method 2: import pymysql pymysql.install_as_MySQLdb()

method 3: conf = spark_conf.set("spark.driver.extraClassPath","path to ")
.set("spark.jars", "path to .jar") sc = SparkContext(conf = conf).getOrCreate()

asked 7 months ago166 views
1 Answer
1

First, you shouldn't need to bring you own MySQL driver, Glue provides one out of the box and in Glue 4 is quite recent.
What do you mean "does not recognized", if the JDBC url is correct it should pick it up automatically.
spark.jar.packages is not the way to add jars in Glue, you use instead --extra-jars with an s3 path.
Using the Python library is installed in a different way and won't run distributed, better don't use that for querying.

profile pictureAWS
EXPERT
answered 6 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions