Glue container does not recognize mysql driver

0

I need to read a dynamic frame from catalog of a table from mysql RDS. I tryed many ways to install mysql driver, but I had no success. Anyone can help me?

The ways I allready tryed were: method 1: %%configure -f { "conf": { "spark.jars.packages": "mysql:mysql-connector-java:8.0.29" } }

method 2: import pymysql pymysql.install_as_MySQLdb()

method 3: conf = spark_conf.set("spark.driver.extraClassPath","path to ")
.set("spark.jars", "path to .jar") sc = SparkContext(conf = conf).getOrCreate()

gefragt vor 7 Monaten171 Aufrufe
1 Antwort
1

First, you shouldn't need to bring you own MySQL driver, Glue provides one out of the box and in Glue 4 is quite recent.
What do you mean "does not recognized", if the JDBC url is correct it should pick it up automatically.
spark.jar.packages is not the way to add jars in Glue, you use instead --extra-jars with an s3 path.
Using the Python library is installed in a different way and won't run distributed, better don't use that for querying.

profile pictureAWS
EXPERTE
beantwortet vor 7 Monaten

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen