Glue container does not recognize mysql driver

0

I need to read a dynamic frame from catalog of a table from mysql RDS. I tryed many ways to install mysql driver, but I had no success. Anyone can help me?

The ways I allready tryed were: method 1: %%configure -f { "conf": { "spark.jars.packages": "mysql:mysql-connector-java:8.0.29" } }

method 2: import pymysql pymysql.install_as_MySQLdb()

method 3: conf = spark_conf.set("spark.driver.extraClassPath","path to ")
.set("spark.jars", "path to .jar") sc = SparkContext(conf = conf).getOrCreate()

demandé il y a 7 mois171 vues
1 réponse
1

First, you shouldn't need to bring you own MySQL driver, Glue provides one out of the box and in Glue 4 is quite recent.
What do you mean "does not recognized", if the JDBC url is correct it should pick it up automatically.
spark.jar.packages is not the way to add jars in Glue, you use instead --extra-jars with an s3 path.
Using the Python library is installed in a different way and won't run distributed, better don't use that for querying.

profile pictureAWS
EXPERT
répondu il y a 7 mois

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions