By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Upsert into MySQL target using AWS Glue

0

Is there a provision to upsert into MySQL database?

Below post on related topic mentioned an "Upsert box" in Glue Studio, which I am not able to find - https://repost.aws/questions/QUNfyJnyqIRbabAa_Drw_ugw/aws-glue-upsert-operation-for-few-columns

Thanks

Topics
AnalyticsDatabase
Tags
AWS GlueExtract Transform & Load Data
Language
English
rePost-User-0629761
asked 2 years ago1287 views
1 Answer
0
Accepted Answer

Hi There

Upsert is available for Redshift targets not MySQL

You will need to have custom logic to do other operations besides Inserts. Check out this article for one way of doing updates on MySQL using pymysql. https://awstip.com/aws-etl-glue-job-insert-update-support-7a396db832b

profile pictureAWS
EXPERT
Matt-B
answered 2 years ago
profile picture
EXPERT
Sedat Salman
reviewed 10 months ago
AWS
EXPERT
Fabrizio@AWS
reviewed 2 years ago
  • rePost-User-0629761
    2 years ago

    I tried the solution proposed in the article. However, it looks like the job is timing out on below line - connection = glue_client.get_connection(Name="<My Connection>")

    I do not see any exception in the logs.

    The same connection worked when I used it in another job for insert-only, created from Visual editor. Also, test connection works fine.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content