[AWS Glue] Insert-Update to Postgre Table

0

Hi folks!

I want to ask can we do insert-update in AWS Glue Visual ETL Job? I want to use Postgre as DB target and will do insert-update.

Prefers use Visual ETL Job...

Ivan
gefragt vor 4 Monaten371 Aufrufe
1 Antwort
0

Not out of the box at this time, the way to do it is a custom code node that writes into a Postgres temporary table and then issue a SQL command to do the upsert/merge into the final table (using JDBC, the psycopg2 library or similar).

profile pictureAWS
EXPERTE
beantwortet vor 4 Monaten
  • can we use 'Custom Transform' to make it? or do i have use in seperate to do it?

  • Yes you can, despite the name "transform" you can put any code, including writing

  • sorry sir, i'm new at here. I think I get your idea, but I want to confirm a few things. This means that after I perform transformations on my data, I have to create a script to perform an upsert query into my table. My next question is whether I can set the connection/target table directly from the script? this is a simple flow image that I have created, and I want to make sure that we line up this 'custom transform' can be used for the script code requirements above.

    https://ibb.co/C64kp2q

  • Your custom transform node has to be part of the graph, it should be the last node and you do the writing/updating there. That's it

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen