By using AWS re:Post, you agree to the Terms of Use
AWS re:Postre:Post

AWS Glue visual job

0

Hi I was trying to work on a simple problem where I am taking data from 3 csv files in s3(source) and after combining them, I am appending them to a table in postgre sql where my database is by establishing a connection in my glue catalog with that database. But I am facing an error stating that this table already exist and the data is not getting attached to it. Kindly help me if there is anything wrong with this approach

Topics
AnalyticsDatabaseStorage
Tags
AWS Glue DataBrewDatabaseAWS GlueData LakesExtract Transform & Load Data
Language
English
AWS-User-6526532
asked a year ago137 views
1 Answer
0

Hello,

There is not any issue in above approach. You can read the data from multiple sources and can perform transformation like join and then write to the existing relational database table without any issue.

By default Glue dynamic frame use append mode and add the data to an existing table. Make sure that all the columns should be present in the database table before as job would not be able to add new column and can fail with column not found in schema.

I would suggest please verify the schema of your data after join operation with the database table schema.

If you still face any issue, we require your sample data and script. Hence Please feel free to reach out to AWS Premium Support with sample data and script and we will be happy to help.

AWS
Shubham_P
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content