AWS Glue visual job
Hi I was trying to work on a simple problem where I am taking data from 3 csv files in s3(source) and after combining them, I am appending them to a table in postgre sql where my database is by establishing a connection in my glue catalog with that database. But I am facing an error stating that this table already exist and the data is not getting attached to it. Kindly help me if there is anything wrong with this approach
Hello,
There is not any issue in above approach. You can read the data from multiple sources and can perform transformation like join and then write to the existing relational database table without any issue.
By default Glue dynamic frame use append mode and add the data to an existing table. Make sure that all the columns should be present in the database table before as job would not be able to add new column and can fail with column not found in schema.
I would suggest please verify the schema of your data after join operation with the database table schema.
If you still face any issue, we require your sample data and script. Hence Please feel free to reach out to AWS Premium Support with sample data and script and we will be happy to help.
Relevant questions
AWS Glue visual job
asked 2 months agoCan't get Partitions to work with my Glue Data Catalog
Accepted Answerasked 2 months agoHow can i run SQL statements in my AWS Glue Script
asked 2 months agoHow could we have Glue to get data from csv as String?
Accepted Answerasked 2 months agoCan AWS Glue read data from different SQL Server table, generate csv files and zipping it to S3?
Accepted Answerasked 4 months agoGlue job s3 file not found exception
asked 5 years agobackslash in CSV with glue
asked 6 months agoAWS glue combining multiple input into a single output csv
asked 5 months agoGlue Job Spark SQL Issue
asked 3 months agoI am trying to write an ETL job to the Data Catalog but its writing the Headers as Data
Accepted Answerasked 4 months ago