跳至内容

Table or view already exists. SaveMode: ErrorIfExists while writing the data to postgress table using glue spark

0

Hi Team,

I'm Getting the below error while writing the table in postgres using glue spark script can you please help me on this issue.

"Table or view "asset_aircraft_rpt" already exists. SaveMode: ErrorIfExists.,"

Using the below code: trf_df.write.jdbc(url=jdbc_url, table='table_name', properties=connection_properties)

if we use the overwrite mode, the table is deleting and creating again. we want to load the data to postgress table form the glue scripts without deleting th table. please share the suggestions for the issue.

已提问 1 年前147 查看次数
1 回答
0

To resolve the "Table or view already exists. SaveMode: ErrorIfExists" error when writing data to a PostgreSQL table using AWS Glue Spark, you need to specify the write mode as "append" instead of the default "ErrorIfExists" mode.

You can modify your code to:

trf_df.write.jdbc(url=jdbc_url, table='table_name', mode="append", properties=connection_properties)

The available write modes in Spark are:

  • "append": Adds the data to the existing table without modifying the existing data
  • "overwrite": Deletes the existing table and creates a new one (which you mentioned you don't want)
  • "errorIfExists": Throws an error if the table exists (this is the default mode causing your error)
  • "ignore": Silently ignores the operation if the table exists

Since you want to keep adding new data to your existing PostgreSQL table without deleting it, the "append" mode is the appropriate choice for your use case.
Sources
how to update database with glue etl job? | AWS re:Post

已回答 1 年前
AWS
支持工程师
已审核 10 个月前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。