跳至內容

Table or view already exists. SaveMode: ErrorIfExists while writing the data to postgress table using glue spark

0

Hi Team,

I'm Getting the below error while writing the table in postgres using glue spark script can you please help me on this issue.

"Table or view "asset_aircraft_rpt" already exists. SaveMode: ErrorIfExists.,"

Using the below code: trf_df.write.jdbc(url=jdbc_url, table='table_name', properties=connection_properties)

if we use the overwrite mode, the table is deleting and creating again. we want to load the data to postgress table form the glue scripts without deleting th table. please share the suggestions for the issue.

已提問 1 年前檢視次數 148 次
1 個回答
0

To resolve the "Table or view already exists. SaveMode: ErrorIfExists" error when writing data to a PostgreSQL table using AWS Glue Spark, you need to specify the write mode as "append" instead of the default "ErrorIfExists" mode.

You can modify your code to:

trf_df.write.jdbc(url=jdbc_url, table='table_name', mode="append", properties=connection_properties)

The available write modes in Spark are:

  • "append": Adds the data to the existing table without modifying the existing data
  • "overwrite": Deletes the existing table and creates a new one (which you mentioned you don't want)
  • "errorIfExists": Throws an error if the table exists (this is the default mode causing your error)
  • "ignore": Silently ignores the operation if the table exists

Since you want to keep adding new data to your existing PostgreSQL table without deleting it, the "append" mode is the appropriate choice for your use case.
Sources
how to update database with glue etl job? | AWS re:Post

已回答 1 年前
AWS
支援工程師
已審閱 1 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。