best practice to move ETL reated files

0

Hi Team,

I have a glue job that reads CSV files from S3 and injects data to Mysql RDS,

I want to move all files from s3 to glacier once they're treated by ETL glue.

I would like to know what is the best architecture to move CSV files from s3 to glacier upon glue job success?

is it better to do it from within the glue script at the end of the script?

or

to do it through another process?

also, any code example will be helpful :)

Thank you :)

1 個回答
0
  1. If you have a table created in Glue catalog and to directly transition the storage class of files related to this table, you can use transition_table API from within the Glue job.
  2. If you do not have a table created, you can use transition_s3_path API to directly transition the storage class of files by specifying an s3 path from within the Glue job.
AWS
支援工程師
Manu_G
已回答 2 年前
AWS
專家
已審閱 2 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南