best practice to move ETL reated files

0

Hi Team,

I have a glue job that reads CSV files from S3 and injects data to Mysql RDS,

I want to move all files from s3 to glacier once they're treated by ETL glue.

I would like to know what is the best architecture to move CSV files from s3 to glacier upon glue job success?

is it better to do it from within the glue script at the end of the script?

or

to do it through another process?

also, any code example will be helpful :)

Thank you :)

1개 답변
0
  1. If you have a table created in Glue catalog and to directly transition the storage class of files related to this table, you can use transition_table API from within the Glue job.
  2. If you do not have a table created, you can use transition_s3_path API to directly transition the storage class of files by specifying an s3 path from within the Glue job.
AWS
지원 엔지니어
Manu_G
답변함 2년 전
AWS
전문가
검토됨 2년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인