best practice to move ETL reated files

0

Hi Team,

I have a glue job that reads CSV files from S3 and injects data to Mysql RDS,

I want to move all files from s3 to glacier once they're treated by ETL glue.

I would like to know what is the best architecture to move CSV files from s3 to glacier upon glue job success?

is it better to do it from within the glue script at the end of the script?

or

to do it through another process?

also, any code example will be helpful :)

Thank you :)

1 réponse
0
  1. If you have a table created in Glue catalog and to directly transition the storage class of files related to this table, you can use transition_table API from within the Glue job.
  2. If you do not have a table created, you can use transition_s3_path API to directly transition the storage class of files by specifying an s3 path from within the Glue job.
AWS
INGÉNIEUR EN ASSISTANCE TECHNIQUE
Manu_G
répondu il y a 2 ans
AWS
EXPERT
vérifié il y a 2 ans

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions