best practice to move ETL reated files

0

Hi Team,

I have a glue job that reads CSV files from S3 and injects data to Mysql RDS,

I want to move all files from s3 to glacier once they're treated by ETL glue.

I would like to know what is the best architecture to move CSV files from s3 to glacier upon glue job success?

is it better to do it from within the glue script at the end of the script?

or

to do it through another process?

also, any code example will be helpful :)

Thank you :)

1 Respuesta
0
  1. If you have a table created in Glue catalog and to directly transition the storage class of files related to this table, you can use transition_table API from within the Glue job.
  2. If you do not have a table created, you can use transition_s3_path API to directly transition the storage class of files by specifying an s3 path from within the Glue job.
AWS
INGENIERO DE SOPORTE
Manu_G
respondido hace 2 años
AWS
EXPERTO
revisado hace 2 años

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas