By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Glue ETL: Set output files to replaces existing files in source

0

Hello, I have recently had to move one of our Jobs from Glue DataBrew to Glue ETL - which is now up and running.

The issue I am having is that everytime the job runs it creates a new series of files. Can we set it so it overwrites the existing files in the S3 bucket source?

In Glue Databrew this option is in settings, "Replace output files for each job run" - but I cannot see it clearly in ETL, can anyone advise how to turn this on?

Thanks, Rob

Topics
Analytics
Tags
AWS Glue DataBrewAWS Glue
Language
English
Robert_Arken
asked 7 months ago253 views
1 Answer
0
Accepted Answer

I managed to achieve this by creating a Lifetime Cycle Rule in my S3 Bucket

Robert_Arken
answered 7 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content