Working only new data in aws glue


Hi everyone,
I have a math need to solve. I have new data importing to s3 everyday, Right now I have to run all data include old one everyday which is costly but the thing is I only need to convert the new data. Can anyone know how to just only run new data in ETL job in Glue? And how to Glue know and skip all the things it done and only run the new things - which haven't done yet

I appreciate all your comment

Edited by: phithang711 on Jan 29, 2019 2:25 PM

asked 5 years ago213 views
1 Answer

discover the job bookmark in aws glue. Hope it helps me. Close this topic right here

answered 5 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions