Skip to content
By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post
About re:Post

DMS Restart removes the delete markers. - how to handle in GLUE

0

Our Transaction application has a hard delete implemented on MySQL DB on record inactivation scenarios . For the lakehouse , The RDS CDC is exported to S3 through DMS . The problem is , if the DMS is restarted for some reason , the Delete marker of the records get erased , causing orphan records and duplicates in the datalake.

What is the best way to handle this scenario ? Should the glue job transforming data from S3 to the datalake handle such data ? Like a source to target comparison and discard records that are present in the Datalake but not in Source ?

Topics
AnalyticsMigration & ModernizationDatabase
Tags
AWS Database Migration ServiceAWS GlueExtract Transform & Load Data
Language
English
asked 3 years ago517 views
1 Answer
0

To answer your question, we require details that are non-public information. Please open a support case with AWS using the following link:

https://support.console.aws.amazon.com/support/home?#/case/create

AWS
SUPPORT ENGINEER
answered 3 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Relevant content