Deleting insert and updates from DMS cdc parquet folder

0

I have a DMS job that writes records from postgres to s3 folder in parquet format. I want, for compliance reasons, to delete records from the parquet file when the records are deleted from source. I know the delete (cdc) row will be null for all columns apart from primary keys but I need the previous insert and updates to be deleted from the parquet files that DMS writes to s3.

1 Answer
0

Hi,

As I understand, you would like to remove the insert and update records from the parquet files if there is a delete in the source.

Currently, this is not possible with AWS Database Migration Service (DMS). Once a record (insert operation or update operation) is written to the parquet file in S3 bucket, DMS cannot go back and update the parquet file and remove the records.

AWS
SUPPORT ENGINEER
answered 19 days ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions