Question about Replication tasks Target table preparation mode options.

0

Exists a replication set up in which an identical table from N Source databases is being replicated into a single larger table at the Target. Each source database has a replication task set as full-load-and-cdc and inside that task the data is pushed into one larger table at the target endpoint with a database identifier field appended to identify the source database.

If the Tasks' Target table preparation mode options is set to Truncate, is there a way to override the functionality to only remove records based on the partition of data based on a sources database identity keys or some other combination, trick?

asked 22 days ago27 views
1 Answer
1

Dear AWS Customer,

Thank you for your query. Hope you are doing good.

After going through the post, I understand that you have a scenario wherein you have N number of source databases and single table at the target database. Each source database has a replication task set as "Full Load + CDC" to pull data from source and push to single target table (on the target) with database identifier as the additional field to identify from which source the data came from.

When the Target table preparation mode options is set to Truncate, you would like to know if there is a way to override the functionality to only remove records based on the partition of data based on a sources database identifier.

Please correct me if I misunderstood your query.

Please allow me to answer your query:

As of today, AWS DMS have only below 3 options for "Target table preparation mode".

  1. Do nothing
  2. Drop tables on target
  3. Truncate

Each option have been programmed with the corresponding action. For example, if you opt for "Target table preparation mode" to "Drop tables on target" and if AWS DMS find tables already present in target database then AWS DMS is designed to drop the tables, and creates new tables in their place and load the data.

Similarly for "Target table preparation mode" to "Truncate" as soon as the AWS DMS find the tables in the target database, AWS DMS will truncate the tables (AWS DMS leaves the tables and their metadata in place, but removes the data from them) and load the data. Refer link below.

[+] https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TaskSettings.FullLoad.html

So as of today AWS DMS doesn't have such functionality that you have mentioned.

Being said that, the new features release updates are available regularly on the official AWS Updates sources, Please be on the lookout using the links attached below for your reference.

[1] RDS What's New - https://aws.amazon.com/new/

[2] AWS Database Blog - https://aws.amazon.com/blogs/database/

[3] AWS Blog - https://aws.amazon.com/blogs/aws/

I hope that above information is useful. Thanks.

answered 21 days ago
  • Thank you for the detailed reply. It has proved very useful.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions