DMS CDC Verification

Question

My customer is migrating from SQL Server (installed on EC2) to Aurora MYSQL. DB size is about 500GB and there is high amount of activity at any given point in time. Source db should be up while migrating. CDC (change data cut over) is the approach planned.

Looking for guidance on how to go about verifying if migration between source and target is good. Client wants DMS to conduct verification upon initial cut over and the subsequent CDC increments. I hear this is not possible as new data keeps getting added to the source db and we cannot store data in the migration instance for verification.

Also, if you can point me to documentation that outlines best practices with CDC scenarios, that will be great.

I looked at blogs and documents but none covers best practices for verification in detail.

VijayK · Accepted Answer

[Validating AWS DMS Tasks][1]: AWS DMS provides support for data validation, to ensure that your data was migrated accurately from the source to the target. If you enable it for a task, then AWS DMS begins comparing the source and target data immediately after a full load is performed for a table.

Data validation is optional. AWS DMS compares the source and target records, and reports any mismatches. In addition, for a CDC-enabled task, AWS DMS compares the incremental changes and reports any mismatches.

During data validation, AWS DMS compares each row in the source with its corresponding row at the target, and verifies that those rows contain the same data. To accomplish this, AWS DMS issues appropriate queries to retrieve the data. Note that these queries will consume additional resources at the source and the target, as well as additional network resources.

[1]: https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Validating.html

DMS CDC Verification

Relevant content