False positives with AWS DMS validations.

Question

Hi,
I am using DMS to move data (5.200.000 records) between an RDS SQL Server Express database and an RDS MySQL database. 
The number of records transferred is perfect : every record count per table matches exactly.

However when running the validation step I get a lot of mismatched records that I believe is an issue in the DMS validation process at the driver level (charset configuration probably):

RECORD_DIFF	[[{'Title': 'سلام ژوند څنګه دی (1) 😀🥳'}, {'Title': 'سلام ژوند څنګه دی (1) ??'}],]

If I do the select query directly in the MySQL database with MySQL Workbench I get exactly the same emojis as I the DMS validation tool gets from the SQL Server.

Besides this I am also getting a lot of RECORD_DIFF	[[{'Content': ''}, {'Content': ''}],] which I am not sure what they mean querying both database I see the values perfectly for those rows.

Any help will be greatly appreciated.

Thanks,
Claudio.

Answer

I have faced a similar issue before. You can raise a support ticket with AWS to help address the issue.

Meanwhile, here are some options. 
1. Remove the table from your validation list. You can verify it using SQL queries and hash function later. 
2. If you know the reason for the mismatch, you can adjust for it using [modified validation rules](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Validating.html) (see section titled *Using JSON editor to modify validation rules*). 
3. If the mismatch is due to charset, you can adjust it using [Character substitution task settings](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TaskSettings.CharacterSubstitution.html)
4. You can use a combination of the 1,2, and 3. For example, you can validate the length of the column using validation settings but then validate the content later using hash function.

Please let us know how you handled it.

False positives with AWS DMS validations.

Relevant content