False positives with AWS DMS validations.
Hi, I am using DMS to move data (5.200.000 records) between an RDS SQL Server Express database and an RDS MySQL database. The number of records transferred is perfect : every record count per table matches exactly.
However when running the validation step I get a lot of mismatched records that I believe is an issue in the DMS validation process at the driver level (charset configuration probably):
RECORD_DIFF [[{'Title': 'سلام ژوند څنګه دی (1) 😀🥳'}, {'Title': 'سلام ژوند څنګه دی (1) ??'}],]
If I do the select query directly in the MySQL database with MySQL Workbench I get exactly the same emojis as I the DMS validation tool gets from the SQL Server.
Besides this I am also getting a lot of RECORD_DIFF [[{'Content': '<VALUE_PRINT_NOT_SUPPORTED>'}, {'Content': '<VALUE_PRINT_NOT_SUPPORTED>'}],] which I am not sure what they mean querying both database I see the values perfectly for those rows.
Any help will be greatly appreciated.
Thanks, Claudio.
I have faced a similar issue before. You can raise a support ticket with AWS to help address the issue.
Meanwhile, here are some options.
- Remove the table from your validation list. You can verify it using SQL queries and hash function later.
- If you know the reason for the mismatch, you can adjust for it using modified validation rules (see section titled Using JSON editor to modify validation rules).
- If the mismatch is due to charset, you can adjust it using Character substitution task settings
- You can use a combination of the 1,2, and 3. For example, you can validate the length of the column using validation settings but then validate the content later using hash function.
Please let us know how you handled it.
Relevant questions
DMS to migrate SQL Server from RDS to EC2
asked 2 years agoDMS charges for target S3 Bucket PUTs
Accepted Answerasked 2 years agoDo source filters speed up DMS from AWS RDS to S3?
asked 6 months agoFalse positives with AWS DMS validations.
asked 3 months agoDMS - When to switch the application to the new target database
Accepted Answerasked 5 months agoETL from an Oracle database into RDS Postgres too slow
Accepted Answerasked 2 years agoCan I import an RDS uncrypted mysql snapshot to an RDS encrypted mysql?
Accepted Answerasked 3 months agoData Base Migration from On Prem My SQL Server to AWS RDS using DMS
asked 2 months agoDMS between MySQL RDS in 2 different accounts
asked 2 months agoIs there a service to move a database from MSSQL to MySQL (both hosted on AWS)
asked 5 months ago