DMS Sync Issues with nullable column and new columns
IHAC who is facing a couple of sync issues: Say we have a table T on mysql src db with a column C that is nullable. The table is fully loaded on the target and the DMS task is operating in CDC mode. If this table T gets altered so column C is made not nullable with a default, this change does not propagate , nor does the DMS task indicate an error. In addition, any new columns added to table T on the source, also do not propagate. When the DMS task is operating in CDC mode, new tables that are created without a Primary Key do not propagate to the target. It can break DMS replication (error shows in logs). Thank you
Hello,
this is a documented limitation of CDC for MySQL as a source: https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.MySQL.html#CHAP_Source.MySQL.Limitations
Hello,
--> In general, DMS supports below DDL: + https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Introduction.SupportedDDL.html
--> Using MySQL(on-prem/Aurora MySQL/RDS MySQL) as source having limitation: "However, DROP TABLE, RENAME TABLE, and updates made to other attributes, such as column default value, column nullability, character set and so on, are not supported." + https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.MySQL.html#CHAP_Source.MySQL.Limitations
--> If create table or add column not working properly, will need to dive deep into the issue using DMS log with detailed debug on SOURCE_CAPTURE, TASK_MANAGER, TARGET_APPLY to check further.
Note: "Using an ALTER TABLE table_name ADD COLUMN column_name statement to add columns to the beginning (FIRST) or the middle of a table (AFTER) isn't supported. Columns are always added to the end of the table."
--> What are the target/source endpoints:
Engine version (source and target) DMS version -- Check release note see if any known issues
--> Go through the current limitations and documentations to verify further. + https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.MySQL.html
Relevant questions
Cannot change column used in a foreign key constraint
asked 5 months agoError 1054 Unknown Column while Inserting into Table in Aurora MySQL
asked 10 months agoDMS Sync Issues with nullable column and new columns
asked 5 months agoWhat is the impact of swapping CMKs that encrypt a DDB table?
Accepted Answerasked 2 years agoDMS not replicating addition of new columns to source
asked 5 days agoAWS DMS doing a SELECT on TEXT column during continuous replication
asked 3 months agoTable id found in `stl_load_errors` doesn't correspond to loaded table
Accepted Answerasked 3 years agoQuickSight Freeze First Column in a table view
asked a month agoAWS Glue problems reading from PostgreSQL DB that has uppercased table and column names
asked 4 days agoCan we add column to an existing table in AWS Athena using SQL query?
Accepted Answerasked 3 years ago
Would this also apply to Aurora MySQL as a source? Customer has Aurora MySQL