Skip to content

DMS Serverless Replications

0

Hi AWS,

I am trying to migrate RDS PostgreSQL from one vpc to another vpc in the same region. The database is big including more than 90 tables. The replication type is Full load and change data capture (CDC) and pgplogical plugin is used. The source and target are same PostgreSQL engine, 12.7. The Target table preparation mode is Do nothing.

The problem is that even though the target database is empty, as the replication proceeds, the error " duplicate key value violates unique constraint" happens. How to deal with it? My assumption is if target database is empty in the first place, the migration shouldn't have problems.

2 Answers
2
Accepted Answer

You might want to try changing the table preparation mode setting to "Drop tables on target". This would cause DMS to recreate the tables, ensuring that the tables are indeed empty and that the target schema matches the source.

I would otherwise agree that there might simply be rows violating the constraint at the source, but I believe PostgreSQL doesn't allow adding a unique constraint when the table contains data that conflicts with it, and earlier versions probably also didn't. Since the product and version are the same (and not, for example, migrating from SQL Server to PostgreSQL, which have divergent rules, as explained in the blog post) and the schemas should be identical, I would guess the likeliest thing to go wrong would be for data already to be present in the destination table. "Drop tables on target" would ensure the replication is started against a blank slate and with matching schemas for replicating the data.

Note that you'll need to create secondary indexes and other configuration not included in the scope of what "Drop tables on target" does, which you should be okay to do after the initial replication is complete. If this is tedious relative to what you've done already, you could alternatively try the "Truncate" option for table preparation, just to ensure that the tables are empty when starting.

For a completely different but much more straightforward approach, if you can tolerate some downtime for your database, you could simply suspend writes, make a snapshot of the RDS database, and create the new database in your preferred VPC from the snapshot. This would avoid the effort getting the replication to work and most of the time taken for the initial replication. The switchover would just not be nearly disruption-free as with continuous replication.

EXPERT
answered 2 years ago
EXPERT
reviewed 2 years ago
2

Hi,

I'd suggest you to read this blog post end to end to see if you fall into the edge cases that it raises around NULLs, etc. after analyzing your db schema with the described constraints in mind. Some of them may lead you to the error message that you get even when starting with an empty database

Blog post: https://aws.amazon.com/blogs/database/migrate-rows-with-unique-constraints-to-amazon-rds-for-postgresql-or-amazon-aurora-postgresql/

Best,

Didier

EXPERT
answered 2 years ago
EXPERT
reviewed 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.