Best practices for changing Aurora source DB for DMS

Question

We have a Redshift instance that is being filled via DMS from an Aurora cluster. Because the DMS requires the binlog to process data through to Redshift we have to have it pointing directly to the writer DB. My question is about best practices if that writer changes instance - either through a failover or even when doing a blue-green deployment change (I presume that the binlog on the new server after a blue-green deployment is not going to match the previous one). In our case, we are upgrading from Aurora MySQL 5.7 to 8.0, and our main DB is not encrypted, so we are creating a new instance to upgrade to MySQL 8.0 that will be replicating from the 5.7 server, but when we go live we need our Redshift cluster to continue to properly consume from the new DB instance.

The only way I can figure to do it is to restart the replication task from scratch, which will take more time than I want to give it. So I wanted to see if there is a best practices for this scenario that will allow the replication task to continue on the new server where it left off from the previous one.

The only way I can figure to do it is to restart the replication task from scratch, which will take more time than I want to give it.  So I wanted to see if there is a best practices for this scenario that will allow the replication task to continue on the new server where it left off from the previous one.

SriniV · Accepted Answer

Please check this blog https://aws.amazon.com/blogs/database/understand-and-optimize-replication-for-amazon-redshift-with-aws-dms/
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Task.CDC.html
you are correct on restarting DMS task for blue-green deployment isn't ideal thing to do. Leverage DMS CDC feature. This allows DMS to pick up from where it left off on the new instance.

subhashr · Answer

one way i could think of is making sure for the tables involved in the task stop making any dml or ddl changes ( stop the application and make app user with readonly access)

upgrade the cluster or even build a new encrypted cluster and setup a cdc only task without cdc start time or cdc start position.

once the task is running you can enable app user with write access so that we dont lose any changes.

Please note this needs planning and testing may be a dry run using lower environment dev/test so as to minimize downtime

Best practices for changing Aurora source DB for DMS

Relevanter Inhalt