Can I run two or more tasks using one replication instance in AWS DMS?

0

Here is the concept that I'm trying to do. Concept: 1 instance Server 1 - MySQL 5.5.59 Server 2 - MariaDB 10.5 Task 1: Migrate and replicate ongoing changes Server 1-database a to Server 2-database a Task 2: Migrate and replicate ongoing changes Server 1-database b to Server 2-database b ....

However, it always returns an error after a while. One of them will always surely fail. Errors that I'm getting: "Task stopped abnormally." "Task has stopped after 6 successive..."

Is this concept feasible? I was testing concepts because my initial loading of a 200GB migration always shows "Running with errors" or "Failed" after running within a few hours.

asked 8 months ago604 views
1 Answer
1

Absolutely! DMS can run multiple tasks in parallel on the same replication instance, each replication task is its own process. For MySQL, another possible reason why you’re facing issues is because they’re hitting limitations. I highly recommend going over this list as it can cause tasks to fail.

  1. Using a MySQL-compatible database as a source for AWS DMS - Limitations on using a MySQL database as a source for AWS DMS - https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.MySQL.html#CHAP_Source.MySQL.Limitations

  2. Modifying Resource Allocation - Stop all currently running replication tasks, navigate to the AWS DMS console, try increasing size of instances by choosing a larger instance type.
https://docs.aws.amazon.com/dms/latest/userguide/Welcome.html
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_GettingStarted.Replication.html
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_ReplicationInstance.Modifying.html#

  3. Verify Network Connectivity via Security Group and NACLs - Make sure your target and source databases are accessible and that their corresponding ports are open. Check for any firewall rules or network configs blocking connectivity between the DMS replication instance and the databases.
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_ReplicationInstance.VPC.html
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Security.html

  4. Task Configuration Settings - Double check your task configuration settings for both replication tasks, make sure your target and source endpoints, table mappings, and migration settings are properly configured. 
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.Creating.html
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Monitoring.html
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TaskSettings.html

  5. Monitoring - Monitor your replication instance’s resource utilization whenever it’s executing its task. Navigate to the AWS DMS console, open replication instances and select desired replication instance and click on the “Monitoring” tab to view metrics that’s related to memory, CPU, network usage (Cloudwatch Metrics)
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Monitoring.html
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_ReplicationInstance.html

  6. Restart/Retry - For the replication tasks that fail try restarting them after modifying any configuration rules. Navigate to the AWS DMS console, click “Tasks”, select your task and choose to either “Resume” or “Retry” based on the error

AWS
Tarun_B
answered 8 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions