Migrate MySQL 8 to Aurora MySQL 8
We are having trouble migrating on-premise MySQL 8.0.26 to AWS Aurora MySQL. For this project we're using AWS Database Migration Service (DMS) but it seems we are reaching the limit of the amount of databases to migrate. When we try to migrate over 500+ databases with one task, the task fails with the error "No tables were found at task initialization". When we edit the filter to migrate < 500 databases, the task starts successfully. Can anyone confirm that 500 databases is the limit for DMS? We have to do over 7k databases, but filtering those will not be easy. Is it possible to increase the current limit of 500? Kind regards, Lennard.
You can find limits for DMS here: https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Limits.html
Multiple tasks will be a better path for you as you also may be hitting some limits of the migration instance.
I dont think there is a limit on databases or schemas to migrate for mysql source. I would suggest enable extra logging (DETAILED_DEBUG) on component SOURCE_UNLOAD and track the query being passed by DMS against source to capture table list. once you have the query from log use that directly on source db to identify any issues. this could be an edge case or a bug. once identified i would suggest working with AWS Support for solution/bug reporting/workaround.
Thank you for the response. I've tried the job again with the suggested settings, but the error remains unchanged. No additional logging is visible in CloudWatch. I'll try to contact AWS Support on this.
Relevant questions
Migrating from RDS MariaDB 10.2 to Aurora MySQL 5.6
Accepted Answerasked 3 years agoBest practices for migrating MySQL RDS to Aurora Serverless
Accepted Answerasked 3 years agoAWS RDS Aurora MySql Serverless: ft_min_word_len is not being recognized
asked 2 months agoMigrate tables with LOBs from RDS MySQL to OpenSearch using DMS
asked 10 days agoMigrate MySQL 8 to Aurora MySQL 8
asked 3 months agoNeed help in upgrading the aurora MYSQL 5.7 to MYSQL 8 [URGENT]
asked 5 months agoMigrating Views from on-prem MySQL to RDS MySQL
asked a year agoAurora MySQL 8 Provisioned - Low Freeable Memory Causing Failover Despite Restricted Parameters
asked 5 months agoWhen will Mysql 8 be available in Aurora Serverless?
asked 3 months agoAre there difficulties migrating from Aurora MySQL to Aurora Serverless
Accepted Answer
See here for how to change the number of tasks: https://docs.aws.amazon.com/dms/latest/userguide/CHAP_BestPractices.html