By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Database Migration Services Full Load and CDC - what is Table(s) loading still in progress status in cloud watch?

0

On the Data Migration Services, it looks like it has gone thru the process. Can someone please tell me if everything is loaded? In CloudWatch I am continually seeing "Table(s) loading still in progress". Is this referring to the data is still loading? Is it hung up?

+===== Source Table Capture List ========+
Table dbname.table01 found with estimated rows 10000
Table dbname.table01 found with estimated rows 457
Table dbname.table02 found with estimated rows 589
Table dbname.table03 found with estimated rows 897
Table dbname.table04 found with estimated rows 47852
+===== End of Source Table Capture List ========+
2023-09-28T14:39:12.603-04:00 Table(s) loading still in progress
2023-09-28T14:39:56.071-04:00 Table(s) loading still in progress
2023-09-28T14:40:39.476-04:00 Table(s) loading still in progress
2023-09-28T14:41:23.889-04:00 Table(s) loading still in progress
(same message repeated every few minutes)
2023-09-28T15:18:50.561-04:00 Table(s) loading still in progress

Thank You

Topics
Migration & ModernizationManagement & Governance
Tags
AWS Database Migration ServiceAmazon CloudWatch Logs
Language
English
mmoore
asked 7 months ago489 views
2 Answers
1

Sorry I did not specify this, but yes I am using the new Homegeneous Data Migration service with Full Load and change data capture (CDC). I ran another migration test this morning and received the same results.

I am seeing that the Migration Process is Completed, but I also see a Load running message. So I assume this would be the CDC process taking over.

mmoore
answered 7 months ago
  • EFK
    7 months ago

    TL;DR My migrations are taking a while likely due to large size of DBs and small target instance size; based on viewing metrics, processes and row counts on target RDS instance, the migration is continuing and not stalled

    NP. That sounds like it. Thanks for the updates.

    My migrations are still running. I used small target instance size for proof-of-concept purposes so that likely is contributing to the slowness.

    Some observations:

    1. On source DBs, the Networktransmitthroughput metric, for example, is back to normal / lower levels
    2. There are no connections from the user created for this (migration) purpose on source DBs
    3. The data migrations monitoring tab shows a storage consumption increase and has stayed at that level for several hours
    4. On target, there are several processes owned by user created for DMS purposes doing inserts; row counts are incremnting.
    5. DMS console doesn't show updated row counts and status is still at "Load running" and 0%
    6. CloudWatch logs are also not getting updated with row counts etc. with the most recent message being "Table(s) loading still in progress" as shared by OP
1

TL;DR Based on the provided log output: data is continuing to be read and stored in an intermediate location but not yet loaded; no indication of data migration being paused / stopped / terminated.

From Migrating data from MySQL databases with homogeneous data migrations in AWS DMS

For homogeneous data migrations of the Full load and change data capture (CDC) type, AWS DMS uses mydumper to read data from your source database and store it on the disk attached to the serverless environment. After AWS DMS reads all your source data, it uses myloader in the target database to restore your data. After AWS DMS completes the full load, it sets up the binlog replication with the binlog position set to the start of the full load."

Source database network transmit throughput, read IOPS etc. metrics while migrating a RDS MySQL DB (several hundred gigabytes) using Homogeneous Data Migration corroborate the above. Migration of a relatively smaller MySQL database displayed a similar set of CloudWatch log messages and completed successfully.

The status of the migration can be viewed by navigating to DMS -> Migration Projects -> Data migrations and also by clicking on the link associated with the data migration and then viewing the metrics under the Monitoring tab.

Related docs:

Notes:

  1. OP appears to be using the recently introduced Homegeneous Data Migration capabilities in DMS and not the "traditional" DMS (endpoints, replication instances, migration tasks).
  2. This is a revised / updated answer based on new observations since my earlier answer (which was heavily biased towards traditional DMS). The prior answer has been since deleted but contents of the same are appended FWIW.

Deleted answer:

Provided I am understanding your question correctly, navigating to Database migratiorstasks-> Table Statistics within the DMS service in AWS management console might help answer your questions by providina information on status of table loading (including whether table load errored out).

Here is a related re:Post Knowledge Center article: How do I use table statistics to monitor an AWS DMS task?.

Curious as to where / how in CloudWatch you are able to view the estimated row counts etc.? This appears to have some additional detail bevond what is available in Table Statistics.

Hope that helps.

EFK
answered 7 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content