What Happens to Running Full-Load Tasks During a DMS Instance Scale-Up/Down.

0

In scale-up/scale-down solution for DMS Replication Instances, there are multiple instances, and each instance is monitored via CloudWatch Alarms. When an alarm is triggered, a message is sent to a topic that in turn calls a lambda to scale the instance appropriately. Eventually, the alarm sources DMS instance is either scaled to a smaller or larger instance.

We are attempting to determine the effects of the scale-up/down occurring while an attached task is in the middle of a "Full-load". The results appear to be inconsistent. Here is what has been observed.

  1. Restart a full load task attached to a target instance.
  2. Trip the alarm for that target instance via set_alarm_state.
  3. Very quickly, the target DMS instance state changes to "modifying" and the links in the UI are greyed out.
  4. While the DMS instance appears, in the UI, to be in a "Modifying" state, the Full-Load Task continues for another 5 minutes and completes.

Is this just happenstance?

Can anyone confirm that a state change from "Available" to "Modifying" for a DMS Instance is smart enough to wait for active full-loads to complete before rebooting or is this just some sort of latency between state changes and the UI?

It would be nice to have an explanation of what happens when a 30-minute full load task starts and the associated DMS instance scales up while that task is still running.

profile picture
asked a year ago489 views
1 Answer
1

Hi,

When you scale up / down your replication instance, its restarted as part of this modification. Upon restart, each DMS task running on the instance is stopped and resumed after new instance is available. DMS doesn't wait for any task to complete before it restarts the replication instance. Behavior that you noticed that full load task continued to run and completed before instance was restarted, is a coincidence. Below is the behavior of a full-load task when it resumes, after replication instance restart.

  1. Tables for which load was completed are not touched/affected
  2. Tables for which full load was in progress when instance restarted, load of such tables is restarted
  3. Tables for which full load was yet to start, will be queued and started as task progresses
AWS
SUPPORT ENGINEER
answered a year ago
  • Exactly the kind of answer I was hoping to get. Thanks again for the fast response!

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions