DMS task stuck


I have a problem with a DMS task connecting an Aurora instance (source) to a redshift instance (target).
Apparently the task is stuck at 97% while migrating some big tables (10 million rows each approx).
We recently migrated from MySql to Aurora, hence the need to rerun the full migration.
The task is "Full load, ongoing replication".

After 8 hours of migration going on, the task restarts with an error:

019-07-30T10:57:35 [SOURCE_UNLOAD ]E: Connection error [1022506] (mysql_endpoint_imp.c:106)
2019-07-30T10:57:35 [SOURCE_UNLOAD ]E: Endpoint is disconnected [1020414] (endpointshell.c:2766)
2019-07-30T10:57:35 [SOURCE_UNLOAD ]E: Error executing source loop [1020414] (streamcomponent.c:1596)
2019-07-30T10:57:35 [TASK_MANAGER ]E: Stream component failed at subtask 1, component st_1_J7ZYBQT4N6DCQRNGEDCD646RRA [1020414] (subtask.c:1350)
2019-07-30T10:57:35 [SOURCE_UNLOAD ]E: Stream component 'st_1_J7ZYBQT4N6DCQRNGEDCD646RRA' terminated [1020414] (subtask.c:1513)
2019-07-30T10:57:35 [TASK_MANAGER ]E: Task error notification received from subtask 1, thread 0 [1020414] (replicationtask.c:2379)
2019-07-30T11:01:47 [SOURCE_UNLOAD ]E: Connection error [1022506] (mysql_endpoint_imp.c:106)
2019-07-30T11:01:47 [SOURCE_UNLOAD ]E: Endpoint is disconnected [1020414] (endpointshell.c:2766)
2019-07-30T11:01:47 [SOURCE_UNLOAD ]E: Error executing source loop [1020414] (streamcomponent.c:1596)
2019-07-30T11:01:47 [TASK_MANAGER ]E: Stream component failed at subtask 2, component st_2_J7ZYBQT4N6DCQRNGEDCD646RRA [1020414] (subtask.c:1350)
2019-07-30T11:01:47 [SOURCE_UNLOAD ]E: Stream component 'st_2_J7ZYBQT4N6DCQRNGEDCD646RRA' terminated [1020414] (subtask.c:1513)
2019-07-30T11:16:02 [SOURCE_UNLOAD ]E: Connection error [1022506] (mysql_endpoint_imp.c:106)
2019-07-30T11:16:02 [SOURCE_UNLOAD ]E: Endpoint is disconnected [1020414] (endpointshell.c:2766)
2019-07-30T11:16:02 [SOURCE_UNLOAD ]E: Error executing source loop [1020414] (streamcomponent.c:1596)
2019-07-30T11:16:02 [TASK_MANAGER ]E: Stream component failed at subtask 3, component st_3_J7ZYBQT4N6DCQRNGEDCD646RRA [1020414] (subtask.c:1350)
2019-07-30T11:16:02 [SOURCE_UNLOAD ]E: Stream component 'st_3_J7ZYBQT4N6DCQRNGEDCD646RRA' terminated [1020414] (subtask.c:1513)
2019-07-30T11:19:24 [SOURCE_UNLOAD ]E: Connection error [1022506] (mysql_endpoint_imp.c:106)
2019-07-30T11:19:24 [SOURCE_UNLOAD ]E: Endpoint is disconnected [1020414] (endpointshell.c:2766)
2019-07-30T11:19:24 [SOURCE_UNLOAD ]E: Error executing source loop [1020414] (streamcomponent.c:1596)
2019-07-30T11:19:24 [TASK_MANAGER ]E: Stream component failed at subtask 4, component st_4_J7ZYBQT4N6DCQRNGEDCD646RRA [1020414] (subtask.c:1350)
2019-07-30T11:19:24 [SOURCE_UNLOAD ]E: Stream component 'st_4_J7ZYBQT4N6DCQRNGEDCD646RRA' terminated [1020414] (subtask.c:1513)

This has happened at least twice since we started the task.


asked 5 years ago523 views
1 Answer

I solved it by setting some endpoint settings(!) on the redshift end of things:


This prevented the timeout from happening again and permitted the completion of the process.


answered 5 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions