DMS and Terraform

Question

We have a DMS task for reading data from MySQL and exporting data to S3 , which is created and updated by Terraform . The updations to the task are about adding tables to the list . Everytime such an update is deployed through Terraform , the task changes its state to Stopped . If we try to resume the task , it fails with the errors

2022-09-17T12:27:44 [SOURCE_CAPTURE  ]E:  Error 1236 reading binlog, exit after 11 retries [1020493]  (mysql_endpoint_capture.c:1152)
2022-09-17T12:27:44 [SOURCE_CAPTURE  ]E:  Error 1236 (Could not find first log file name in binary log index file) reading binlog [1020493]  (mysql_endpoint_capture.c:1153)

Restarting the task works fine though. I am guessing the pointer to the binlog file is lost somehow . 
Note , no changes are being done on the MYSQL end .

Any pointers on how to avoid this problem , while using Terraform ?

Answer

Whenever you Stop and Resume an AWS DMS task, it resumes from the position where it was paused. Meaning, if you resume the task after X minutes, AWS DMS task will try to get the changes since it was paused by reading the Binary Logs. Looking at the error, it looks like the task was not able to find the required Binary Logs to resume the replication.

If you happen to Stop the DMS task, make sure that the DB host has all the required Binary Logs available so that the task can be successfully resumed.

Answer

Hi!
Looking at the docs https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/dms_replication_task there is a note which seems relevant:

> NOTE:
Changing most arguments will stop the task if it is running. You can set start_replication_task to resume the task afterwards.

Unfortunately, the docs don't indicate the type of the parameter so I suggest you look at your state to see if it should be `start_replication_task = true` or something else.

Hope this helps!

DMS and Terraform

Relevant content