Al usar AWS re:Post, aceptas las AWS re:Post Términos de uso
AWS re:Postre:Post

How to recover from invalid resume token on DMS MongoDB connector?

0

When we pause our mongo connections on DMS for some hours and the resume point is no longer in the oplog we are unable to restart or resume our task.

Is there any way we can ignore these errors and continue replication or setup a hearbeat similar to Postgres RDS endpoint to update the resume point? The only fix we found was deleting the task and recreating it.

Resume of change stream was not possible, as the resume point may no longer be in the oplog
Temas
Migración y modernización
Etiquetas
Servicio de AWS Database Migration
Idioma
English
Gustavo Lee
preguntada hace 2 años1288 visualizaciones
1 Respuesta
0

Above error basically means the last oplog position when the dms task was stopped is no longer and there are chances of data loss due to missing transactions. I would suggest increasing the oplog so that the required entry postion is not overwritten

https://www.mongodb.com/docs/manual/tutorial/change-oplog-size/#c.-change-the-oplog-size-of-the-replica-set-member

AWS
subhashr
respondido hace 2 años
  • Gustavo Lee
    hace 2 años

    How can I recover after the problem has already happened withou deleting the task? Our main issue is because our task are created with terraform. We need to open a PR to fix this issue by recreating the resource because actions available on the UI won't recover from the error. Kafka mongo connector has an entire topic on how to recover from the error but I couldn't find anything similar for Mongo DMS.

    https://www.mongodb.com/docs/kafka-connector/current/troubleshooting/recover-from-invalid-resume-token/

    It looks like DMS never updates the "Change data capture (CDC) recovery checkpoint". We are unable to keep all the history in the oplog forever. How can we change the CDC to update the recovery checkpoint or restart it programatically?

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas

Contenido relevante