Failed to send an end of data record error in DMS

Hello there !

ERROR - "Failed to send an end of data record (ID=30256) for table with ID '1'. [1021911] (endpointshell.c:3741)"

This error typically occurs when there's an issue with the parallel data migration process, specifically when trying to send data in parallel chunks. Here are some potential reasons and solutions to help you troubleshoot and resolve the issue:

• Inconsistent data ranges: Double-check that the ID ranges for each parallel task are correctly defined and don't overlap. Make sure the ranges are contiguous and cover the entire ID range of the table.

• Data inconsistencies: Verify that the data in the table is consistent, and there are no gaps or missing IDs within the ranges. If there are gaps, you might need to adjust your range definitions or use a different segmentation strategy.

• Network connectivity issues: Ensure that the network connection between the migration tool and the database is stable and reliable. Network timeouts or disconnections can cause this error.

• Database connection limits: Check if the database has connection limits or restrictions that might be preventing the parallel tasks from connecting or sending data. You might need to increase the connection limit or adjust the parallel task configuration.

• Resource constraints: Verify that the system running the migration task has sufficient resources (CPU, memory, disk space) to handle the parallel processing. Resource constraints can cause the task to fail or hang.

• Migration tool configuration(DMS): Review the configuration of your migration tool to ensure it's set up correctly for parallel processing. Check the documentation for any specific settings or options related to parallel data migration.

• Database locking or contention: If the migration task is running concurrently with other processes or queries, it might be causing locking or contention issues. Consider running the migration task during a maintenance window or using row-level locking to minimize conflicts.

• Data type or formatting issues: Ensure that the data types and formatting of the ID field are consistent across the table and match the expectations of the migration tool.

To troubleshoot the issue, you can try the following:

1. Enable more detailed logging in the migration tool to capture additional error information.
2. Run the migration task in non-parallel mode to see if the issue persists.
3. Test the migration task with a smaller ID range or a subset of the data to isolate the problem.
4. Verify that the ID field is correctly indexed and optimized for range-based queries.

If you're still stuck, please try reaching out to AWS support with correct DMS task details so they can check and assist.