Data Migration from EBS to FSx for OpenZFS Stops partially

There are several potential reasons why your data migration from EBS to FSx for OpenZFS is stopping around the 7GB mark. Here are some possible causes and recommendations to address this issue:

1. Network connectivity: Ensure that your EC2 instance's security group is properly configured to allow traffic to the FSx for OpenZFS file system. Check that the necessary inbound rules are in place, particularly for the NFS port (2049). Additionally, verify that other required ports such as 111 and 20001 (TCP and UDP) are open.

2. File system capacity: Confirm that your FSx for OpenZFS file system has sufficient storage capacity to accommodate the entire 13GB dataset. If the file system is nearly full, it may cause the transfer to stop.

3. Large file transfer: If you're transferring large files, network interruptions can cause the process to halt. Consider using a tool that supports restartable transfers. For example, when using rsync, you can add the "--partial" flag to allow resuming interrupted transfers.

4. EC2 instance resources: A t2.medium instance may have limited network bandwidth and CPU credits. Monitor the instance's CPU utilization and network performance during the transfer. If you're hitting resource limits, consider upgrading to a larger instance type temporarily for the migration.

5. FSx for OpenZFS performance: Check if you're reaching the IOPS or throughput limits of your FSx for OpenZFS file system. You may need to increase the file system's performance capacity if you're encountering bottlenecks.

To ensure a full data transfer without interruption, try the following steps:

1. Use rsync with appropriate flags: sudo rsync -avz --partial --progress /source_path /fsx/destination_path

   The "--partial" flag allows resuming interrupted transfers, and "--progress" provides real-time transfer information.

2. If using Robocopy on a Windows instance, enable restartable mode: robocopy /e /zb /copy:DAT /mt:8 Y:\ Z:\

   The "/zb" option enables restartable mode for large files over potentially unreliable connections.

3. Monitor the transfer process closely and check system logs for any error messages or warnings that might indicate the cause of the interruption.

4. If possible, try the transfer during off-peak hours when network traffic might be lower.

5. Consider breaking the transfer into smaller chunks if a single large transfer continues to fail.

By implementing these suggestions, you should be able to complete your data migration from EBS to FSx for OpenZFS successfully. If issues persist, you may need to engage AWS support for further investigation into any potential file system or network-related problems.

Sources
Troubleshooting Amazon FSx for OpenZFS issues - FSx for OpenZFS
Mounting FSx for OpenZFS fails on Windows EC2 instance | AWS re:Post
Migrating files to Amazon FSx for OpenZFS using Robocopy - FSx for OpenZFS
Migrating files to Amazon FSx for OpenZFS using rsync - FSx for OpenZFS