Datasync is not transferring the files

0

Hi,

I have Datasync implemented at source account A with one location having source EFS ID and one location with NFS and agent. Datasync Task is successful but haven't transferred any files. I can see files are verified and prepared at destination but hasn't transferred. Both source and destinations are EFS.

asked 3 months ago152 views
4 Answers
0
Accepted Answer

Issue got resolved. I mentioned the Include path while creating the task and because of which files were not transferring. Paths were already mentioned in the location while creating.

answered 3 months ago
profile picture
EXPERT
reviewed 2 months ago
0

I understand you're experiencing an issue where your AWS DataSync task is completing successfully, but no files are being transferred. This can happen for a few reasons, and I'll explain some potential causes and solutions.

  1. File skipping: DataSync might skip files during the transfer process for various reasons. For example:

    • If the files in the source haven't been modified since the previous task execution, DataSync won't transfer them again if you're only set to transfer changed data.
    • If files exist in both source and destination but have changed in the source, DataSync might skip them if your task isn't configured to overwrite data in the destination.
  2. Permissions issues: Ensure that DataSync has the necessary permissions to read from your source EFS and write to the NFS destination. For the NFS destination, DataSync typically needs root access to write, set ownership, and modify metadata.

  3. NFS version compatibility: If you're using NFS version 4 with ID mapping, DataSync doesn't support this. You might need to create a new location using NFS version 3 or disable NFS version 4 ID mapping on the file system.

  4. Task configuration: Double-check your task settings to ensure you haven't accidentally configured it to only verify files without transferring them.

  5. Archival storage class: If any objects in your source are using an archival storage class and haven't been restored, DataSync will skip them.

To troubleshoot this issue:

  1. Review your task reports to understand why files are being skipped or not transferred.
  2. Check the permissions on both your source EFS and destination NFS to ensure DataSync has the necessary access.
  3. Verify your task configuration, especially settings related to data transfer and overwrite behavior.
  4. If using NFS version 4, try creating a new location with NFS version 3.

If the issue persists after checking these points, you may need to contact AWS Support for further assistance.
Sources
Troubleshooting issues with AWS DataSync transfers - AWS DataSync
Monitoring your data transfers with task reports - AWS DataSync
Configuring AWS DataSync transfers with an NFS file server - AWS DataSync
How AWS DataSync works - AWS DataSync

profile picture
answered 3 months ago
0

Hi there!, I checked and NFS version is set to Automatic. I tried to change the destination path and run the task again but this time also its successful but files are not transferred. So the overwriting setting is out of the box.

answered 3 months ago
0

Hi, is there a specific reason you are using Amazon EFS as an NFS location with an agent, rather than an agentless EFS to EFS DataSync task [1]? If the DataSync task is not transferring files, verify the mount point of the configured locations. The NFS location that is mounting the EFS file system, and the EFS location, can have a mount path as /. You can then use include and exclude filters to specify specific files or folders [2].

[1] https://docs.aws.amazon.com/datasync/latest/userguide/how-datasync-transfer-works.html#in-cloud-transfer

[2] https://docs.aws.amazon.com/datasync/latest/userguide/filtering.html

AWS
answered 3 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions