Skip to content

DataSync AWS to external, transient error

0

Hi all,

I am an AWS newbie trying to copy my data onto a second cloud server. I am using DataSync and can start the task execution. It goes through "launching" and "preparing" but when it's ready to execute and gets to "processing - 0%", it crashes with the error: "Task failed due to a transient error. Please re-run the task and contact AWS support if the problem persists". I have now tried to re-run the task twice and it takes about 30 minutes for it to get to the crash point each time. It has managed to transfer some files (less than a GB's worth).

The CloudWatch logs do not show any info on the error as far as I can tell.

Any advice?

Thanks :)

2 Answers
0

The error you're encountering, "Task failed due to a transient error," is typically related to network connectivity issues between the DataSync agent and the AWS DataSync service. This can occur for various reasons, such as temporary network glitches or insufficient resources allocated to the DataSync agent.

Here are some steps you can try to resolve the issue:

  1. Restart the DataSync agent: Sometimes, simply restarting the agent can resolve transient connectivity problems.

  2. Check network connectivity: Ensure that your network configuration allows the DataSync agent to maintain a stable connection with the AWS DataSync service. This includes checking firewalls and security groups.

  3. Verify resource allocation: Make sure the virtual machine or EC2 instance running the DataSync agent meets the recommended requirements. Insufficient resources can lead to connection issues.

  4. Split the transfer into smaller tasks: If you're transferring a large amount of data, try breaking it down into multiple smaller tasks using filters. This can help if the issue is related to the volume of data being transferred.

  5. Check permissions: Ensure that DataSync has the necessary permissions to access both the source and destination locations.

  6. Review task configuration: Double-check your task settings, particularly those related to data transfer and overwrite behavior.

If the problem persists after trying these steps, it would be advisable to contact AWS Support for further assistance, as they can provide more detailed troubleshooting based on your specific setup and task configuration.

Remember that DataSync is designed to handle transient errors and will typically retry operations automatically. However, if the error continues to occur, it may indicate a more persistent underlying issue that requires further investigation.
Sources
Datasync : S3 to GCP sync error (SyncDaemonRestart) | AWS re:Post
Troubleshooting issues with DataSync tasks - AWS DataSync

answered 8 months ago
AWS
SUPPORT ENGINEER
reviewed 8 months ago
  • I am trying 1, 4 and 6. How would I go about doing 3 for an instance that is ongoing? I think 5 is fine because some files have gone through and all my files have the same permissions.

0

Hi, I'm also getting the below error when using AWS Datasync in ENHANCED mode, but it works in BASIC mode. Both source and destination locations are s3 but in different regions. I don't use an agent. I'm creating locations, tasks & executions from Ruby SDK. I tried both from SDK & from the console. Volume of transfer is less than 2 GB. Below are the configurations & options.

{
    "Status": "ERROR",
    "TaskId": "task-0cce5c1f69b0e3888",
    "ErrorCode": "SyncInternalError",
    "ErrorDetail": "Task failed due to a transient error. Please re-run the task and contact AWS support if the problem persists"
}
Location:
Source: A bucket in us-east-1 from within a sub-directory
Destination: A bucket in us-west-1 to within a sub-directory
Both locations have permissions to access the buckets, auto-generated when configuring locations.
{
      verify_mode: 'NONE',
      overwrite_mode: 'ALWAYS',
      atime: 'BEST_EFFORT',
      mtime: 'PRESERVE',
      uid: 'NONE',
      gid: 'NONE',
      preserve_deleted_files: 'PRESERVE',
      preserve_devices: 'NONE',
      posix_permissions: 'NONE',
      task_queueing: 'ENABLED',
      log_level: 'TRANSFER',
      transfer_mode: 'ALL',
      security_descriptor_copy_flags: 'NONE',
      object_tags: 'NONE'
    }
answered 5 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.