- Newest
- Most votes
- Most comments
Hi Dhairya,
DataSync does not support EBS as a direct location to transfer data from. If you can create an NFS or SMB share for that EBS data then you can use DataSync to move data from the source file share to an EFS location.
Hello, if you are able to mount the EFS on the instance with that EBS attached, you can copy the information using tools like rsync, which will allow to sync all the data from source to destination, here you have an example of how to use it
rsync -apv /data/mydisk/ /mount/myefs/
This command will sync all the info from the specific path to your mount point.
I hope this works for you
I cannot believe the answers given by AWS support: They have provided a minimum amount of information. while rsync works, it depends upon a cron - which may be unreliable. For DataSync to use it, it's complicated. AWS is clearly 'still working' on making this easy.
(1) You need to set up an NFS service using the instance that you have your EBS attached to. cf. https://linuxhint.com/install-and-configure-nfs-server-ubuntu-22-04/ for example and step-by-steps. You can test that your NFS server works by using another instance, and mounting it there using /etc/fstab .. (I think the link above shows you how to do that). You will need the IP address of your NFS server (for me, this is 10.0.33.5)
(2) You will need to deploy a DataSync Agent - this is a new instance. It need lots of ram (so, expensive) - eg m1.xlarge - especially if your EBS is big and has many thousands of files. look here for that https://docs.aws.amazon.com/datasync/latest/userguide/deploy-agents.html#ec2-deploy-agent
You now have an DataSync Agent Instance (which should be on the same subnet and AZ as your NFS instance) showing on your EC2. You will need it's private IP number.. (for me, this is 10.0.33.111)
(3) You need to create an AWS Endpoint. (in VPC) You are going to add one with AWS Services - search for and choose 'datasync', and add that to the subnets that your NFS Server and Agent are on. Once that is created, you will need the IP address of the subnet / AZ that you are using. (For me this is 10.0.33.222)
(4) You will need to get your Agent Activation Key. ssh into an instance (like your nfs server) and then you need to get your key, using your region (mine is eu-west-1 ) and the two IP numbers you have recorded.. Do not use MY ones!!
curl "http://10.0.33.111/?gatewayType=SYNC&activationRegion=eu-west-1&privateLinkEndpoint=10.0.33.222&endpointType=PRIVATE_LINK&no_redirect"
If all is well you will get a long Activation Key string like XXXX-XXXX-XXXX-XXXX
(5) Now you need to add all this into your DataSync Agents list (it's an Amazon EC2 Hypervisor, using a VPC endpoint using "AWS PrivateLink". The endpoint should show automatically) and paste in your ID Activation Key from step 4 above. You should now see an active Agent in your Agents list (in DataSync).
(6) Now you can create a Location that uses that agent. Select NFS, and your Agent, Now put the ip address of your nfs server (from step 1, eg 10.0.33.5) and the mount path (the same as what you used in your /etc/exports file on the instance where you have attached your EBS eg /mnt/mydrive
(7) NOW you can create a DataSync task from your NFS to your EFS..
Relevant content
- asked 2 years ago
- asked 6 years ago
- AWS OFFICIALUpdated 4 days ago
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated a year ago
Thank yuo so much Gustavo for the answer. However this use case requires me to specifically use the DataSync service to transfer data from EBS volume to EFS site. Any thoughts on how can I accomplish this with DataSync?