How do I transfer data from EBS volume to EFS site using DataSync?

1

Hi There,

I want to copy some data from my EBS volume to EFS site and for this purpose I want to use AWS DataSync. When I am trying to choose source location (while setting up Datasync Task) there is no option to choose EBS volume as source point. Can someone guide on how can I accomplish data transfer from EBS volume to EFS using DataSync service?

3 Answers
1

Hi Dhairya,

DataSync does not support EBS as a direct location to transfer data from. If you can create an NFS or SMB share for that EBS data then you can use DataSync to move data from the source file share to an EFS location.

AWS
answered 2 years ago
0

Hello, if you are able to mount the EFS on the instance with that EBS attached, you can copy the information using tools like rsync, which will allow to sync all the data from source to destination, here you have an example of how to use it

rsync -apv /data/mydisk/ /mount/myefs/

This command will sync all the info from the specific path to your mount point.

I hope this works for you

AWS
answered 2 years ago
  • Thank yuo so much Gustavo for the answer. However this use case requires me to specifically use the DataSync service to transfer data from EBS volume to EFS site. Any thoughts on how can I accomplish this with DataSync?

0

I cannot believe the answers given by AWS support: They have provided a minimum amount of information. while rsync works, it depends upon a cron - which may be unreliable. For DataSync to use it, it's complicated. AWS is clearly 'still working' on making this easy.

(1) You need to set up an NFS service using the instance that you have your EBS attached to. cf. https://linuxhint.com/install-and-configure-nfs-server-ubuntu-22-04/ for example and step-by-steps. You can test that your NFS server works by using another instance, and mounting it there using /etc/fstab .. (I think the link above shows you how to do that). You will need the IP address of your NFS server (for me, this is 10.0.33.5)

(2) You will need to deploy a DataSync Agent - this is a new instance. It need lots of ram (so, expensive) - eg m1.xlarge - especially if your EBS is big and has many thousands of files. look here for that https://docs.aws.amazon.com/datasync/latest/userguide/deploy-agents.html#ec2-deploy-agent

You now have an DataSync Agent Instance (which should be on the same subnet and AZ as your NFS instance) showing on your EC2. You will need it's private IP number.. (for me, this is 10.0.33.111)

(3) You need to create an AWS Endpoint. (in VPC) You are going to add one with AWS Services - search for and choose 'datasync', and add that to the subnets that your NFS Server and Agent are on. Once that is created, you will need the IP address of the subnet / AZ that you are using. (For me this is 10.0.33.222)

(4) You will need to get your Agent Activation Key. ssh into an instance (like your nfs server) and then you need to get your key, using your region (mine is eu-west-1 ) and the two IP numbers you have recorded.. Do not use MY ones!!

curl "http://10.0.33.111/?gatewayType=SYNC&activationRegion=eu-west-1&privateLinkEndpoint=10.0.33.222&endpointType=PRIVATE_LINK&no_redirect"

If all is well you will get a long Activation Key string like XXXX-XXXX-XXXX-XXXX

(5) Now you need to add all this into your DataSync Agents list (it's an Amazon EC2 Hypervisor, using a VPC endpoint using "AWS PrivateLink". The endpoint should show automatically) and paste in your ID Activation Key from step 4 above. You should now see an active Agent in your Agents list (in DataSync).

(6) Now you can create a Location that uses that agent. Select NFS, and your Agent, Now put the ip address of your nfs server (from step 1, eg 10.0.33.5) and the mount path (the same as what you used in your /etc/exports file on the instance where you have attached your EBS eg /mnt/mydrive

(7) NOW you can create a DataSync task from your NFS to your EFS..

Konchog
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions