Questions tagged with AWS DataSync
Content language: English
Sort by most recent
AWS FSx for DR and interaction with S3
Hello, a customer has 30TB of Windows shares and would like to have a DR solution for it in AWS. I thought first about using FSx for Windows and using DataSync to continuously import data from on-prem to FSx. However I would like to decrease the price of the solution: is it possible to have this kind of flow? 1. First import the data from on-prem to S3 2. Store the data in S3 when the DR is not activated 3. Make incremental backups regularly to S3 4. Finally, import to FSx when the DR is activated ? The customer is also using CommVault as a third-party tool. Thank you in advance.
File deletion with datasync if versioning is enabled.
What is happening if I'm syncing a File System with S3 versioning enabled, and I delete a file? I suppose that a delete marker is added, so the previous versions are kept. This mean that the client will have probably to do additional activities to clean up files deleted over a period of time if they want to keep their storage cost low. correct? THanks.
Sharing files between VPCs
My customer wants to share the CI/CD artifacts that they generate in their build environment to be shared in DEV/UAT/PROD and other environments. They are looking at using EFS for this file share, but they don't like the need to do VPC peering in order to make the EFS available across VPCs. They also dont want to use S3 as they are looking for a traditional file system which can be mounted and used during the build process. One option I could think of is to setup a EFS file system in every VPC and sync from the build environment to the target VPC using AWS DataSync. It would not be an automatic sync, so they will have have to call "StartTaskExecution" the source file system changes. Are there any other solutions which they can consider?
Datasync Agent in VPC
A customer is interested in DataSync and their security department is telling them to use VPC Endpoints/PrivateLink. As the team is looking through the documentation to provide the security department, a point of clarification is needed (https://docs.aws.amazon.com/datasync/latest/userguide/datasync-in-vpc.html). Documentation states that the agent can be "either on-premises or in-cloud and the DataSync service doesn't need to travel across the public internet or need public IP addresses". The following paragraph goes on to describe considerations when deploying the agent in a VPC. The customer would prefer to deploy the VMWare Datasync agent on-premises. East question that I would like answered: Does the agent have to be deployed in a VPC to use VPCE? Thank you in advance.
DataSync Validate Existing Data in S3 then Copy Changes
A customer manually copied over data via Snowball to S3 from an on-prem NFS mount. They are looking to copy changes since that transfer as well as verify all the files that are already in S3. Can DataSync compare hashes for existing files in S3 and then copy changes?
Datasync from AWS EFS to NFS Server Hosted in Another EC2 Instance
I have a EFS instance in Oregon with all the data that I need to sync. The location were I want to store that data could be anywhere, including AWS. For now, we want to make a POC using a EC2 instance in Ireland. We aready setup NFS server where the Datasync Agent should write the files from the EFS, but the problem is that we are not able to create the Datasync Agent in another EC2 instance using the .OVA image provided by Amazon. We tried to import it using this guide: <https://aws.amazon.com/ec2/vm-import/>. Is possible to do what I'm trying? To use Datasync to synchronize data from EFS to another NFS server hosted in AWS that emulates the on-premise datacenter, which should be the same as from _AWS EFS_ to _On-Premises_. If so, how can I install the Datasync Agent image provided by Amazon? If not, which solution should be the one that works for the scenario I'm trying to implement? PD: The intend of doing this is to have different instances with the same data and if any of them fails, have an instance backup. This is something similar as a fail-over, but we want to keep completely independent data architectures with the same data.
Feature Requirement: Automatically sync when folder content changes
Hello, Does anyone know if a future DataSync update will allow the agent to be 'always on' or automatically run tasks? This will help decide if it fits my use case. Thanks, Carter Edited by: Carter11 on Feb 7, 2019 5:36 PM Edited by: Carter11 on Feb 7, 2019 5:37 PM
Hi Hopefully a quick question, but are the memory system requirements flexible? https://docs.aws.amazon.com/datasync/latest/userguide/requirements.html#hardware I can get a VM with 16GB of ram, but 32GB isn't possible at this time. Will the data sync agent run with 16GB? Is there setting that might need to be tweaked? I'm not that concerned with the highest performance, we will likely need to cap the transfer at 100Mbps Cheers, Jason
Console not working
Hi, I wanted to give DataSync a try, but the first step would be to download the VM image from the console. And the console is not working for me. The browser just shows the header, the footer, and a spinner in the middle. Some details: 1. my browser is Google Chrome Version 70.0.3538.110 2. the console works for other services 3. region does not matter, when I change it the console still does not work 4. I am behind a corporate proxy, the browser is supposed to handle 5. The proxy does some filtering, which I have not met in AWS websites yet, and it gives a readable message, which I do not see in this case. Does it work for you? Any tips or tricks, maybe downloading the VM image in a different way? Thanks in advance Edited by: kanya on Jan 22, 2019 1:40 AM