Storage Gatweay (file) for migration?

0

I have around 20TB of data that I need to migrate to AWS over the next couple of weeks, and I know that Snowball will make this a breeze. However, I've been considering other options since I have a 1Gb internet circuit + time and I'm thinking about leveraging the Storage Gateway in File Mode. I launched an appliance on my VMware network and was able to get everything working. However, if I understand this service correctly, it copies the data to the "Cache" disk.

From the docs:

The cache storage acts as the on-premises durable store for data that is pending upload to Amazon S3 from the upload buffer. When your application performs I/O on a volume or tape, the gateway saves the data to the cache storage for low-latency access. When your application requests data from a volume or tape, the gateway first checks the cache storage for the data before downloading the data from AWS.

So let's say I want to move data in 5TB batches, I believe I would need a 5TB Cache volume? Is that accurate? So the process would be copy 5TB of data to the appliance, when in turn sends that data to S3. Wouldn't it just be quicker to run a tool like Cloudberry and just send the data right to the S3 bucket? Seems like the process is operations intensive, one job to copy the data and the other job to send to AWS. Just curious to see other folks experience with using Storage Gateway in this manner. Also, we don't have direct connect, so that's off the table.

asked 3 years ago27 views
2 Answers
0

Looks like I'm gonna roll with Datasync, and will use File Gateway to access any needed files.

answered 3 years ago
0

Ovidio B. wrote:
Looks like I'm gonna roll with Datasync, and will use File Gateway to access any needed files.

That's what we recommend as well. POSIX (NFS) metadata is preserved when you copy over the files using DataSync, so when your NFS clients access the migrated files via File Gateway, original ownership and permissions on the files are presented to them.

answered 3 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions