File Storage Gateway temporary storage / option to delay uploads

0

Hi,

So a random one, is there a means to either have a temporary storage area on the the file gateway appliance on prem, or an option to delay file uploads to S3 by a set amount of time?

The rational is that on prem we want to move data to the file gateway via NFS BUT also want to be prudent (as its archive data) and hash the file data after our internal transfer to ensure no corruption occurred as data transferred across our network (overkill - possibly, but the data is really important, so safe rather than sorry - and the average file size is large enough 50 - 100GB that its is a real possibility with some being hundreds GBs to TB range).

Ideally we don't want to have data uploaded whilst hashing just in case there is an issue with it and our systems have to delete and re-copy the data.

If there is a way to delay data being uploaded, or a means to have a temporary storage area no linked to a bucket from which good data can then be copied after the fact that would be awesome. It would also overcome the pitfalls of using rsync and an S3 bucket with version locking (which seems a bit of an oversight).

Thanks in advance,

Owen.

ommps
質問済み 2年前363ビュー
1回答
0
承認された回答

Currently there is no provision to delay the upload. Since file gateway can support files larger than the size of the Cache disk, as soon as 5 MiB of data is available, it will start uploading that data.

Your use case is interesting, we will consider that for a future consideration.

AWS
回答済み 2年前
  • Thank you for the info. It would be amazing if it could be added as like a system-wide user configurable option in the future for those who need just a bit of time between it hitting the cache disks and starting upload. And as mentioned it overcomes the 'issue' of rsync's temp files and S3 version locking.

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ