File Storage Gateway temporary storage / option to delay uploads

0

Hi,

So a random one, is there a means to either have a temporary storage area on the the file gateway appliance on prem, or an option to delay file uploads to S3 by a set amount of time?

The rational is that on prem we want to move data to the file gateway via NFS BUT also want to be prudent (as its archive data) and hash the file data after our internal transfer to ensure no corruption occurred as data transferred across our network (overkill - possibly, but the data is really important, so safe rather than sorry - and the average file size is large enough 50 - 100GB that its is a real possibility with some being hundreds GBs to TB range).

Ideally we don't want to have data uploaded whilst hashing just in case there is an issue with it and our systems have to delete and re-copy the data.

If there is a way to delay data being uploaded, or a means to have a temporary storage area no linked to a bucket from which good data can then be copied after the fact that would be awesome. It would also overcome the pitfalls of using rsync and an S3 bucket with version locking (which seems a bit of an oversight).

Thanks in advance,

Owen.

ommps
已提問 2 年前檢視次數 363 次
1 個回答
0
已接受的答案

Currently there is no provision to delay the upload. Since file gateway can support files larger than the size of the Cache disk, as soon as 5 MiB of data is available, it will start uploading that data.

Your use case is interesting, we will consider that for a future consideration.

AWS
已回答 2 年前
  • Thank you for the info. It would be amazing if it could be added as like a system-wide user configurable option in the future for those who need just a bit of time between it hitting the cache disks and starting upload. And as mentioned it overcomes the 'issue' of rsync's temp files and S3 version locking.

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南