How to sync aws s3 glacier files not checking timestamp of file size but only file existence on destination ?
We regularly backed up our picture source files (it's about a couple of hundred thousands) to AWS glacier, using --size-only and --delete option (as timestamp of the files could be differ depending of the fact that editing software change timestamp every time it has been accessing a file). So that files only edited, so differed in size were synced, as well as those files that were no more existing on source were deleted on s3 destination. But recently when we changed the editing software version it had to upgrade its catalog and, much worse, while updating it wrote to some files such as JPG or DNG some inner metadata, which itself led to change in their size. Consequently, now --size-only option is not enough, it now doesn't matter at all as AWS tries to sync about 80% of files that were not updated by editing and don't have any actual differences on source, but about of metadata adding has tiny differences in size. In fact it looks like we have to re-upload 80% of the files again, without any reasons to.
So I wonder, is there a possibility to sync local and s3 buckets based only on the existence of a file, not about its timestamp of size ?
AWS 官方已更新 2 年前