How does DataSync determine if a file has changed?

0

Example use case is a daily data export locally that needs to sync to S3. There is no way to check what has changed before exporting, so must export the full dataset. Aim is to only upload what has changed to S3. DataSync seems like it will work (it will only be one-way transfer daily so choosing DataSync over Transfer or File Gateway) but how does it determine what has changed?

I found the docs which say:

In the PREPARING status, DataSync examines the source and destination file systems to determine which files to sync. It does so by recursively scanning the contents of the source and destination file systems for differences.

Will this scan the contents of the files themselves, or just the filesystem metadata? Anyone know anymore details on how it determines what has changed?

AWS
已提问 4 年前973 查看次数
1 回答
0
已接受的回答

This page says

When a task starts, DataSync performs a recursive directory listing to discover all files and file metadata in the source and destination file system. These listings are used to identify differences and determine what to copy, and usually takes between a few minutes to a few hours.

Which suggests it's based on properties such as modification time and size, rather than a checksum.

profile pictureAWS
专家
James_S
已回答 4 年前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则