My typical use is to sync a series of directories/subdirectories from S3 to a cifs mounted SMB share on a local Linux machine.
After a recent local server reboot/remount of network storage, the sync command now re-transfers EVERYTHING in the directories every time I run it. My belief is that the sync command is pulling the CURRENT time as the local timestamp instead of the modify time of the files.
I ran it with dryrun and debug, and got a series of syncstrategy statements that appear to show a comparison between the S3 and local file. If I'm reading this correctly, the filesize is the same, the S3 timestamps showing correctly, but the local file timestamp is showing the immediate current time.
The local linux environment ls shows the correct modified timestamp, which matches the s3 ls of the same file.
Here is example output from the debug. Note the "modify time:" section. I believe that this shows the correct modify time for the S3 files, but shows the time the command was run for the local files. (modified time: 2022-03-18 16:52:02-07:00 -> 2022-03-24 12:48:39.973111-07:00 <-- the command was run at this datetime, and it seems to climb with each file as the seconds tick by)
2022-03-24 12:48:40,066 - MainThread - awscli.customizations.s3.syncstrategy.base - DEBUG - syncing: com.my.bucket.share/FZAOD/album-VariousArtists_TimelessHits-10.mp3 -> /mnt/qnapprd_integration/AOD/FZAOD/album-VariousArtists_TimelessHits-10.mp3, size: 2827911 -> 2827911, modified time: 2022-03-18 16:52:02-07:00 -> 2022-03-24 12:48:39.973111-07:00
(dryrun) download: s3://com.my.bucket.share/FZAOD/album-VariousArtists_TimelessHits-10.mp3 to FZAOD/album-VariousArtists_TimelessHits-10.mp3
2022-03-24 12:48:40,066 - MainThread - awscli.customizations.s3.syncstrategy.base - DEBUG - syncing: com.my.bucket.share/FZAOD/album-VariousArtists_TimelessHits-11.mp3 -> /mnt/qnapprd_integration/AOD/FZAOD/album-VariousArtists_TimelessHits-11.mp3, size: 3248378 -> 3248378, modified time: 2022-03-18 16:52:12-07:00 -> 2022-03-24 12:48:39.945111-07:00
(dryrun) download: s3://com.my.bucket.share/FZAOD/album-VariousArtists_TimelessHits-11.mp3 to FZAOD/album-VariousArtists_TimelessHits-11.mp3
2022-03-24 12:48:40,067 - MainThread - awscli.customizations.s3.syncstrategy.base - DEBUG - syncing: com.my.bucket.share/FZAOD/album-VariousArtists_TimelessHits-12.mp3 -> /mnt/qnapprd_integration/AOD/FZAOD/album-VariousArtists_TimelessHits-12.mp3, size: 4518138 -> 4518138, modified time: 2022-03-18 16:52:12-07:00 -> 2022-03-24 12:48:39.981111-07:00
(dryrun) download: s3://com.my.bucket.share/FZAOD/album-VariousArtists_TimelessHits-12.mp3 to FZAOD/album-VariousArtists_TimelessHits-12.mp3
2022-03-24 12:48:40,067 - MainThread - awscli.customizations.s3.syncstrategy.base - DEBUG - syncing: com.my.bucket.share/FZAOD/album-VariousArtists_TimelessHits-13.mp3 -> /mnt/qnapprd_integration/AOD/FZAOD/album-VariousArtists_TimelessHits-13.mp3, size: 8270994 -> 8270994, modified time: 2022-03-18 16:53:03-07:00 -> 2022-03-24 12:48:40.001111-07:00
(dryrun) download: s3://com.my.bucket.share/FZAOD/album-VariousArtists_TimelessHits-13.mp3 to FZAOD/album-VariousArtists_TimelessHits-13.mp3
2022-03-24 12:48:40,068 - MainThread - awscli.customizations.s3.syncstrategy.base - DEBUG - syncing: com.my.bucket.share/FZAOD/album-VariousArtists_TimelessHits-14.mp3 -> /mnt/qnapprd_integration/AOD/FZAOD/album-VariousArtists_TimelessHits-14.mp3, size: 5135882 -> 5135882, modified time: 2022-03-18 16:52:33-07:00 -> 2022-03-24 12:48:39.941111-07:00
Does anyone have any insight into how this timestamp is pulled, or what might stop s3 sync from retrieving the correct modify time of local files?
AWS OFFICIALAktualisiert vor einem Jahr