I use very often the 'aws s3 sync s3://xxxx/ local-dir/' command, without any problem, but today a problem surfaced.
A file is downloaded each time I run the command, and in a wrong position. Just to be shure that it is not a problem with my local computer, I'm running this on AWS CloudShell.
[cloudshell-user@ip-10-0-96-146 ~]$ aws --version
aws-cli/2.1.28 Python/3.8.8 Linux/4.14.225-168.357.amzn2.x86_64 exec-env/CloudShell exe/x86_64.amzn.2 prompt/off
So here is my problem (sorry I cannot share here exact details of the pathnames).
$ mkdir X
$ aws s3 sync s3://my-bucket/ X/
After the download a single file, say 'ghost.abc' shows up in 'X/' both in its correct position, say 'X/ghost-town/ghost.abc' and as 'X/ghost.abc'
If I repeat the command, each time I get the very same message, and the file is donwloaded again and again.
$ aws s3 sync s3://my-bucket/ X/
download: s3://my-bucket//ghost.abc to X/ghost.abc
$ aws s3 sync s3://my-bucket/ X/
download: s3://my-bucket//ghost.abc to X/ghost.abc
I have no way to make this bug reproducible, because in 'my-bucket' there are hundreds of files with similar names, but this problem surfaced only recently for a single file. Moreover other buckets sync nicely.
Please also note the '//' in the download message: the correct URI of the file is 's3://my-bucket/ghost-town/ghost.abc', its like as if the prefix is lost...
I just got another glimpse to the problem. With '$ aws s3 ls s3://proface' the file does not show up, but
$ aws s3 ls s3://my-bucket --recursive | grep ghost.abc
2021-06-16 21:26:38 4053 /ghost.abc
2021-06-16 21:28:03 4053 ghost-town/ghost.abc
Note again the spurious slash... Bucket versioning is not enabled.
The problem is quite annoying: how can I investigate this further?