Error with awscli sync command


I use very often the 'aws s3 sync s3://xxxx/ local-dir/' command, without any problem, but today a problem surfaced.

A file is downloaded each time I run the command, and in a wrong position. Just to be shure that it is not a problem with my local computer, I'm running this on AWS CloudShell.

[cloudshell-user@ip-10-0-96-146 ~]$ aws --version
aws-cli/2.1.28 Python/3.8.8 Linux/4.14.225-168.357.amzn2.x86_64 exec-env/CloudShell exe/x86_64.amzn.2 prompt/off

So here is my problem (sorry I cannot share here exact details of the pathnames).

$ mkdir X
$ aws s3 sync s3://my-bucket/ X/

After the download a single file, say '' shows up in 'X/' both in its correct position, say 'X/ghost-town/' and as 'X/'

If I repeat the command, each time I get the very same message, and the file is donwloaded again and again.

$ aws s3 sync s3://my-bucket/ X/
download: s3://my-bucket// to X/
$ aws s3 sync s3://my-bucket/ X/
download: s3://my-bucket// to X/

I have no way to make this bug reproducible, because in 'my-bucket' there are hundreds of files with similar names, but this problem surfaced only recently for a single file. Moreover other buckets sync nicely.
Please also note the '//' in the download message: the correct URI of the file is 's3://my-bucket/ghost-town/', its like as if the prefix is lost...

I just got another glimpse to the problem. With '$ aws s3 ls s3://proface' the file does not show up, but

$ aws s3 ls s3://my-bucket --recursive | grep
2021-06-16 21:26:38 4053 /
2021-06-16 21:28:03 4053 ghost-town/

Note again the spurious slash... Bucket versioning is not enabled.

The problem is quite annoying: how can I investigate this further?

asked 3 years ago293 views
1 Answer

Maybe I've nailed down the problem: in my bucket there is an empty prefix

aws s3 ls s3://my-bucket
PRE ghost-town/

in which the spurious copy of the file is stored. So actually awscli is confused because there is no way to map the remote file correctly to the local filesystem.

Edited by: stefano-m on Jun 29, 2021 8:25 AM

Edited by: stefano-m on Jun 29, 2021 8:32 AM

answered 3 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions