Error with awscli sync command

0

I use very often the 'aws s3 sync s3://xxxx/ local-dir/' command, without any problem, but today a problem surfaced.

A file is downloaded each time I run the command, and in a wrong position. Just to be shure that it is not a problem with my local computer, I'm running this on AWS CloudShell.

[cloudshell-user@ip-10-0-96-146 ~]$ aws --version
aws-cli/2.1.28 Python/3.8.8 Linux/4.14.225-168.357.amzn2.x86_64 exec-env/CloudShell exe/x86_64.amzn.2 prompt/off

So here is my problem (sorry I cannot share here exact details of the pathnames).

$ mkdir X
$ aws s3 sync s3://my-bucket/ X/

After the download a single file, say 'ghost.abc' shows up in 'X/' both in its correct position, say 'X/ghost-town/ghost.abc' and as 'X/ghost.abc'

If I repeat the command, each time I get the very same message, and the file is donwloaded again and again.

$ aws s3 sync s3://my-bucket/ X/
download: s3://my-bucket//ghost.abc to X/ghost.abc
$ aws s3 sync s3://my-bucket/ X/
download: s3://my-bucket//ghost.abc to X/ghost.abc

I have no way to make this bug reproducible, because in 'my-bucket' there are hundreds of files with similar names, but this problem surfaced only recently for a single file. Moreover other buckets sync nicely.
Please also note the '//' in the download message: the correct URI of the file is 's3://my-bucket/ghost-town/ghost.abc', its like as if the prefix is lost...

I just got another glimpse to the problem. With '$ aws s3 ls s3://proface' the file does not show up, but

$ aws s3 ls s3://my-bucket --recursive | grep ghost.abc
2021-06-16 21:26:38 4053 /ghost.abc
2021-06-16 21:28:03 4053 ghost-town/ghost.abc

Note again the spurious slash... Bucket versioning is not enabled.

The problem is quite annoying: how can I investigate this further?

已提问 3 年前315 查看次数
1 回答
0

Maybe I've nailed down the problem: in my bucket there is an empty prefix

aws s3 ls s3://my-bucket
PRE /
PRE ghost-town/
...

in which the spurious copy of the file is stored. So actually awscli is confused because there is no way to map the remote file correctly to the local filesystem.

Edited by: stefano-m on Jun 29, 2021 8:25 AM

Edited by: stefano-m on Jun 29, 2021 8:32 AM

已回答 3 年前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则