Error with awscli sync command

0

I use very often the 'aws s3 sync s3://xxxx/ local-dir/' command, without any problem, but today a problem surfaced.

A file is downloaded each time I run the command, and in a wrong position. Just to be shure that it is not a problem with my local computer, I'm running this on AWS CloudShell.

[cloudshell-user@ip-10-0-96-146 ~]$ aws --version
aws-cli/2.1.28 Python/3.8.8 Linux/4.14.225-168.357.amzn2.x86_64 exec-env/CloudShell exe/x86_64.amzn.2 prompt/off

So here is my problem (sorry I cannot share here exact details of the pathnames).

$ mkdir X
$ aws s3 sync s3://my-bucket/ X/

After the download a single file, say 'ghost.abc' shows up in 'X/' both in its correct position, say 'X/ghost-town/ghost.abc' and as 'X/ghost.abc'

If I repeat the command, each time I get the very same message, and the file is donwloaded again and again.

$ aws s3 sync s3://my-bucket/ X/
download: s3://my-bucket//ghost.abc to X/ghost.abc
$ aws s3 sync s3://my-bucket/ X/
download: s3://my-bucket//ghost.abc to X/ghost.abc

I have no way to make this bug reproducible, because in 'my-bucket' there are hundreds of files with similar names, but this problem surfaced only recently for a single file. Moreover other buckets sync nicely.
Please also note the '//' in the download message: the correct URI of the file is 's3://my-bucket/ghost-town/ghost.abc', its like as if the prefix is lost...

I just got another glimpse to the problem. With '$ aws s3 ls s3://proface' the file does not show up, but

$ aws s3 ls s3://my-bucket --recursive | grep ghost.abc
2021-06-16 21:26:38 4053 /ghost.abc
2021-06-16 21:28:03 4053 ghost-town/ghost.abc

Note again the spurious slash... Bucket versioning is not enabled.

The problem is quite annoying: how can I investigate this further?

已提問 3 年前檢視次數 344 次
1 個回答
0

Maybe I've nailed down the problem: in my bucket there is an empty prefix

aws s3 ls s3://my-bucket
PRE /
PRE ghost-town/
...

in which the spurious copy of the file is stored. So actually awscli is confused because there is no way to map the remote file correctly to the local filesystem.

Edited by: stefano-m on Jun 29, 2021 8:25 AM

Edited by: stefano-m on Jun 29, 2021 8:32 AM

已回答 3 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南