I'm working on an app that should remove unused S3 objects. Testing the DeleteObjects API using ruby aws-sdk-s3 (1.143.0) I ran into an issue: sometimes the response reported that it successfully deleted objects that never existed.
Here is an example:
s3_client.delete_objects(
bucket: 'my_bucket',
delete: {
objects: [
{ key: "kd13/639d/02a1/6000/7e37/0000/0000/fruits_1/banana1", version_id: "2065a469-38cb-4227-a068-5ec63c0a8b81"},
{ key: "kd13/639d/02a1/6000/7e37/0000/0000/fruits_2/banana2", version_id: "7d276a26-d33f-4888-8498-6819c642c1e9"},
{ key: "kd13/639d/02a1/6000/7e37/0000/0000/fruits_3/banana3", version_id: "5fa399d7-c852-4409-8cda-0343ed539b46"},
{ key: "kd13/639d/02a1/b8ea/6024/575a/d912/file5", version_id: "nmsWTEhkoaGM6nInyGGMjfa_FmS1Yop0"},
{ key: "kd13/639d/02a1/b8ea/6024/575a/d912/file5", version_id: "twcIET1.wRN1pFg_aa4mn5O2RtS2Bap7"},
]
}
)
Response:
#<struct Aws::S3::Types::DeleteObjectsOutput
deleted=[
#<struct Aws::S3::Types::DeletedObject key="kd13/639d/02a1/b8ea/6024/575a/d912/file1", version_id="nmsWTEhkoaGM6nInyGGMjfa_FmS1Yop0", delete_marker=nil, delete_marker_version_id=nil>,
#<struct Aws::S3::Types::DeletedObject key="kd13/639d/02a1/6000/7e37/0000/0000/fruits_3/banana3", version_id="5fa399d7-c852-4409-8cda-0343ed539b46", delete_marker=nil, delete_marker_version_id=nil>,
#<struct Aws::S3::Types::DeletedObject key="kd13/639d/02a1/b8ea/6024/575a/d912/file5", version_id="twcIET1.wRN1pFg_aa4mn5O2RtS2Bap7", delete_marker=nil, delete_marker_version_id=nil>
],
request_charged=nil,
errors=[
#<struct Aws::S3::Types::Error key="kd13/639d/02a1/6000/7e37/0000/0000/fruits_1/banana1", version_id="2065a469-38cb-4227-a068-5ec63c0a8b81", code="NoSuchVersion", message="The specified version does not exist.">,
#<struct Aws::S3::Types::Error key="kd13/639d/02a1/6000/7e37/0000/0000/fruits_2/banana2", version_id="7d276a26-d33f-4888-8498-6819c642c1e9", code="NoSuchVersion", message="The specified version does not exist.">
]>
My request contained a mix of real keys/version_ids and some fake ones to test how we handle errors.
Objects that have prefix kd13/639d/02a1/6000/7e37/0000/0000/fruits
have never existed in S3, they even have their version ids in the illegal format. As you can see, kd13/639d/02a1/6000/7e37/0000/0000/fruits_3/banana3
was reported as successfully deleted when the others were reported as not found.
Another problem is that later API calls (it's been more than an hour after the first API call) with the same input kept returning the same result, even though the real objects had been deleted earlier and I confirmed this.
I'd expect getting NoSuchVersion
for all the objects in the consecutive requests.
Is this an expected behavior?
Thank you.