removing expired delete markers, how does it work?

Question

I'm trying to set up some simple lifecycle rules for deleting objects after a period of time and the inscrutable documentation is making this much more difficult then it should be.

I have a versioned bucket that I place files I'd like backed up. After 100 days, I'd like to delete them automatically.

I'm setting up a lifecycle with the **Expire current versions of objects** action
and set the *Days after object creation parameter* to 100.

According to the docs: "Amazon S3 adds a delete marker and the current version of an object is retained as a noncurrent version."

Ok, so now we need to add another action **Permanently delete noncurrent versions of objects**
with the  *Days after objects become noncurrent* parameter set to 1

That should delete all the noncurrent versions, leaving only the delete marker, which gets set to 'expired'

One would think you would then add the **Delete expired object delete markers** action to clean the expired delete marker up. But no. S3 won't let you add that action.
*You cannot enable Delete expired object delete markers if you enable Expire current versions of objects.*
But it doesn't clearly explain why.

An example if it helps clarify.
Starting with the versioned object like so:

```
example.txt 
 1. (current version) - example.txt v2
 2. (noncurrent version) - example.txt v1
```

After 100 days the current version of example.txt would expire, a delete marker is created for it and the current version becomes a noncurrent version. At this point it looks like so:

```
example.txt 
 1. (current version) - delete marker
 2. (noncurrent version) - example.txt v2
 3. (noncurrent version) - example.txt v1
```

Another 1 day later, the noncurrent example.txt versions would be permanently deleted and the "current" version (the delete marker) would "expire". So now it looks like so.

```
example.txt 
 1. (current version) - expired delete marker
```

So now what do I do?

Does S3 automatically remove expired delete markers on it's own?
Do I need to set up another lifecycle rule get around that weird restriction and clear them out?

thx!

Aymen · Answer

I ran into the same issue, and the documentation didn't provide a clear solution.  
For a Versioned Bucket, I had to create **two** Lifecycle Rules.

**Rule 1:** Delete After 100 Days
```
Actions:
- Expire current versions of objects
- Permanently delete noncurrent versions of objects

Configure:
- Days after object creation ( 100 days )
- Days after objects become noncurrent ( 1 day )
```

**Rule 2:** Cleanup Delete Markers
```
Actions:
- Delete expired object delete markers or incomplete multipart uploads

Configure:
- Delete expired object delete markers
- Delete incomplete multipart uploads ( 1 day )
```

jhmartin1 · Answer

https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-configuration-examples.html

> When you specify the Days tag, Amazon S3 automatically performs ExpiredObjectDeleteMarker cleanup when the delete markers are old enough to satisfy the age criteria.

No special  configuration required. You can either clean it up as part of the existing retire-current-objects rule, or create a special rule that explicitly targets expired delete markers sooner.

removing expired delete markers, how does it work?

Add your answer

Relevant content