Audit of S3 Bucket Deletion

0

We recently had an issue where one of our buckets mysteriously was deleted and we cannot track how this happened. Is there some management level logging that keeps a record of events like the deletion of S3 buckets?

We were able to recreate the bucket and have enabled server access logging in case this happens again, but is this where it would log the deletion activity or would it be someplace else?

asked 3 years ago3239 views
4 Answers
0

Hi jswanson,

From the notes, I understand that you are trying to keep a record of events like the deletion of S3 buckets.

In order to track down deletion of your S3 bucket, you can review AWS CloudTrail logs. The event "DeleteBucket" is logged in CloudTrail whenever a user delete an S3 bucket. You can use CloudTrail to check for the past 90 days. The event "DeleteBucket", which is logged in CloudTrail, should at least provide information in terms of the IAM role used to delete the bucket.

For AWS CloudTrail logs: https://docs.aws.amazon.com/AmazonS3/latest/userguide/cloudtrail-logging.html
Note: Logging must be enabled on the bucket before the deletion event occurs. You receive logs only for events that occurred after logging was enabled. I would recommend setting up a trail for multi-region to keep track of all event in all regions.

I would recommend having a look at this as well for CloudTrail pricing information: https://aws.amazon.com/cloudtrail/pricing/

The S3 bucket could have been deleted because the policies or permissions were too open and thus someone outside your organization accessed the bucket and deleted it. When editing the permissions to your S3 buckets, make sure to be cautious because it could result in your bucket being publicly accessible. (Also as a side note, buckets need to be empty in order to delete them so this means someone first emptied it and then deleted it.)
You can have a look at this to better understand S3 access and permissions: https://aws.amazon.com/blogs/aws/amazon-s3-block-public-access-another-layer-of-protection-for-your-accounts-and-buckets/

With regards to protection against deletions: https://d0.awsstatic.com/whitepapers/protecting-s3-against-object-deletion.pdf

You can also reach out to our Support team for assistance on this, they can check your account to see what is going on and assist you. You can create the case via the support center: https://console.aws.amazon.com/support/home?#

I hope this helps.

amz99
answered 3 years ago
0

Thank you for your response. Unfortunately, when I look through the CloudTrail logs, I do not see any DeleteBucket events in the list.

To test this, I created a new bucket and deleted it, then tried to verify what I had done in the CloudTrail logs. In my test, I still could not see a "DeleteBucket" event, even when specifically searching for it (via Event name filter).

Is there some kind of configuration required to be able to see these types of events?

answered 3 years ago
0

Edit: Apparently it seems to take some time for these events to be logged to CloudTrail. After about 20 minutes or so the events became searchable.

Unfortunately, it does not explain why we lost our S3 bucket yesterday, since there is no DeleteBucket event for the time range in which it went dark, just from my tests this morning.

However my original question is answered, thank you. I will reach out to support regarding this missing S3 bucket.

answered 3 years ago
0

As a final update, we discovered that the bucket was deleted due to slight quirk with the DeleteObjectRequest method. The SDK we are using is from before there was a change to address this quirk (Oct 8, 2018), so the issue was happening.

We have logic in place that makes an API call to delete an object and occasionally the DeleteObjectRequest is called without a key. Instead of throwing an exception, it would attempt to delete the S3 bucket instead. We finally ran into the issue where the bucket was removed because prior to yesterday, there was always an object present in the bucket, thus causing every attempt to delete the bucket to fail. The bucket was recently cleaned up and contained no objects, so the latest attempt to delete the bucket was successful.

We have done the following to prevent it from happening again:

  • add an object to the bucket with intention to keep it there indefinitely
  • remove delete bucket permissions from the role that was calling the method to delete the object
  • add code to our own software that checks the key length before the API call is made and throws an exception instead

Ultimately, updating to the latest SDK should be enough (includes a changeset that checks for an empty key, and throws an exception if true), and we plan to do this as well.

answered 3 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions