AWS announces AWS Interconnect – multicloud (preview), providing simple, resilient, high-speed private connections to other cloud service providers. AWS Interconnect - multicloud is easy to configure and provides high-speed, resilient connectivity with dedicated bandwidth, enabling customers to interconnect AWS networking services such as AWS Transit Gateway, AWS Cloud WAN, and Amazon VPC to other cloud service providers with ease.
How can I audit deleted or missing objects from my Amazon S3 bucket?
I want to get information about an object or file that's missing from my Amazon Simple Storage Service (Amazon S3) bucket, and prevent future accidental deletions.
Resolution
To find out how an S3 object was deleted, review either server access logs or AWS CloudTrail logs.
Note: You must turn on logging for the bucket before the deletion event occurs. You receive logs only for events that occurred after you turned on logging.
Server access logs
Server access logs track Amazon S3 operations that you perform manually or as part of a lifecycle configuration. To turn on server access logging, see Enabling Amazon S3 server access logging. For more information on how to analyze server access logs, see Using Amazon S3 server access logs to identify requests.
Note: Amazon S3 delivers server access logs on a best-effort basis. Your server access logs might be incomplete.
CloudTrail logs
CloudTrail logs can track object-level data events in an S3 bucket, such as GetObject, DeleteObject, and PutObject. By default, CloudTrail records bucket-level events. To turn on CloudTrail logging for object-level events, see Enabling CloudTrail event logging for S3 buckets and objects. For more information on how to find specific events, see Why don't Amazon S3 object-level API actions appear in my CloudTrail Event history?
Note: Because object-level logging incurs additional charges, make sure to review the pricing for CloudTrail data events.
To prevent future accidental deletions, it's a best practice to use one of the following features:
- Turn on versioning to keep historical versions of an object.
- Turn on Cross-Region Replication of objects.
- Turn on MFA delete to require multi-factor authentication (MFA) when you delete an object version.
- Use AWS Backup to restore an object or file.
- Use S3 Object Lock to prevent the deletion of an object.
Related information
How do I use Athena to analyze my Amazon S3 server access logs?
- Topics
- Storage
- Language
- English
This article implies that S3 server access logging and CloudTrail data event logging would be equally effective at tracking object deletions. That is not correct. CloudTrail data events don't include the object keys or version IDs at all, when objects are deleted as part of a batch delete operation with the DeleteObjects API. I believe the management console uses DeleteObjects even to delete a single object or object version, making this a very typical situation.
Keys not being logged for batch deletes is mentioned as a limitation in this comparison table in documentation: https://docs.aws.amazon.com/AmazonS3/latest/userguide/logging-with-S3.html ("Logging of keys in a batch delete operation").
The solution is to use S3 server access logging to audit all object deletions, regardless of the method used to trigger each deletion, and not to rely solely on CloudTrail data event logging.
Please note that S3 server access logging is based on best-effort delivery [1]. In another word, it's informative but cannot be deemed as an auditing tool.
Reference: [1] https://docs.aws.amazon.com/AmazonS3/latest/userguide/ServerLogs.html#LogDeliveryBestEffort
Thank you for your comment. We'll review and update the Knowledge Center article as needed.
Relevant content
- asked a year ago
- asked 5 years ago
- asked 2 years ago
- asked 2 years ago
- asked 3 years ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 2 years ago