AWS Bucket Resource Uses Breakdown
Hi Community,
I would like to know how I can find the resources from my S3 bucket that are being utilised on my website and in what bandwidth they are being fetched. I want this breakdown to understand the uses of the resources being uploaded on my bucket and are being fetched via the users. I would like to see which resources are being called and fetched multiple times and in what density. This will help me understand the AWS-provided bill breakdown as well because recently; we received the S3 bucket uses size, which did not match our server calculation of read and write of the resources.
Here is an image from Vercel which gives uses according to the resources:
Please let me know if there is a way we can check for the resourced that have been called in a specific month.
- Newest
- Most votes
- Most comments
While AWS S3 doesn't provide a direct feature to show resource utilization in the exact format you're looking for, there are several ways to gain insights into your S3 bucket usage and resource access patterns:
-
S3 Server Access Logging: You can enable server access logging for your S3 bucket. This will provide detailed records of all requests made to the bucket, including the requested object, requester, request time, and response status. This can help you identify which resources are being accessed and how frequently.
-
Amazon CloudWatch Metrics: S3 integrates with CloudWatch, allowing you to monitor various metrics such as NumberOfObjects, BucketSizeBytes, and RequestCount. While this doesn't provide object-level detail, it can give you an overview of bucket usage.
-
AWS CloudTrail: If enabled, CloudTrail can log S3 object-level API activity, which can provide information on which objects are being accessed and by whom.
-
S3 Storage Lens: This feature provides organization-wide visibility into object storage usage and activity. It can help you discover outliers and optimize storage.
-
Amazon Athena: You can use Athena to query your S3 access logs stored in another S3 bucket. This allows you to run custom SQL queries to analyze your access patterns and resource usage.
-
Third-party tools: There are various third-party tools available that can provide more detailed analytics on S3 usage, similar to what you're seeing in the Vercel image.
To check for resources that have been called in a specific month, you would need to analyze your S3 server access logs or CloudTrail logs for that time period. You can use Amazon Athena to query these logs and generate reports on object access frequency and patterns.
Regarding the discrepancy between your S3 bucket usage size and your server calculations, keep in mind that S3 billing includes factors beyond just the storage of your objects. It also accounts for data transfer, requests, and features like versioning or replication if enabled. Review your detailed billing report and consider factors like multipart uploads, incomplete multipart uploads, or deleted objects that may still incur charges during the billing period.
For a more precise breakdown similar to what you're seeing in the Vercel image, you might need to implement custom logging or use a third-party analytics solution that integrates with S3. These solutions can provide more granular insights into resource usage and help you optimize your S3 costs and performance.
Sources
Welcome - AWS Resource Explorer
Listing Amazon S3 buckets - Amazon Simple Storage Service
Monitoring AWS Resource Explorer - AWS Resource Explorer
Community | Tidy Your Bedroom: Identify Unused Resources
Relevant content
- Accepted Answerasked a year ago
- asked 3 years ago
AWS OFFICIALUpdated 15 days ago- AWS OFFICIALUpdated 4 months ago
- AWS OFFICIALUpdated a year ago
