Lambda python code to get the total storage size of s3

1

Lambda code for that function :-

import boto3

def lambda_handler(event, context): # Create an S3 client s3_client = boto3.client('s3')

# Get a list of all buckets in the account
response = s3_client.list_buckets()

# Initialize a variable to track the total storage size
total_storage_size = 0

# Iterate through each bucket
for bucket in response['Buckets']:
    # Get the bucket name
    bucket_name = bucket['Name']

    # Get the size of the bucket
    bucket_size = get_bucket_size_recursive(s3_client, bucket_name)

    # Add the bucket size to the total storage size
    total_storage_size += bucket_size

# Print or log the total storage size
print(f"Total storage size for all buckets: {total_storage_size} GB")

# Return a response
return {
    'statusCode': 200,
    'body': f"Total storage size for all buckets: {total_storage_size} GB"
}

def get_bucket_size_recursive(s3_client, bucket_name): # Get the size of the bucket recursively bucket_size = 0

# Paginate through the objects in the bucket
paginator = s3_client.get_paginator('list_objects_v2')
pages = paginator.paginate(Bucket=bucket_name)

for page in pages:
    # Sum the size of objects in the bucket
    bucket_size += sum([obj['Size'] for obj in page.get('Contents', [])])

# Paginate through the subfolders (common prefixes)
pages = paginator.paginate(Bucket=bucket_name, Delimiter='/')

for page in pages:
   
    for common_prefix in page.get('CommonPrefixes', []):
        subfolder_size = get_bucket_size_recursive(s3_client, bucket_name + '/' + common_prefix['Prefix'])
        bucket_size += subfolder_size

return bucket_size / (1024 ** 3)  # Convert to GB

This is the code im used for to get the storage size of the s3. but i didn't get the exact storage match for this.

Test Event Name TestEvent

Response { "statusCode": 200, "body": "Total storage size: 128.38 GB" }

but in the storage its 280.80 GB. its not getting exact match, help me with this

sandy
已提問 6 個月前檢視次數 398 次
1 個回答
2

Instead of iterating through each file, which would take some time and increase your S3 costs (as you pay per request), it would be easier to just get S3 metrics from CloudWatch with a single call: https://github.com/boto/boto3/issues/212 or some other ways: https://serverfault.com/questions/84815/how-can-i-get-the-size-of-an-amazon-s3-bucket

profile picture
已回答 6 個月前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南