S3 bandwidth limit

Question

A customer is managing the cloud infrastructure for a large company. Recently they had an issue of a fairly large chunk of the bill caused by company employees downloading data from S3. The customer is asking how can he restrict:

1. S3 bandwidth used per user
2. Amount of data downloaded per user

AFAIK, there are no native options to achieve this. The possible solutions I thought of:

1. Bandwidth can be restricted by using a proxy server on the customer end (capable of SSL interception)
2. Amount of data:
    - can be potentially reported on using customer proxy reporting services
    - collect the bucket access logs and send them to ElasticSearch / Splunk. Then use their reporting capabilities to report on top S3 download users. Based on the report results, contact the user to find out the reason.

Any other options I am missing?

\[EDIT]
They are using AWS CLI and SDKs to get the data, so CloudFront is less suitable. DX is an option to reduce costs.

Accepted Answer

You are correct, there is no way to control this natively within AWS. If they are downloading content from S3, I would look at using Direct Connect or CloudFront to reduce the data transfer costs, if possible. These may reduce costs enough that it is no longer a concern.

If it still is a concern, I would consider using the access logs to calculate the amount of data transfer by each user/role and then internally-charge the teams. This will give them the responsibility to manage their usage, without imposing a hard limit.

S3 bandwidth limit

Relevanter Inhalt