S3 bandwidth limit
A customer is managing the cloud infrastructure for a large company. Recently they had an issue of a fairly large chunk of the bill caused by company employees downloading data from S3. The customer is asking how can he restrict:
- S3 bandwidth used per user
- Amount of data downloaded per user
AFAIK, there are no native options to achieve this. The possible solutions I thought of:
- Bandwidth can be restricted by using a proxy server on the customer end (capable of SSL interception)
- Amount of data:
- can be potentially reported on using customer proxy reporting services
- collect the bucket access logs and send them to ElasticSearch / Splunk. Then use their reporting capabilities to report on top S3 download users. Based on the report results, contact the user to find out the reason.
Any other options I am missing?
[EDIT] They are using AWS CLI and SDKs to get the data, so CloudFront is less suitable. DX is an option to reduce costs.
- 최신
- 최다 투표
- 가장 많은 댓글
You are correct, there is no way to control this natively within AWS. If they are downloading content from S3, I would look at using Direct Connect or CloudFront to reduce the data transfer costs, if possible. These may reduce costs enough that it is no longer a concern.
If it still is a concern, I would consider using the access logs to calculate the amount of data transfer by each user/role and then internally-charge the teams. This will give them the responsibility to manage their usage, without imposing a hard limit.
