AWS Builder Center is the official home for builders on AWS. Share and read what others are working on, follow people who inspire you, explore training and workshops, and find tools to support what you're building.
S3 bucket cross-region usage
Dear S3 Users,
I faced a small issue, and as I'm un-experienced in S3 and the deeper settings, I would kindly ask for your support on the following topic: I have set up an S3 bucket in the London data centre, and we started to use it in our organisation. Everything is fine so far, but we noticed some increased fees, which we did not expect. Just then we learned: that if you download outside your region, it costs way more. In our case, downloading from Romania from a London data centre costs ~16 USD compared to the 0.08 USD which would cost if you downloaded within the region, as Romania is not in the same region. This caused us some trouble. We are using Cyberduck for managing downloads and uploads, there is no VPN or self-hosted server in the organisation. We have learned that CloudFront could be a solution, but to use Cyberduck, the data shall be public, which we cannot allow. I kindly ask your support, how I fix this situation? In my understanding, if Cloudfront is set up, we can download outside of the S3 region without higher costs (ofc, within the CloudFront data budget), but we need the Cyberduck or any other file browser option to manage uploads and downloads. I know Mountain Duck might be a solution, but we do not want to pay for that one. Can you please recommend a file browser, which can handle CloudFront with non-public domains? Or Any other approach, which allows us to store and manage our data using S3? I know that, the Windows FSx could be a solution, but it's too expensive per month. THank you for your kind support!
- Language
- English
asked 2 years ago1.3K views
- Newest
- Most votes
- Most comments
Hello, I would like to answer some questions based on my experience with S3.
As you said:
I have set up an S3 bucket in the London data centre, and we started to use it in our organisation. Everything is fine so far, but we noticed some increased fees, which we did not expect.
In our case, downloading from Romania from a London data centre costs ~16 USD compared to the 0.08 USD which would cost if you downloaded within the region, as Romania is not in the same region. This caused us some trouble. We are using Cyberduck for managing downloads and uploads, there is no VPN or self-hosted server in the organisation.
The information I got from it is that:
- you have an office in Romania
- you have an S3 bucket in Europe (London)
- you use a software called Cyberduck on your office computer to download files from S3 over the Internet
Is my understanding correct? If there is any misunderstanding, please comment.
Now I will correct some of your descriptions based on the above understanding
Just then we learned: that if you download outside your region, it costs way more.
According to AWS S3 pricing, cross-region transfer fees are only charged when files are transferred from Europe (London) to services in other AWS regions.
For example, an EC2 in Europe (Paris) downloads the contents of an S3 bucket in Europe (London).
So if you download the contents of the S3 bucket from the office in Romania through the Internet, you should then charge for the transfer from S3 to the Internet.
In our case, downloading from Romania from a London data centre costs ~16 USD compared to the 0.08 USD which would cost if you downloaded within the region, as Romania is not in the same region.
AWS customers receive 100GB of data transfer out to the internet free each month, aggregated across all AWS Services and Regions (except China and GovCloud). The 100 GB free tier for data transfer out to the internet is global and does not apply separately or individually to AWS Regions.
If your charges skyrocket, check your bill to see if the amount of data transferred from S3 to the Internet exceeds this limit.
answered 2 years ago
From your description, I can give the following suggestions:
- Use Cloudfront with S3 to reduce transfer costs(Cloudfront includes 1TB of free transfer quota, and the transfer fee per GB thereafter is slightly lower than that of S3), but Cloudfront is public, you can also use signed URLs or signed Cookies to restrict access to files
- Compress your files as much as possible. The final traffic charges for both S3 and Cloudfront depend on the size of the downloaded file. The smaller the file, the lower the fee.
- If possible, use AWS EC2 in Europe (London) to view and process data. There is no data transfer fee between EC2 and S3 in the same region. Do not download data locally. Processing data directly from EC2 (Windows) can save data transfer fees, but you need to consider the additional fees of EC2.
From what I know, I don't have any other suggestions, maybe you can submit a support ticket to AWS to get more help, or use Cloudflare R2 (compatible with AWS S3 API, without egress fees)
answered 2 years ago
Thank you for your clarification, it does help a lot! We had a contractor in Romania, who downloaded the data to work with it. As we work with bigger datasets commonly, like 100GB+ (civil engineering field), it may happen in the future, to repeat this. Does any AWS service provide a solution for this at a reasonable price?