Skip to content

Can S3 ListBucket requests result in Data Transfer Out charges?

0

I am trying to understand S3 charges I'm seeing in my account. I have a bucket with hundreds of thousands of objects, and a service (outside of AWS), that is repeatedly using "aws s3 sync" against that bucket, hundreds or thousands of times a day.

s3 sync needs to list the contents of the bucket in order to know what to sync, and it can only list 1,000 objects at a time, so a single sync call will result in hundreds of LIST requests for this bucket. I will be charged for these requests at $0.005 per thousand. This is easily understandable so far.

However, I am also seeing very significant egress charges that I can only attribute to these LIST requests. In cost explorer, the charges for usage types Requests-Tier1 and DataTransfer-Out-Bytes exactly sum to the charges for the ListBucket operation. Note that the charge for GetObject, which is the only operation I thought would cause egress charges, is always $0.

S3 charges broken down by usage type

S3 charges broken down by API operation.

The only thing that actually retrieves objects from this bucket are EC2 instances in the same region and account, which I expect to be free, and the cost data backs that up ($0 for GetObject). I have confirmed this with server access logging, nothing outside of AWS is doing anything except ListBucket and PutObject.

I cannot find anything in AWS documentation or elsewhere on the internet that discusses this issue. The "Ask AWS" AI (which I don't trust but decided to ask for fun), is very confidently assuring me that "metadata returned by LIST operations doesn't count as billable data transfer (egress) because it's considered part of the API operation itself, which you're already paying for through the LIST request charges". This is also what I would assume from the documentation, but it's not what I'm seeing in practice.

3 Answers
0

Hi,

Looking at the image you shared, "DataTransfer-Out-Bytes" shows charges of $8.39.

Your understanding is correct for GetObject from EC2 (same region) and ListBucket request pricing.

BUT the confusion is: When your external service calls ListBucket, AWS sends back metadata (object names, sizes, timestamps, ETags, etc.) in the API response for all objects. This metadata travels over the internet to your service and is charged at ~$0.09/GB as egress charges. That's the $8.39 "DataTransfer-Out-Bytes" charge you're seeing.

Image 2 shows ListBucket = $10.24, which is $8.39 (data transfer) + $1.85 (request fees), confirming this.

following shows Data transfer out to the internet :->

https://docs.aws.amazon.com/cur/latest/userguide/cur-data-transfers-charges.html#data-transfer-out-internet

https://aws.amazon.com/s3/pricing/

Best

answered a year ago

  • Thanks, my confusion was that I think the average person would assume "data transfer" refers to the data stored in the objects themselves. I still can't find anything that explicitly says that metadata transfer is counted for egress pricing, although it does seem to be the case. For storage pricing I think it's different - you only get charged for the size of the objects, not the size of storing metadata about the objects (https://stackoverflow.com/a/76433288).

  • I agreed, there is not doc I can find for metadata transfer, thanks for sharing stackoverflow. Your confusion was reasonable. Most people would assume the same thing.

0

Hi Jeff,

Great analysis — your instinct to distrust the AI answer was spot on. Here's the definitive answer with an explanation of why.

Yes, S3 LIST Responses Cause Data Transfer Out Charges

Any bytes that leave AWS to the internet are counted as Data Transfer Out, including API response bodies from LIST operations. This is not specific to S3 — it's how AWS meters all internet-bound traffic at the network level.

When your external service calls ListObjectsV2, the S3 API returns an XML response body containing metadata for up to 1,000 objects (key name, ETag, size, last modified, storage class). This XML response travels over the internet to your service, and those bytes are metered as egress.

Why This Adds Up Fast

Let's do the math for your scenario:

  • Per-object metadata in a LIST response: ~200-300 bytes (key name + ETag + size + timestamp + storage class)
  • Per LIST call with 1,000 objects: ~200-300 KB of XML response data
  • Your bucket: hundreds of thousands of objects → ~200-500 LIST calls per sync
  • Thousands of syncs per day: Let's say 1,000 syncs/day

Daily egress from LIST alone: 1,000 syncs x 300 LIST calls x 250 KB = ~75 GB/day

At $0.09/GB, that's ~$6.75/day or ~$202/month — which aligns with the charges you're seeing.

Why GetObject Shows $0

Your EC2 instances fetching objects are in the same region as the bucket. Data transfer between S3 and EC2 in the same region is free. But your external service calling LIST is going over the internet — that's where the egress charges come from.

How to Verify in CUR

If you enable the AWS Cost and Usage Report, you can see this broken down:

  • Usage type: {region}-DataTransfer-Out-Bytes — this is your egress
  • Operation: ListBucket — confirms it's from LIST, not GET
  • The Data Transfer charges reference documents the usage types

Cost Optimization Options

Here are several ways to dramatically reduce these charges:

1. Run s3 sync from an EC2 instance in the same region (Best Option)

Move your sync operations to an EC2 instance or Lambda in the same region as the bucket. Same-region traffic between S3 and EC2/Lambda is free — both request charges and data transfer. You could use a tiny t4g.nano ($0.0042/hr) as a sync relay.

2. Use S3 Inventory instead of repeated LIST calls

S3 Inventory generates a daily or weekly CSV/ORC/Parquet listing of all objects in your bucket. Your external service can download this single inventory file (~50 MB for hundreds of thousands of objects) instead of making hundreds of LIST calls thousands of times per day. Cost: $0.0025 per million objects listed.

3. Use S3 Event Notifications + SQS

Instead of polling with s3 sync, set up S3 Event Notifications to push notifications when objects are created/deleted. Your service only fetches what actually changed. This eliminates LIST calls entirely.

4. Reduce sync frequency

If thousands of syncs per day is more than needed, reducing to every 5-10 minutes instead of continuously would cut costs proportionally.

5. Use a CloudFront distribution (if applicable)

For repeated access patterns, CloudFront can cache LIST responses and reduce both request charges and egress. However, this adds complexity and may not help for sync use cases.

Summary

SourceCharge TypeWhy
External service → S3 LISTRequest fee ($0.005/1,000)API call pricing
S3 LIST response → External serviceData Transfer Out ($0.09/GB)XML metadata bytes crossing internet boundary
EC2 (same region) → S3 GET$0.00Same-region, free

The key takeaway: all API response bytes that cross the AWS network boundary to the internet incur egress charges, not just object data from GET requests. This is documented under S3 Data Transfer pricing in the "Data Transfer OUT From Amazon S3 To Internet" section — it applies to any bytes sent to the internet, regardless of the API operation.

References:

Hope this clears things up!

AWS

answered 4 months ago

-1

Yes, S3 ListBucket operations can result in Data Transfer Out charges when the listing requests are made from outside AWS.

When you use "aws s3 sync" from a service outside of AWS, the metadata returned by the ListBucket operations counts as data transferred out to the internet, which incurs egress charges. This is why you're seeing charges for both Requests-Tier1 (for the API calls themselves) and DataTransfer-Out-Bytes (for the metadata being transferred out).

The reason your GetObject operations show $0 is because those requests are coming from EC2 instances in the same AWS Region as your S3 bucket, which is free as per AWS's pricing model. Data transferred from S3 to EC2 instances in the same region doesn't incur charges.

For buckets with hundreds of thousands of objects, the metadata returned by repeated ListBucket operations can add up to significant egress charges, especially when these operations are performed hundreds or thousands of times daily from outside AWS. Each ListBucket response includes metadata about the objects (such as names, sizes, ETags, and timestamps), which can become substantial when multiplied by many requests.

To reduce these costs, you might consider:

  1. Reducing the frequency of sync operations
  2. Using more targeted sync commands that limit the scope of objects being listed
  3. If possible, running the sync operations from within AWS (like from an EC2 instance in the same region)
    Sources
    AWS DataSync Cost - AWS
    Understanding data transfer charges - AWS Data Exports
    AWS Billing reports for Amazon S3 - Amazon Simple Storage Service

answered a year ago

  • This is exactly the opposite of what the documentation LLM told me, so I can't trust this answer. I will wait for a human with expertise in this area.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.