Skip to content

Applying Backup Tags to AWS Resources Triggered Unexpected Cost Increases in S3, CloudWatch, and CloudTrail

0

Hi AWS Team, we are performing a backup audit activity where we need to apply AWS Backup plans as tags (K–V pairs) to resources within an AWS account, including S3 buckets, DynamoDB tables, DocumentDB clusters, EFS volumes, Neptune clusters, and others. However, immediately after enabling the backup plans, we observed a significant cost spike in AWS CloudTrail and Amazon CloudWatch, as reported by AWS Cost Explorer.

Cost Explorer Parameters

  • Time: Standard
  • Date Range: 2025-12-01 to 2025-12-12
  • Granularity: Daily
  • Group By: Usage type
  • Service Filter: CloudWatch

We noticed a sharp increase in Usage Type: EU-DataProcessing-Bytes, where the daily cost increased from a few hundred dollars to several thousand dollars in last one week.

We believe multiple factors related to AWS Backup behaviour and CloudTrail settings may be contributing to the cost spike:

  1. Continuous Backups Disabled for S3: Since PITR (continuous backups) is disabled, AWS Backup must walk the entire S3 bucket for each periodic backup snapshot, scanning all objects to detect changes potentially generating large volumes of API calls.
  2. High Number of S3 GET-Equivalent API Calls: During the initial full backup, AWS Backup invokes GetObjectAcl and GetObjectTagging on every object. For subsequent backups, these calls are invoked only for changed/new objects. These requests are billed as S3 GET operations, which may explain increased S3-related costs.
  3. Excessive Delete Markers: Buckets with large numbers of delete markers can incur additional charges (counted as 128-KiB objects).
  4. Cost Impact of Glacier/IA Storage Classes: For objects stored in Glacier Instant Retrieval, S3-IA, or S3 One Zone-IA, AWS Backup must perform multiple retrieval calls, leading to retrieval fees.
  5. CloudTrail Data Events Enabled for S3: S3 Data Events are currently enabled in CloudTrail, which dramatically increases CloudTrail and CloudWatch Logs ingestion when backup operations generate large numbers of API calls.

CloudWatch Log Insights Investigation: We ran the following Log Insights query on the CloudTrail log group (stored in our centralized log archive account):

fields @timestamp, @message
| parse @message /"userIdentity"\s*:\s*\{[^}]*"arn"\s*:\s*"(?<userArn>[^"]+)"/
| filter userArn != ""
| stats count() as dailyCount by userArn, bin(@timestamp, 1d)
| sort dailyCount desc

The results showed a massive increase in dailyCount for multiple userArn values during the backup window, indicating that backup operations generated a surge in logged API activity.

To reduce costs without compromising security, we adjusted the CloudTrail Trail’s Data Event selectors as follows:

AWS Backup Search Data APIs
Log selector template: Exclude AWS service-initiated events
S3 Data Events
Log selector template: Exclude AWS service-initiated events

These changes reduce unnecessary logging of automated backup-related API calls while continuing to capture human-initiated actions.

Can the AWS Backup and CloudTrail/CloudWatch SMEs confirm, if this is really the cost effective solution following security as the primary requirement or will it double the cost spendings?

Please help.

1 Answer
0

Based on your description, you're experiencing a significant cost increase after implementing AWS Backup plans via tags, particularly in CloudTrail and CloudWatch costs. This is a common scenario when implementing backup solutions at scale.

Your analysis of the cost factors is accurate. When AWS Backup runs against S3 buckets, it does generate multiple API calls per object (including GetObjectAcl and GetObjectTagging) to properly back up the data. For 16 million objects, this could result in a substantial number of API calls. These operations are indeed billed as S3 GET operations, contributing to increased costs.

The CloudTrail data events for S3 are likely the primary cost driver here. When CloudTrail logs are sent to CloudWatch Logs, you incur two separate charges: the CloudTrail delivery charge (approximately $0.25/GB) and the CloudWatch Logs ingestion charge (approximately $0.50/GB). With the volume of API calls generated by AWS Backup operations, this can quickly escalate costs.

Your approach to exclude AWS service-initiated events from CloudTrail Data Event logging is a cost-effective solution that maintains security requirements. By filtering out automated service actions while still capturing human-initiated activities, you're addressing the root cause of the cost spike. This approach allows you to:

  1. Maintain visibility into user actions for security and compliance
  2. Significantly reduce the volume of logged events from automated backup processes
  3. Lower both CloudTrail and CloudWatch costs without compromising security posture

Additionally, consider these optimization strategies:

  1. For AWS Backup plans, create multiple backup plans with identical rules but different resource assignments to stay within the 100-resource limit per backup plan
  2. Use tag-based resource assignments strategically to manage large numbers of resources
  3. Implement a tagging strategy where new resources automatically receive the appropriate backup tags
  4. Ensure the IAM role used for tag-based backup plans has the necessary permissions

This combined approach should effectively reduce your costs while maintaining your security and backup requirements.
Sources
AWS Backup Plan tagging strategy when applying plans to multiple resources | AWS re:Post
EU-S3-Egress-Bytes | AWS re:Post
AWS CloudTrail pricing
AWS S3 Backup encryption cost | AWS re:Post

answered 9 months ago

EXPERT

reviewed 9 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.