Skip to content

AWS Bedrock - errorCodeThrottlingException

0

Hi, I deployed an AI application that uses AWS Bedrock foundational models to aid in answering questions for my application users. Starting yesterday, I thought it was the AWS outage, the application users started reporting an error that says "Too many requests, please wait before trying again." When I look at the logs in Cloudwatch, I see my my model invocations are getting errorCodeThrottlingException We are barely using this application, we only have 5 foundational models exposed and today the forecasted cost for the month is less than $40. How can I determine what limit the application is hitting? Thanks, Rodolfo

Field Value @aws.region us-east-1 @ingestionTime 1761051608937 @log :BedrockCloudWatchLoggingLogGroup @logStream aws/bedrock/modelinvocations @message {"timestamp":"2025-10-21T13:00:07Z","accountId":"043309335824","identity":{"arn":"arn:aws:iam:::user/@"},"region":"us-east-1","requestId":"70bc8dd8-e030-4923-8205-dc48363b0c39","operation":"ConverseStream","modelId":"anthropic.claude-3-5-sonnet-20240620-v1:0","errorCode":"ThrottlingException","schemaType":"ModelInvocationLog","schemaVersion":"1.0"} @timestamp 1761051607000 accountId

errorCode ThrottlingException identity.arn arn:aws:iam:: modelId anthropic.claude-3-5-sonnet-20240620-v1:0 operation ConverseStream region us-east-1 requestId 70bc8dd8-e030-4923-8205-dc48363b0c39 schemaType ModelInvocationLog schemaVersion 1.0 timestamp 2025-10-21T13:00:07Z

asked a year ago370 views

1 Answer
0

You're experiencing a "ThrottlingException" with Amazon Bedrock, specifically with the Claude 3.5 Sonnet model. This error occurs when you exceed the service quotas for the model, even if your overall usage seems low.

For Claude models on Amazon Bedrock with on-demand provisioning, there are two types of limits that are enforced concurrently:

  1. Requests Per Minute (RPM)
  2. Tokens Per Minute (TPM)

For example, Claude models typically have quotas like 100 requests and 200,000 tokens per minute. When either of these limits is exceeded, Bedrock returns a throttling exception.

To determine which specific limit you're hitting:

  1. Check your CloudWatch metrics at 1-minute increments to identify throttling patterns. Look specifically at these metrics:
  • Invocations
  • InvocationThrottles
  • InputTokenCount
  • OutputTokenCount
  1. Review your current Amazon Bedrock service quotas in the AWS Management Console to understand your limits.

Note that the Service Quota dashboard shows only configured quotas, not real-time usage. To monitor real-time usage, you must use CloudWatch.

Even though your monthly cost is low, you might be sending too many requests or tokens in short bursts that exceed the per-minute limits. This can happen even with previously successful batches when usage exceeds quotas at peak times.

To resolve this issue, you can:

  1. Implement retry logic with exponential backoff
  2. Space out your requests to stay within per-minute limits
  3. Request a quota increase through AWS Support if needed
  4. Consider using Provisioned Throughput if you need dedicated capacity (though this is more costly)
  5. Optimize your prompts to use fewer tokens
    Sources
    Troubleshoot Amazon Bedrock on-demand resource 429 Throttling error | AWS re:Post
    Building Agentic AI app with Strands Agents & MCP | AWS re:Post
    [AWS Bedrock] ThrottlingException occurs randomly for Claude-2.1 Runtime | AWS re:Post

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.