- Newest
- Most votes
- Most comments
You're experiencing a "ThrottlingException" with Amazon Bedrock, specifically with the Claude 3.5 Sonnet model. This error occurs when you exceed the service quotas for the model, even if your overall usage seems low.
For Claude models on Amazon Bedrock with on-demand provisioning, there are two types of limits that are enforced concurrently:
- Requests Per Minute (RPM)
- Tokens Per Minute (TPM)
For example, Claude models typically have quotas like 100 requests and 200,000 tokens per minute. When either of these limits is exceeded, Bedrock returns a throttling exception.
To determine which specific limit you're hitting:
- Check your CloudWatch metrics at 1-minute increments to identify throttling patterns. Look specifically at these metrics:
- Invocations
- InvocationThrottles
- InputTokenCount
- OutputTokenCount
- Review your current Amazon Bedrock service quotas in the AWS Management Console to understand your limits.
Note that the Service Quota dashboard shows only configured quotas, not real-time usage. To monitor real-time usage, you must use CloudWatch.
Even though your monthly cost is low, you might be sending too many requests or tokens in short bursts that exceed the per-minute limits. This can happen even with previously successful batches when usage exceeds quotas at peak times.
To resolve this issue, you can:
- Implement retry logic with exponential backoff
- Space out your requests to stay within per-minute limits
- Request a quota increase through AWS Support if needed
- Consider using Provisioned Throughput if you need dedicated capacity (though this is more costly)
- Optimize your prompts to use fewer tokens
Sources
Troubleshoot Amazon Bedrock on-demand resource 429 Throttling error | AWS re:Post
Building Agentic AI app with Strands Agents & MCP | AWS re:Post
[AWS Bedrock] ThrottlingException occurs randomly for Claude-2.1 Runtime | AWS re:Post
answered a year ago
Relevant content
asked a year ago
asked a year ago
asked 2 years ago
