Claude 3.5 Sonnet V2 On-demand InvokeModel requests per minute quota increase

Question

Hi, I was trying to figure out why this inference API is giving me 429 randomly

Region:  `us-east-2` \
ModelId:  `us.anthropic.claude-3-5-sonnet-20241022-v2:0` \
Mode:  Cross region inference

So was going through quotas and saw this:

```
Applied account-level quota value: 1
AWS default quota value: 50
Adjustability: Not adjustable
Quota code: L-79E773B3
```

And more text saying: This quota isn't adjustable and you can't request a quota increase. If you have questions or concerns, please contact Support Center .

I was trying to raise a service ticket for quota increase but in the dropdowns, I don't see any option for On demand quota increase instead provisioned output comes up (NI)
Thoughts on this?

1. I am not sure if quota is the sole reason of getting 429
2. Is there any possibility that cross region inference might be causing 429
3. How do I request for quota increase for on demand cross inference models

re:Post Agent · Answer

Based on the information provided, there are a few important points to consider regarding your Claude 3.5 Sonnet V2 On-demand InvokeModel requests and the 429 errors you're experiencing:

1. The quota you've mentioned (L-79E773B3) with an applied value of 1 and a default value of 50 is not adjustable. This means you cannot request an increase for this specific quota through the standard quota increase process.

2. The 429 errors you're encountering may indeed be related to quota limits, but it's not necessarily the sole reason. These errors typically indicate that you're exceeding the allowed request rate.

3. Cross-region inference itself is not likely to be the direct cause of 429 errors. However, it's possible that cross-region requests might have different quota limits or be subject to additional constraints compared to same-region requests.

4. For Claude 3.5 Sonnet V2, the current inference quota is 50 requests and 400,000 tokens per minute. This quota is not adjustable at the moment.

5. Regarding requesting a quota increase for on-demand cross-region inference models, unfortunately, if the quota is marked as "Not adjustable," there isn't a standard process to request an increase through the Service Quotas console.

Given these points, here are some recommendations:

1. Double-check all your quotas related to Amazon Bedrock and Claude 3.5 Sonnet V2 in the Service Quotas console, as there might be other quotas affecting your usage.

2. Implement retry logic with exponential backoff in your application to handle occasional 429 errors gracefully.

3. If you consistently need higher throughput, you might want to explore other model options that support Provisioned Throughput, although this isn't currently available for Claude 3.5 Sonnet.

4. As suggested in the quota description, if you have ongoing concerns or questions about the non-adjustable quota, the best course of action would be to contact AWS Support directly. They may be able to provide more specific guidance or escalate your case if necessary.

5. Consider optimizing your application to work within the current quota limits, such as batching requests or implementing client-side rate limiting to stay within the 50 requests per minute limit.  
**Sources**  
[AWS Bedrock Claude 3.5 Sonnet throttled randomly "Too many tokens, please wait before trying again." | AWS re:Post](https://repost.aws/questions/QU2tqWTuP-Sje1rX_uHV0vqw/aws-bedrock-claude-3-5-sonnet-throttled-randomly-too-many-tokens-please-wait-before-trying-again)  
[Understand why there are service quotas on my account | AWS re:Post](https://repost.aws/knowledge-center/service-quota-new-account)  
[Submit quota increase | AWS re:Post](https://repost.aws/knowledge-center/lightsail-quota-increase)

Claude 3.5 Sonnet V2 On-demand InvokeModel requests per minute quota increase

Add your answer

Relevant content