- Newest
- Most votes
- Most comments
The issue you're experiencing with Claude 3.5 Sonnet v2 on Amazon Bedrock is related to the default quota limits set for the model. These quotas are designed to manage resource allocation and prevent overuse, but they can be restrictive for some use cases.
For Claude 3.5 Sonnet v2, the default quota for on-demand InvokeModel requests per minute is indeed very low compared to some other models. This limit is set at the account level and is not adjustable through the standard Service Quotas console process, which is why you see the "not adjustable" label and unavailable checkboxes in the quota management interface.
However, this doesn't mean you're completely stuck with this limit. To increase your quota for on-demand InvokeModel requests, you'll need to contact AWS Support directly. Here's what you can do:
- Visit the AWS Support Center and create a support case.
- In your request, specify that you need an increase for the "On-demand InvokeModel requests per minute" quota for the Anthropic Claude 3.5 Sonnet v2 model.
- Clearly explain your use case, detailing why you need higher quotas for your application.
- Provide information about your current usage and the performance issues you're facing due to the current limits.
- Specify the desired quota increase you're requesting.
AWS will review your request, but approval is not guaranteed and may take some time. They need to ensure proper resource allocation across all users.
In the meantime, you might consider using a multi-account strategy as a temporary solution, since token limits are at the account level. For high-throughput requirements, exploring Provisioned Throughput options could be an alternative, though this comes with additional costs.
Remember that AWS typically looks for accounts to be utilizing a high percentage of their current quota before approving significant increases, so be prepared to justify your need for the increased capacity.
Sources
Can not increate invokeModel request to Claude 3.5 Sonnet v2 | AWS re:Post
Anthropic Claude Usage Tiers Using Bedrock | AWS re:Post
Relevant content
- asked 9 months ago
- AWS OFFICIALUpdated 10 months ago
- AWS OFFICIALUpdated 3 years ago
