跳至內容

How to increase AWS Bedrock quota for the specific model?

0

Hi, Currently I have enabled a model Claude 3.5 Sonnet V2 but both RPM and TPM is very low compared to our Anthropic rates. Also it does not seem I can request to increase the quota from the console.

Specifically - On-demand InvokeModel requests per minute for Anthropic Claude 3.5 Sonnet V2

Kind regards, Abyl

已提問 2 年前檢視次數 2618 次

2 個答案
0

To increase the AWS Bedrock quota for a specific model like Claude 3.5 Sonnet V2, you'll need to follow a different process than the standard quota increase request through the console. For on-demand model invocation quotas, including the "On-demand InvokeModel requests per minute" for Anthropic Claude 3.5 Sonnet V2, these are not adjustable through the regular Service Quotas dashboard.

Instead, you have a few options:

  1. You can submit a request through the AWS Support Center or use the limit increase form. When doing so, make sure to provide:

    • The name of the quota you want to increase (On-demand InvokeModel requests per minute)
    • The specific model ID (Anthropic Claude 3.5 Sonnet V2)
    • The AWS Region for which you need the quota increase
  2. In your request, clearly explain your use case and why you need a higher quota. This can help AWS better understand your needs and potentially speed up the approval process.

It's important to note that while you can request an increase, approval is not guaranteed and may take some time. AWS reviews these requests carefully to ensure proper resource allocation.

If you need an immediate solution while waiting for the quota increase, you might consider using a multi-account strategy, as the token limit per minute is an account-level limit. However, this is not ideal for long-term use.

Alternatively, for high-throughput requirements, you could consider purchasing Provisioned Throughput, though this comes with significant costs and may not be suitable for all use cases.
Sources
increasing Bedrock quotas without provisioning throughput | AWS re:Post
Quotas for Amazon Bedrock - Amazon Bedrock
Troubleshoot Amazon Bedrock on-demand resource 429 Throttling error | AWS re:Post

已回答 2 年前

專家

已審閱 2 年前

專家

修訂 2 年前

0

Hello.

Please contact AWS Support from the URL below.
https://support.console.aws.amazon.com/support/home#/

When making an inquiry, I think there will be no problem if you select the items below.
a

專家

已回答 2 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。