Passer au contenu

How to increase AWS Bedrock quota for the specific model?

0

Hi, Currently I have enabled a model Claude 3.5 Sonnet V2 but both RPM and TPM is very low compared to our Anthropic rates. Also it does not seem I can request to increase the quota from the console.

Specifically - On-demand InvokeModel requests per minute for Anthropic Claude 3.5 Sonnet V2

Kind regards, Abyl

demandé il y a 2 ans2,6 k vues

2 réponses
0

To increase the AWS Bedrock quota for a specific model like Claude 3.5 Sonnet V2, you'll need to follow a different process than the standard quota increase request through the console. For on-demand model invocation quotas, including the "On-demand InvokeModel requests per minute" for Anthropic Claude 3.5 Sonnet V2, these are not adjustable through the regular Service Quotas dashboard.

Instead, you have a few options:

  1. You can submit a request through the AWS Support Center or use the limit increase form. When doing so, make sure to provide:

    • The name of the quota you want to increase (On-demand InvokeModel requests per minute)
    • The specific model ID (Anthropic Claude 3.5 Sonnet V2)
    • The AWS Region for which you need the quota increase
  2. In your request, clearly explain your use case and why you need a higher quota. This can help AWS better understand your needs and potentially speed up the approval process.

It's important to note that while you can request an increase, approval is not guaranteed and may take some time. AWS reviews these requests carefully to ensure proper resource allocation.

If you need an immediate solution while waiting for the quota increase, you might consider using a multi-account strategy, as the token limit per minute is an account-level limit. However, this is not ideal for long-term use.

Alternatively, for high-throughput requirements, you could consider purchasing Provisioned Throughput, though this comes with significant costs and may not be suitable for all use cases.
Sources
increasing Bedrock quotas without provisioning throughput | AWS re:Post
Quotas for Amazon Bedrock - Amazon Bedrock
Troubleshoot Amazon Bedrock on-demand resource 429 Throttling error | AWS re:Post

répondu il y a 2 ans

EXPERT

vérifié il y a 2 ans

EXPERT

révisé il y a 2 ans

0

Hello.

Please contact AWS Support from the URL below.
https://support.console.aws.amazon.com/support/home#/

When making an inquiry, I think there will be no problem if you select the items below.
a

EXPERT

répondu il y a 2 ans

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.