AWS Bedrock nova pro concurrency limit

For AWS Bedrock Nova Pro, there are specific concurrency limits that govern how many requests can be processed simultaneously.

While the sources don't explicitly state the exact concurrent request limit for Nova Pro specifically, they do indicate that Amazon Bedrock enforces concurrency limits on model inference requests. These limits are separate from the requests per minute (RPM) quota that you mentioned has been increased to 2500 in Ohio.

When dealing with high concurrent traffic for Nova Pro, you should be aware that:

1. The default quota for Nova Pro in some regions (like Frankfurt) is only 50 requests per minute (RPM).

2. The on-demand model inference requests per minute quota for Nova Pro is typically marked as "not adjustable" through the standard Service Quotas console, though you've already received a custom increase.

3. Cross-region inference (which you've enabled) helps distribute traffic across multiple AWS Regions, increasing your overall throughput beyond single-region limits.

If you're experiencing throttling or need higher concurrency, you might consider:

• Implementing request queuing mechanisms between your application and the model
• Optimizing your application architecture, including batching requests where possible
• Contacting AWS Support directly if you need further increases beyond what's already been granted

For specific concurrency needs beyond what's currently allocated, you should contact AWS Support directly as they can provide guidance tailored to your specific use case and account.
Sources
AWS Bedrock : Nova Pro with high concurrent traffic | AWS re:Post
Bedrock Quota For Nova Pro | AWS re:Post
Requests to Invoke Agent in Bedrock are being queued | AWS re:Post
How to increase AWS Bedrock quota for the specific model? | AWS re:Post

To view service quotas for any Amazon Bedrock models , do one of the following:

1. Follow the steps at Viewing service quotas and select Amazon Bedrock as the service: https://docs.aws.amazon.com/servicequotas/latest/userguide/gs-request-quota.html
2. Refer to the Amazon Bedrock service quotas in the AWS General Reference: https://docs.aws.amazon.com/general/latest/gr/bedrock.html#limits_bedrock

You can follow get approval of a quota increase request to increase limits: https://docs.aws.amazon.com/bedrock/latest/userguide/quotas-increase.html.

Source: https://docs.aws.amazon.com/bedrock/latest/userguide/quotas.html