Skip to content

Using Nova Lite mode. I get this error: Your request rate is too high. Reduce the frequency of requests. Check your Bedrock model invocation quotas to find the acceptable frequency. Status Code: 429

0

Hi, I have already checked quotas I have on my account, but cannot find limits for specific Nova Lite model. I am making 6 requests to an agent and immediately i get this error at the 6th. I have already checked that the total input tokens is below my quota.

Use case: extracting data from 3 pdf files (I send content directly) and I made two agents, each with it's own backend_action tool. It hallucinated a lot using one agent with 2 tools in actions.

I am stuck and not sure which way to go now so any ideas are appreciated. Thank you.

asked a year ago521 views

1 Answer
0
The "Your request rate is too high" error you're encountering with Nova Lite in AWS Bedrock is related to request frequency rather than total token usage. This is a rate limiting issue that can occur even when you're within your total token quota.

Here are some approaches to address this issue:

1. **Implement request rate limiting**: Add small delays between your requests or implement a queue mechanism to manage the flow of requests to the Bedrock API. This can help prevent exceeding the rate limits.

2. **Request a quota increase**: Contact AWS Support specifically for a quota increase for the Nova Lite model. When submitting your request, include:
   - The specific quota name you need increased (e.g., "On-demand InvokeModel requests per minute")
   - The model ID (Nova Lite)
   - The AWS Region for which you need the increase
   - A clear explanation of your use case

3. **Implement retry logic with exponential backoff**: Add retry mechanisms in your code that include increasing delays between retries when you encounter this error.

4. **Consider batching**: If possible, try to combine smaller requests into batch calls to reduce the frequency of API calls.

5. **Consider Provisioned Throughput**: For high-throughput requirements, you might want to look into purchasing Provisioned Throughput for your model, which can increase the rate and number of tokens that can be processed.

6. **Optimize your agent design**: Since you mentioned creating two agents with separate backend_action tools to avoid hallucinations, you might need to further optimize your agent design or consider if there are ways to reduce the number of concurrent requests.

Note that AWS Bedrock is a relatively new service, and quota policies may be somewhat restrictive by default. Due to high demand, AWS may prioritize quota increase requests for customers who are consistently using their existing quota allocation.
  

Sources
How to increase request rate in AWS Bedrock | AWS re:Post
Bedrock Test Agent quota error | AWS re:Post
Question regarding Error invoking RetrieveAndGenerateCommand: Your request rate is too high. Reduce the frequency of request: | AWS re:Post
Troubleshoot Amazon Bedrock on-demand resource 429 Throttling error | AWS re:Post

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.