1 Answer
- Newest
- Most votes
- Most comments
0
The "Your request rate is too high" error you're encountering with Nova Lite in AWS Bedrock is related to request frequency rather than total token usage. This is a rate limiting issue that can occur even when you're within your total token quota.
Here are some approaches to address this issue:
1. **Implement request rate limiting**: Add small delays between your requests or implement a queue mechanism to manage the flow of requests to the Bedrock API. This can help prevent exceeding the rate limits.
2. **Request a quota increase**: Contact AWS Support specifically for a quota increase for the Nova Lite model. When submitting your request, include:
- The specific quota name you need increased (e.g., "On-demand InvokeModel requests per minute")
- The model ID (Nova Lite)
- The AWS Region for which you need the increase
- A clear explanation of your use case
3. **Implement retry logic with exponential backoff**: Add retry mechanisms in your code that include increasing delays between retries when you encounter this error.
4. **Consider batching**: If possible, try to combine smaller requests into batch calls to reduce the frequency of API calls.
5. **Consider Provisioned Throughput**: For high-throughput requirements, you might want to look into purchasing Provisioned Throughput for your model, which can increase the rate and number of tokens that can be processed.
6. **Optimize your agent design**: Since you mentioned creating two agents with separate backend_action tools to avoid hallucinations, you might need to further optimize your agent design or consider if there are ways to reduce the number of concurrent requests.
Note that AWS Bedrock is a relatively new service, and quota policies may be somewhat restrictive by default. Due to high demand, AWS may prioritize quota increase requests for customers who are consistently using their existing quota allocation.
Sources
How to increase request rate in AWS Bedrock | AWS re:Post
Bedrock Test Agent quota error | AWS re:Post
Question regarding Error invoking RetrieveAndGenerateCommand: Your request rate is too high. Reduce the frequency of request: | AWS re:Post
Troubleshoot Amazon Bedrock on-demand resource 429 Throttling error | AWS re:Post
answered a year ago
Relevant content
asked a year ago
