Bedrock knowledge base converse API token limit

Since there is a bug. could you contact the support team

Regarding the issue with the maxTokens validation, the Bedrock team is aware of it and has plans to increase the maxTokens validation number accordingly. You can follow the blog and CHANGELOG for updates on this matter.

For now, as a temporary workaround, you can either remove the inferenceConfig from the code snippet or specify a value less than or equal to 4096 for maxTokens. This should allow the code to work correctly until the maxTokens validation limit is increased in a future release.

There appears to be some confusion in your question regarding token limits for the Amazon Bedrock Converse API and knowledge base retrieval. The Converse API itself does not impose a specific token limit of 4096 for all models when using a knowledge base. Token limits can vary depending on the specific foundation model being used.

The Converse API is designed to provide a consistent way to interact with various foundation models on Amazon Bedrock, removing the complexity of adjusting for model-specific differences. Each model may have its own token limits and capabilities, which are not uniformly set at 4096 tokens for knowledge base retrieval.

Regarding future plans to increase supported token limits, Amazon regularly updates and improves its services. However, there is no specific information available about plans to increase token limits for the Converse API or knowledge base integration. Token limits are often determined by the underlying foundation models rather than the API itself.

If you're experiencing limitations with token counts, it's recommended to review the documentation for the specific foundation model you're using, as capabilities can vary between models. Additionally, you may want to explore different foundation models available through Amazon Bedrock to find one that better suits your token requirements for your particular use case.
Sources
Foundation Model Choice and Customization - Amazon Bedrock Developer Experience - AWS
Community | Customizing AI Behavior: System prompts and inference parameters in Bedrock's Converse API