How to replicate retrieve_and_generate with Retrieve + Converse

To replicate retrieve_and_generate functionality using Retrieve + Converse APIs, you need to carefully configure both steps to ensure consistency in results.

The key difference you're noticing is that retrieve_and_generate has default settings that limit the number of chunks used, while the retrieve API returns up to the max_num_results you specify. Here's how to align them:

1. Configure your Retrieve call properly:

• Set the numberOfResults parameter to match what retrieve_and_generate would use (typically a smaller number like 10)
• Use the same searchType (HYBRID is often the default)
• Apply the same metadata filters if applicable

2. Format the retrieved results properly for Converse:

• Create a template that formats the retrieved chunks similar to how retrieve_and_generate would present them
• A common pattern is to use a $search_results$ placeholder in your prompt template

3. Structure your Converse call:

• Include a system prompt that instructs the model how to use the retrieved information
• Pass the retrieved chunks in the appropriate format
• Use the same model as you would in retrieve_and_generate

For example, if using retrieve_and_generate with a Knowledge Base, you might set:

retrievalConfiguration: {
vectorSearchConfiguration: {
numberOfResults: 10,
overrideSearchType: 'HYBRID'
}
}

Then in your separate Retrieve + Converse approach, ensure your retrieve call uses the same numberOfResults and searchType parameters, and format the results appropriately for the Converse API.

You may also need to experiment with the prompt template to ensure the model processes the retrieved information similarly to how retrieve_and_generate would handle it.
Sources
Metadata filtering for tabular data with Amazon Bedrock Knowledge Bases | AWS Machine Learning Blog
Creating a retrieve-and-generate RAG evaluation job - Amazon Bedrock
Community | Build Retrieval Augmented Generation (RAG) based Generative AI (GenAI) application with Amazon Bedrock
Using Amazon Nova for Multimodal RAG - Amazon Nova