Amazon Bedrock - Question about RetrieveAndGenerateCommand

Question

I'm using the RetrieveAndGenerateCommand through the API and i need help,
 Below is the template for my RetrieveAndGenerateRequest:

```
{
    return { // RetrieveAndGenerateRequest
        input: { // RetrieveAndGenerateInput
            text: "Hi", // required
        },
        retrieveAndGenerateConfiguration: { // RetrieveAndGenerateConfiguration
            knowledgeBaseConfiguration: { // KnowledgeBaseRetrieveAndGenerateConfiguration
                knowledgeBaseId: "STRING", // required
                modelArn: "STRING", // required
                retrievalConfiguration: { // KnowledgeBaseRetrievalConfiguration
                    vectorSearchConfiguration: { // KnowledgeBaseVectorSearchConfiguration
                        filter: { // RetrievalFilter Union: only one key present
                            equals: { // FilterAttribute
                                key: "STRING", // required
                                value: "STRING", // required
                            },
                        },
                    },
                },
            },
            type: "KNOWLEDGE_BASE", // required
        },
    }
};
```
When I ask questions based on data within the knowledge base, the model responds accurately. However, for generic inputs like "Hi," the model replies with "Sorry, I am unable to assist you with this request."

I want to customize the prompt in the request and i don't know how.
I attempted to add the following under both knowledgeBaseConfiguration and generationConfiguration, but it didn’t work:

```
promptTemplate: {
          textPromptTemplate: "Human: You are a helpful AI assistant. Respond to the following: Hi"
}
```

Wael · Answer

Can you test the same format as here : 
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/bedrock-agent-runtime/client/retrieve_and_generate.html

```
client.retrieve_and_generate(
    input={
        'text': 'string'
    },
    retrieveAndGenerateConfiguration={
        'externalSourcesConfiguration': {
            'generationConfiguration': {
                'additionalModelRequestFields': {
                    'string': {...}|[...]|123|123.4|'string'|True|None
                },
                'guardrailConfiguration': {
                    'guardrailId': 'string',
                    'guardrailVersion': 'string'
                },
                'inferenceConfig': {
                    'textInferenceConfig': {
                        'maxTokens': 123,
                        'stopSequences': [
                            'string',
                        ],
                        'temperature': ...,
                        'topP': ...
                    }
                },
                'promptTemplate': {
                    'textPromptTemplate': 'string'
                }
            },
...
```

Can you double check this part :

```
 'promptTemplate': {
                    'textPromptTemplate': "Human: You are a helpful AI assistant. Respond to the following: Hi
                }
```

Didier Durand · Answer

Hi,

RetrieveAndGenerate will produce the answer you get when the chunks retrieved from the database are empty or irrelevant to the question.

So, you should investigate what you get back in the RAG part part of the Knowledge Database service.

For that, you may want to log the activity the way I describe in my article: https://repost.aws/articles/AR-LV1HoR_S0m-qy89wXwHmw/the-leverage-of-llm-system-prompt-by-knowledge-bases-for-bedrock-in-rag-workflows

Best,

Didier

Amazon Bedrock - Question about RetrieveAndGenerateCommand

Add your answer

Relevant content