How do I optimize prompts to achieve deterministic responses on Amazon Bedrock?

Resolution

If your Amazon Bedrock prompts don't function correctly, or you want better deterministic responses, then use the following solutions.

Select the right foundation model

Amazon Bedrock provides access to multiple pretrained foundation models (FMs), such as Amazon Titan, Anthropic Claude, and Mistral.

To find the right FM for your use case, take the following actions:

• Adjust model inference parameters, such as temperature and top-p, to reduce randomness in the generated text.
• Test different models. To perform a basic test, use the Compare mode feature on the Amazon Bedrock console's Chat playground.
• Use different models, and then compare the consistency of responses across multiple runs of the same prompt.
• Choose the model that has the most deterministic behavior for your use case.

Use model evaluation jobs

To perform an advanced test, use model evaluation jobs. To identify potential issues or biases in the prompt or training data, use an automatic model evaluation. To check the output consistency on the input data, use a model evaluation job that uses human workers.

Create clear and specific prompts

To create clear and specific prompts that generate the appropriate response, take the following actions:

• Write the prompt in different ways. For example, change the structure, keywords, or format to get more consistent responses from the model.
• Use prompts that limit the model's ability to generate open-ended or divergent outputs.
• Adjust the FM's inference parameters to check for variations in the outputs for each prompt.

Example of an effective prompt:

 ``` You are a professional proofreader. Please review the following text for any grammatical errors, spelling mistakes, or inconsistencies, and provide corrections:   
  
[Insert text to be proofread] ```

The preceding prompt clearly defines the model's role and provides specific instructions. It also limits the scope of the response, which increases the chances of deterministic and relevant outputs.

Customize your FM

If the pretrained FMs don't work for your specific requirements, then you can use continued pretraining and fine tuning to customize the FMs.

To fine tune a model, complete the following steps:

1. Prepare a dataset that represents the kinds of inputs that you want to use with the FM.
2. Fine tune or use continued pretraining to train the FM on your dataset. Adjust the custom model hyperparameters as needed.
3. Evaluate the fine-tuned FM's performance, and then repeat the steps until you get the appropriate results. For more information, see Guidelines for model customization.

Combine and refine FM outputs

Ensemble techniques

To get more accurate and reliable results, set up a workflow that combines outputs from multiple models to improve the final prediction's reliability and robustness. You can also add custom logic to analyze and refine the outputs.

To modify or remove non-deterministic elements from the outputs, use post-processing tasks.

Prompt chaining

Use chained prompts to build more sophisticated and capable generative AI applications. 

Break down your complex task into smaller, more manageable subtasks that have their own prompt. Then, combine the subtasks to form the complete complex task that you want the FM to accomplish. Use Amazon Bedrock with AWS Step Functions to arrange the prompts in a predefined order or by a defined set of rules.

To get started, see the amazon-bedrock-serverless-prompt-chaining repository on the GitHub website. For information about how to construct effective AI applications, see Build and orchestrate generative AI applications with Amazon Bedrock and Step Functions.

Set up guardrails

To control the responses from the FMs, set up guardrails for Amazon Bedrock. You can consistently apply the guardrails across multiple FMs. Use guardrails to block certain topics, filter out harmful or unpredictable content, remove undesirable words, and hide sensitive information. For more information, see Guardrails for Amazon Bedrock.

You can also customize blocked messages and use the built-in test window to test your guardrail configurations. To get controlled and predictable outputs, connect guardrails directly with the FMs when you generate responses.