Amazon-Titan Express inconsistent answers

Question

Hello,

I invoked repeatedly the Amazon-Titan Express java SDK InvokeModel API, and the responses are very different conceptually.

The request was to recommend dress and activities for some weather forecast for next 12 hours given as JSON. 
The concluded, on short, responses were:
1. The model cannot provide recommendations for clothing and activities
2. The weather forecast for temperature will range from 18.27°C to 12.01°C
3. The recommended dress for the next 12 hours is casual, comfortable clothing

The requested and desired response is 3. It seams that this variation in output is due to the price of Express, because with Premier, the answer was 3.
Is there a way to receive consistent answers from Express after tunning the input parameters (`temperature`, `topP`), or from changing to Converse API ?

Thank you,  
Mihai ADAM

re:Post Agent · Accepted Answer

The inconsistency you're experiencing with Amazon Titan Express model responses is not uncommon and can be attributed to the nature of large language models and the specific configuration of the Express model. However, there are ways to potentially improve consistency:

1. Adjusting input parameters: You can try tuning the `temperature` and `topP` values in the `textGenerationConfig` of your request. Lower `temperature` values (closer to 0) will make the model's outputs more deterministic and focused, while higher values (closer to 1) increase randomness. Similarly, a lower `topP` value will make the model consider only the most probable tokens, potentially leading to more consistent outputs.

2. Using the Converse API: The Converse API could potentially provide more consistent results, as it's designed for more structured, conversational interactions. It allows you to include system prompts and maintain context across multiple turns, which might help in getting more consistent responses.

3. Improving prompt engineering: You could try to make your prompt more specific and structured. For example, you could explicitly ask for clothing recommendations based on the weather forecast, which might guide the model to more consistently provide the type of answer you're looking for.

4. Consider using Premier: As you've noticed, the Premier model seems to provide more consistent results for your use case. If consistency is crucial for your application, and if it's feasible within your constraints, using the Premier model might be the most reliable solution.

It's important to note that while these strategies can help improve consistency, some level of variation is inherent in the nature of these models, especially with more affordable options like Express. The trade-off between cost and consistency is a common consideration when choosing between different tiers of language models.

Remember to test these adjustments thoroughly to find the configuration that works best for your specific use case.  
**Sources**  
[Amazon Titan Text models - Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-titan-text.html)  
[Using the Converse API - Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-call.html)

Amazon-Titan Express inconsistent answers

Add your answer

Relevant content