Nova Sonic Inconsistent Tool Calling

Hi there,

This behavior is actually the expected functionality of the tool implementation architecture. The standard tool execution workflow follows this sequence:

1. The calling application submits tool definitions and an initiating message to the language model
2. Upon matching a tool definition, the model generates a tool use request with appropriate parameters
3. The calling application extracts these parameters and routes them to the corresponding local tool function
4. The calling application can either:
   • Utilize the tool result directly
   • Return the result to the model for additional processing
5. The model then either generates a final response or initiates another tool request

The observed pausing occurs because the model is generating the tool request but not automatically executing it. The subsequent user prompt triggers the complete workflow of tool execution and response generation.

To resolve this, you'll need to implement an additional Converse API call that handles the tool request and returns the tool output to the user.

For detailed implementation guidance, please reference:

Tool workflow documentation: https://builder.aws.com/content/2hW5367isgQOkkXLYjp4JB3Pe16/intro-to-tool-use-with-the-amazon-bedrock-converse-api Code examples for tool request handling: https://docs.aws.amazon.com/bedrock/latest/userguide/tool-use-examples.html

This inconsistent tool calling behavior with Amazon Nova Sonic appears to be related to how the model decides when to use tools. The issue you're experiencing is that the model sometimes gathers the necessary parameters but doesn't immediately proceed to call the tool.

To address this issue, you can try the following approaches:

1. Use the toolChoice parameter to more explicitly control tool usage:

• Set toolChoice to "any" to force the model to select at least one available tool each time
• Set toolChoice to "tool" with a specific tool name to force the model to use that particular tool
• The default "auto" setting leaves tool selection entirely up to the model, which may be causing your inconsistency

2. Optimize your tool configurations:

• Ensure your tool descriptions are clear and specific
• Use descriptive tool names that clearly indicate their function
• Make sure your input schema is well-defined

3. Consider your temperature and top_p settings:

• While you've already set temperature and top_p to "greedy" levels (1.0), you might try reducing temperature further (closer to 0) to make the model's behavior more deterministic
• This can help ensure more consistent tool calling patterns

4. Improve your system prompt:

• Include explicit instructions about when tools should be called
• Specify the expected conversation flow
• Provide clear guidance on gathering parameters and immediately using tools once parameters are collected

The key issue appears to be that the model sometimes enters a "thinking" state after collecting parameters rather than proceeding directly to tool use. By using the toolChoice parameter and optimizing your prompts, you should be able to achieve more consistent tool calling behavior.
Sources
Tool choice with Amazon Nova models | AWS Machine Learning Blog
Invoking a tool - Amazon Nova