ModelErrorException: Model produced invalid sequence as part of ToolUse

Question

I keep getting this ModelErrorException whenever my user input gets around 1k tokens. The tool calling works great when the input is smaller. The docs claim 30k token context window.

Any thoughts?

```
{
  "name": "AI_APICallError",
  "url": "https://bedrock-runtime.us-east-2.amazonaws.com/model/inference-profile%2Fus.amazon.nova-lite-v1%3A0/converse",
  "requestBodyValues": {
    "system": [
      {
        "text": ""
      }
    ],
    "inferenceConfig": {
      "temperature": 0
    },
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "text": ""
          }
        ]
      }
    ],
    "toolConfig": {
      "tools": [
        {
          "toolSpec": {
            "name": "json",
            "description": "Respond with a JSON object.",
            "inputSchema": {
              "json": {
                "type": "object",
                "properties": {
                  "people": {
                    "type": "array",
                    "items": {
                      "type": "object",
                      "properties": {
                        "givenName": {
                          "type": "string",
                          "description": "First name or identifier of the person"
                        },
                        "familyName": {
                          "type": "string",
                          "description": "Last name or identifier of the person"
                        }
                      },
                      "required": [
                        "givenName",
                        "familyName"
                      ],
                      "additionalProperties": false
                    },
                    "description": "A flat list of all identified people in the transcript. Each element in the array should be an object with givenName and familyName."
                  }
                },
                "required": [
                  "people"
                ],
                "additionalProperties": false,
                "$schema": "http://json-schema.org/draft-07/schema#"
              }
            }
          }
        }
      ],
      "toolChoice": {
        "tool": {
          "name": "json"
        }
      }
    }
  },
  "statusCode": 424,
  "responseHeaders": {
    "connection": "keep-alive",
    "content-length": "123",
    "content-type": "application/json",
    "date": "Thu, 15 May 2025 23:03:04 GMT",
    "x-amzn-errortype": "ModelErrorException:http://internal.amazon.com/coral/com.amazon.bedrock/",
    "x-amzn-requestid": "7136b234-485f-465b-b6c3-e3f7424ab753"
  },
  "responseBody": "{\"message\":\"Model produced invalid sequence as part of ToolUse. Please refer to the model tool use troubleshooting guide.\"}",
  "isRetryable": false,
  "data": {
    "message": "Model produced invalid sequence as part of ToolUse. Please refer to the model tool use troubleshooting guide."
  }
}
```

re:Post Agent · Answer

The issue you're experiencing with the ModelErrorException when your user input approaches 1,000 tokens is likely related to the model's handling of tool use with larger inputs, rather than the overall context window size.

While the model may have a 30,000 token context window, there could be limitations specific to tool use functionality that are causing the error when processing larger inputs. The error message suggests that the model is having difficulty producing a valid sequence for the tool use component of your request.

To address this issue, you could try the following approaches:

1. Break down larger inputs into smaller chunks and process them separately if possible.

2. Simplify the tool specification or input schema to reduce complexity for larger inputs.

3. Experiment with different prompt structures or instructions that might help the model handle larger inputs more effectively when using tools.

4. Contact AWS support for more specific guidance on handling this error with the particular model you're using, as they may have additional insights or recommendations for optimizing tool use with larger inputs.

5. Consider using a different model or version that might handle larger inputs with tool use more effectively, if available.

If the issue persists, it would be helpful to review the model's documentation or reach out to AWS support for more detailed troubleshooting steps specific to the model and tool use functionality you're working with.  
**Sources**  
[Issue with Bedrock- Claude Sonnet 3.5 | AWS re:Post](https://repost.aws/questions/QUh5t0kGHIRdajNCDm1UHZxA/issue-with-bedrock-claude-sonnet-3-5)  
[Using bedrock: claude3 Haiku how can I enable more than 4096 output tokens? | AWS re:Post](https://repost.aws/questions/QUluuPYfhHQb-WQuXOUE3cgA/using-bedrock-claude3-haiku-how-can-i-enable-more-than-4096-output-tokens)  
[botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the InvokeModel operation: Input is too long for requested model. | AWS re:Post](https://repost.aws/questions/QUnNP6iMAnSIGym93PufUmFw/botocore-errorfactory-validationexception-an-error-occurred-validationexception-when-calling-the-invokemodel-operation-input-is-too-long-for-requested-model)

ModelErrorException: Model produced invalid sequence as part of ToolUse

Add your answer

Relevant content