Does the bedrock runtime Converse API toolConfig use constrained sampling

Question

As a follow up to [this question](https://repost.aws/questions/QU9LcSgNTsSZmk0NffTQ7v2w/aws-bedrock-runtime-converse-or-invoke-to-get-response-in-a-given-json-schema-format), does the Converse API toolConfig mechanism use genuine constrained sampling / guided decoding, where a mask is applied to the vocabulary at each token generation to enforce the specified structure, or is the schema adherence ultimately probabilistic.

When I asked this question in re:Post, I got an AI response:
"ToolSpec Schema Adherence is Probabilistic
The Bedrock Converse API's toolSpec does not enforce constrained decoding. Schema adherence remains probabilistic - the model attempts to follow the JSON schema provided in the tool definition, but there's no guarantee of structural validity."

And I was directed to [this post](https://aws.amazon.com/blogs/machine-learning/structured-data-response-with-amazon-bedrock-prompt-engineering-and-tool-use/?utm_source=repost.aws&utm_medium=intelligent_search). However, I don't see a conclusive statement about this question here, so I was hoping somebody could answer definitively.

I was also directed to [this post](https://aws.amazon.com/blogs/machine-learning/introducing-structured-output-for-custom-model-import-in-amazon-bedrock/?utm_source=repost.aws&utm_medium=intelligent_search) for Custom Model Import, where it is explicitly stated that constrained sampling can be configured when invoking the custom model

re:Post Agent · Answer

Based on the available information, the Converse API's toolConfig does not use constrained sampling or guided decoding. The toolConfig field in the Converse API is used to define tools that the model can use when generating a response, but there is no indication that it applies vocabulary masking at each token generation step to enforce structural validity.

In contrast, Amazon Bedrock does offer genuine constrained sampling through structured output for Custom Model Import. This capability, also known as constrained decoding, introduces constraints during generation that limit token choices to only those that maintain structural validity. If a particular token would violate the schema, it is rejected and the model must select another allowed option. This real-time validation ensures consistent, machine-readable outputs without additional post-processing.

However, this structured output capability with constrained decoding is specifically mentioned in the context of Custom Model Import, not the standard Converse API toolConfig. The toolConfig in the Converse API appears to be a configuration mechanism for defining available tools, but the documentation does not indicate that it employs the same constrained decoding approach used in Custom Model Import's structured output feature.

Therefore, while Amazon Bedrock does support genuine constrained sampling through its structured output feature for Custom Model Import, the standard Converse API toolConfig does not appear to use this same mechanism.  
**Sources**  
[Introducing structured output for Custom Model Import in Amazon Bedrock | Artificial Intelligence](https://aws.amazon.com/blogs/machine-learning/introducing-structured-output-for-custom-model-import-in-amazon-bedrock/)  
[Using the Converse API - Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-call.html)  
[ConverseTokensRequest - Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseTokensRequest.html)

Does the bedrock runtime Converse API toolConfig use constrained sampling

Add your answer

Relevant content