Batch Inference, Anthrophic Claude 3.5, and Tool Configs

Question

Hi! I an unable to successfully execute jobs with Batched Inference via the `get_model_invocation_job` action with tool configurations. I am using `anthropic.claude-3-5-sonnet-20240620-v1:0 `I can validate my batches are executing successfully, with correct output structure in S3, but the jobs themselves all result in errors. This is due to input format with the following error:

`errorMessage":"Malformed input request: #: subject must not be valid against schema {\"required\":[\"messages\"]}#/tools/0: extraneous key [input_schema] is not permitted#/tools/0: extraneous key [description] is not permitted#/tools/0/name: #/tools/0/name: extract_cancer_timeline is not a valid enum value#/tools/0/type: function is not a valid enum value#/tools/0/type: #/tools/0: required key [display_height_px] not found#/tools/0: required key [display_width_px] not found#/tools/0: extraneous key `

I have 2 hard criteria: 1) I need to use batching, 2) I need to use tool configuration and enforce adherent output.

I am currently passing in my tool configuration like:
`"tools": [tool_config],` 
where tool config is like: 
`{
    "type": "function",
    "name": "...",
    "description": "...",
    "input_schema": {"type": "object", "properties": {...}}}`

I have searched through multiple documentation sources [https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/implement-tool-use, https://docs.aws.amazon.com/bedrock/latest/userguide/batch-inference-data.html, etc. ], plus stackoverflow, and Anthropics own Claude search chatbot- and cannot find explicit documentation for the combined usage of Batch Invocation + Claude + Tool Usage + JSON adherent.

Any tips per the error I have received?

re:Post Agent · Answer

Based on the error message you're receiving, it appears there's a mismatch between the tool configuration format you're using and what's expected by the Amazon Bedrock batch inference service for Claude 3.5 Sonnet.

The error indicates several issues with your tool configuration:
        
        1. Extraneous keys like "input_schema" and "description" are not permitted
        2. The tool name "extract_cancer_timeline" is not a valid enum value
        3. "function" is not a valid value for the "type" field
        4. Required keys "display_height_px" and "display_width_px" are missing

This suggests that the tool configuration format for batch inference with Claude 3.5 Sonnet differs from the standard Claude API format. For batch inference with Amazon Bedrock, you need to structure your input differently.

When formatting your JSONL file for batch inference with Claude 3.5 Sonnet, each record should include:
        - A unique "recordId" 
        - A "modelInput" field that contains the appropriate request format

For tool usage in batch inference, you'll need to adapt your tool configuration to match the expected format. Based on the error message, it appears the tools configuration for batch inference requires:
        - Different type values than "function"
        - Display dimensions (height and width in pixels)
        - No "input_schema" or "description" fields in the format you're currently using

I recommend trying a different tool configuration format that includes the required display dimensions and uses the supported type values. You might also need to check if there are specific enum values allowed for tool names.

Unfortunately, the specific format for tool configurations in batch inference with Claude 3.5 Sonnet isn't explicitly documented in the provided sources. You may need to contact AWS Support for the exact schema required for tool configurations in batch inference scenarios.
      
**Sources**  
[Batch inference general operation error | AWS re:Post](https://repost.aws/questions/QUamrH1jdJSNGk_cPyjJ0r6A/batch-inference-general-operation-error)  
[AWS Bedrock Batch Inference Operation Not Allowed | AWS re:Post](https://repost.aws/questions/QU5TzybUeyT7qYwmAk6ID72w/aws-bedrock-batch-inference-operation-not-allowed)  
[Format and upload your batch inference data - Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/batch-inference-data.html)

Batch Inference, Anthrophic Claude 3.5, and Tool Configs

Add your answer

Relevant content