claude-3-haiku-20240307 is not supported on this API. Please use the Messages API instead

Question

Hi, I operate in the `eu-west-1` region and I want to use the bedrock batch [API](https://docs.aws.amazon.com/bedrock/latest/userguide/batch-inference-example.html). According to docs, the anthropic claude-3 model is supported in `eu-west-1` region. When I use the code below, the batch job finishes successfully but the result file e.g `result.jsonl.output` is like following

```
{"modelInput":{"prompt":"

Human: Sample prompt 40

Assistant:","max_tokens_to_sample":100},"error":{"errorCode":400,"errorMessage":""claude-3-haiku-20240307" is not supported on this API. Please use the Messages API instead.","retriable":false},"recordId":"0000040XYZ"}
```

my code
```
_client.create_model_invocation_job(
                roleArn=,
                modelId="anthropic.claude-3-sonnet-20240229-v1:0",
                jobName="job-name",
                inputDataConfig=inputDataConfig,
                outputDataConfig=outputDataConfig,
            )
```

the input I provide for this batch job is a `.jsonl`file with hundred lines like following
```
{"recordId": "0000001XYZ", "modelInput": {"prompt": "

Human: Sample prompt 1

Assistant:", "max_tokens_to_sample": 100}}
...
```

re:Post Agent · Answer

The error you're encountering suggests that you're trying to use the Claude 3 model with an API that doesn't support it. The Claude 3 models, including Claude 3 Sonnet, require the use of the Messages API rather than the older Text Completion API.

For batch inference with Claude 3 models, you need to adjust your input format to match the Messages API structure. Instead of using the "prompt" field, you should structure your input as a list of messages. Here's an example of how your input .jsonl file should look:

```json
{"recordId": "0000001XYZ", "modelInput": {"anthropic_version": "bedrock-2023-05-31", "max_tokens": 100, "messages": [{"role": "user", "content": "Sample prompt 1"}]}}
```

Make sure to update all entries in your input file to follow this format. The "anthropic_version" field is required for Claude 3 models.

Additionally, ensure that you have the necessary permissions and have been granted access to use the Claude 3 model in your region. If you continue to face issues, you may need to check your IAM permissions, verify your model access in the AWS Bedrock console, or contact AWS support for further assistance.

Remember that the Bedrock batch inference feature might still be in preview, which could lead to some inconsistencies or limitations across different regions and models. If you encounter persistent issues, consider using the real-time inference API as an alternative until batch inference is fully supported for your specific use case.  
**Sources**  
[Accessing Amazon Bedrock Claude-3-Haiku and Claude-3-Sonnet models via Lambda Functions | AWS re:Post](https://repost.aws/questions/QUXt-V9PaiRUq-EfnYBjelwg/accessing-amazon-bedrock-claude-3-haiku-and-claude-3-sonnet-models-via-lambda-functions)  
[problem with anthropic claude haiku | AWS re:Post](https://repost.aws/questions/QUpeoj_4DxSZKUeEi2rdFCDg/problem-with-anthropic-claude-haiku)  
[Bedrock API invocation error - on demand throughput isn's supported | AWS re:Post](https://repost.aws/questions/QUEU82wbYVQk2oU4eNwyiong/bedrock-api-invocation-error-on-demand-throughput-isn-s-supported)

claude-3-haiku-20240307 is not supported on this API. Please use the Messages API instead

Add your answer

Relevant content