Skip to content

meta.llama3-2-11b-instruct-v1:0 not supported for on-demand throughput?

0

The following code produces the error listed below even though everything I've read says meta.llama3-2-11b-instruct-v1:0 should be available for on demand?

$modelId = 'meta.llama3-2-11b-instruct-v1:0'; // Replace with your model ID $payload = [ 'input_text' => 'Your input text for the model here', // Replace with model-specific input ];

// Convert payload to JSON
$jsonPayload = json_encode($payload);

if (!$jsonPayload) {
    throw new Exception("Failed to encode JSON payload: " . json_last_error_msg());
}

// Call the InvokeModel method
$result = $client->invokeModel([
    'modelId' => $modelId,
    'body' => $jsonPayload,              // Ensure payload is a JSON string
    'contentType' => 'application/json', // Content-Type header
]);

Error: Error executing "InvokeModel" on "https://bedrock-runtime.us-east-1.amazonaws.com/model/meta.llama3-2-11b-instruct-v1%3A0/invoke"; AWS HTTP error: Client error: POST https://bedrock-runtime.us-east-1.amazonaws.com/model/meta.llama3-2-11b-instruct-v1%3A0/invoke resulted in a 400 Bad Request response: {"message":"Invocation of model ID meta.llama3-2-11b-instruct-v1:0 with on-demand throughput isn’t supported. Retry yo (truncated...) ValidationException (client): Invocation of model ID meta.llama3-2-11b-instruct-v1:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model. - {"message":"Invocation of model ID meta.llama3-2-11b-instruct-v1:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model."}

asked 2 years ago2.3K views

1 Answer
1

It appears that the Meta Llama 3.2 11B model (meta.llama3-2-11b-instruct-v1:0) is not currently supported for on-demand throughput in Amazon Bedrock as you expected. The error message suggests that this particular model requires the use of an inference profile rather than on-demand invocation.

To use this model, you would need to create an inference profile that includes the Meta Llama 3.2 11B model and then use the ID or ARN of that profile in your API call instead of directly specifying the model ID.

It's worth noting that model availability and supported invocation methods can vary. While some Llama models are available for on-demand use, this specific 11B version seems to have different requirements.

If you need to use this exact model, you'll need to set up an inference profile through the AWS Bedrock console or API. Alternatively, you could check if other versions of Llama 3.2 are available for on-demand use in your region, such as the 90B version which has been reported to work with on-demand invocation in some cases.

Always refer to the latest AWS documentation or contact AWS support for the most up-to-date information on model availability and invocation methods, as these can change over time.
Sources
How to get access of Meta3.2 Multi-modal models on bedrock. | AWS re:Post

answered 2 years ago

EXPERT

reviewed 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.