Python Error when prompting LLM in Bedrock via Apigee API Endpoint

Question

Using requests and AWS4Auth I am receiving the below error when prompting a response the Bedrock using InvokeModel:

Response status code: 200
Response from Bedrock via Apigee:
{
  "Output": {
    "__type": "com.amazon.coral.service#UnknownOperationException"
  },
  "Version": "1.0"
}

------------

Here is the request JSON payload:

Request Payload:
{
  "schemaVersion": "messages-v1",
  "modelId": "us.amazon.nova-lite-v1:0",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "text": "In 5-6 sentences, summarize the basic principles of quantum mechanics"
        }
      ]
    }
  ],
  "system": [
    {
      "text": "You are a positive, helpful, and funny AI assistant. Insert funny comments into responses."
    }
  ],
  "inferenceConfig": {
    "max_new_tokens": 500,
    "top_p": 0.9,
    "top_k": 20,
    "temperature": 0.7
  }
}

Thanks

re:Post Agent · Answer

It appears that you're encountering an error when trying to invoke a model in Amazon Bedrock using an API Gateway (Apigee) endpoint. The error message suggests that there's an unknown operation exception, which could be due to a few reasons:

1. Incorrect API endpoint: Ensure you're using the correct Bedrock runtime API endpoint for invoking models.

2. Payload format: The payload format you're using seems to be a mix of different model invocation styles. For Amazon Bedrock's Nova model, you should use the Converse API format instead of the InvokeModel format.

3. Authentication: Make sure your AWS4Auth is correctly set up with the appropriate credentials and region.

4. Model availability: Verify that you have access to the Nova Lite model in your AWS account and region.

To resolve this, try modifying your payload to match the Converse API format for Nova:

```python
payload = {
    "modelId": "amazon.nova-lite-v1:0",
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "text": "In 5-6 sentences, summarize the basic principles of quantum mechanics"
                }
            ]
        }
    ],
    "inferenceConfig": {
        "maxTokens": 500,
        "temperature": 0.7,
        "topP": 0.9,
        "topK": 20
    }
}
```

Also, double-check your API endpoint and ensure you're using the Converse operation instead of InvokeModel. If the issue persists, verify your AWS credentials and permissions, and make sure you have the necessary access to use the Nova model in Amazon Bedrock.  
**Sources**  
[Invoke Amazon Nova on Amazon Bedrock using Bedrock's Converse API - Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-runtime_example_bedrock-runtime_Converse_AmazonNovaText_section.html)  
[Build generative AI applications on Amazon Bedrock with the AWS SDK for Python (Boto3) | AWS Machine Learning Blog](https://aws.amazon.com/blogs/machine-learning/build-generative-ai-applications-on-amazon-bedrock-with-the-aws-sdk-for-python-boto3/)

Python Error when prompting LLM in Bedrock via Apigee API Endpoint

Add your answer

Relevant content