Bedrock Responce : "Sorry, I am unable to assist you with this request."

Question

Hello,

I have a small Python code that makes the same API calls multiple times, always with the same prompt. Sometimes I get an incorrect response from the LLM: "Sorry, I can't help you with this request."

For some LLMs, this is always the case (llama3-3-70b-instruct-v1), for some it's only sometimes (mistral-large-2402-v1:0), for others it's never (claude-3-haiku-20240307-v1).

Example of a log with mistral-large-2402-v1:0

responce KO
```

{
    "ResponseMetadata": {
        "HTTPHeaders": {
            "connection": "keep-alive",
            "content-length": "301",
            "content-type": "application/json",
            "date": "Tue, 01 Apr 2025 15:50:56 GMT",
            "x-amzn-requestid": "d30762e1-d072-448d-9239-e7fd5cc58a09"
        },
        "HTTPStatusCode": 200,
        "RequestId": "d30762e1-d072-448d-9239-e7fd5cc58a09",
        "RetryAttempts": 0
    },
    "citations": [
        {
            "generatedResponsePart": {
                "textResponsePart": {
                    "span": {
                        "end": 50,
                        "start": 0
                    },
                    "text": "Sorry, I am unable to assist you with this request."
                }
            },
            "retrievedReferences": []
        }
    ],
    "output": {
        "text": "Sorry, I am unable to assist you with this request."
    },
    "sessionId": "46f8b074-b8fa-4dbd-bce7-c18ef83e35a9"
}

```

responce OK
```
{
    "ResponseMetadata": {
        "HTTPHeaders": {
            "connection": "keep-alive",
            "content-length": "70138",
            "content-type": "application/json",
            "date": "Tue, 01 Apr 2025 15:51:36 GMT",
            "x-amzn-requestid": "59832753-d62a-4a4d-9150-03952a2ced10"
        },
        "HTTPStatusCode": 200,
        "RequestId": "59832753-d62a-4a4d-9150-03952a2ced10",
        "RetryAttempts": 0
    },
    "citations": [
        {
            "generatedResponsePart": {
                "textResponsePart": {
                    "span": {
                        "end": 146,
                        "start": 0
                    },
                    "text": "Th\u00e8me 1: Strat\u00e9gie client; La f........ cl\u00e9s."
                }
            },
            "retrievedReferences": [
                {
                    "content": {
                        "text": "CCM2 : OK.  Billet ..... concerne.",
                        "type": "TEXT"
                    },
                    "location": {
                        "s3Location": {
...

]
        }
    ],
    "output": {
        "text": "Th\u00e8me 1: Strat\u00e9gie c .... rtage d'id\u00e9es."
    },
    "sessionId": "9b43ea18-4f71-41d9-84bb-c7db41551889"
}

re:Post Agent · Answer

The issue you're experiencing with inconsistent responses from different language models in Amazon Bedrock is not uncommon. Here are a few potential reasons and solutions for this problem:

1. Model-specific behavior: As you've noticed, some models (like llama3-3-70b-instruct-v1) consistently give the "Sorry, I can't help you with this request" response, while others (like claude-3-haiku-20240307-v1) don't. This could be due to differences in how these models are trained or integrated with Bedrock.

2. Input format: Ensure that your input prompt is formatted correctly for each specific model. Different models may have different requirements for prompt structure.

3. Token limits: Check if you're hitting token limits for certain models. If your prompt is too long, some models might not be able to process it properly.

4. API versioning: Make sure you're using the correct API version for each model. For example, Claude 3 models require the use of the Messages API rather than the older Text Completion API.

5. Permissions and access: Verify that you have the necessary permissions and access to use each model in your region.

6. Retry logic: Implement retry logic with exponential backoff and jitter in your code. This can help handle intermittent issues.

7. Error handling: Improve your error handling to catch and log specific error messages. This can provide more insight into why certain calls are failing.

8. Quota limits: Check if you're hitting any service quotas or rate limits, especially for models that fail intermittently.

To address these issues:

1. Review your input format for each model and ensure it matches the required structure.
2. Implement robust error handling and logging to capture more details about failed requests.
3. Add retry logic to your code to handle transient errors.
4. If problems persist with specific models, consider reaching out to AWS support for further assistance.

Remember that some features or models in Bedrock might still be in preview, which could lead to inconsistencies across different models and regions.
      
**Sources**  
[Python Error when prompting LLM in Bedrock via Apigee API Endpoint | AWS re:Post](https://repost.aws/questions/QUhms9CEeVRzues3mUlxtrBg/python-error-when-prompting-llm-in-bedrock-via-apigee-api-endpoint)  
[Bedrock Llama 3 empty string response problem | AWS re:Post](https://repost.aws/questions/QUiQzpqzf1S-WREgfGKcDTMA/bedrock-llama-3-empty-string-response-problem)  
[Troubleshooting Amazon Bedrock API Error Codes - Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html)  
[claude-3-haiku-20240307 is not supported on this API. Please use the Messages API instead | AWS re:Post](https://repost.aws/questions/QU2rBPakwcTK2XiE_rLnKEAg/claude-3-haiku-20240307-is-not-supported-on-this-api-please-use-the-messages-api-instead)

Bedrock Responce : "Sorry, I am unable to assist you with this request."

Add your answer

Relevant content