Bedrock relevance check not working

Question

Hi all, 
When I'm executing my bedrock api with only relevance check turned on, it is not taking any intervention action even when the response/guard content is very different from the  query.

This is my code below and I've attached the results directly underneath:

```

response = client.apply_guardrail(
    guardrailIdentifier='*****',  # Replace with your guardrail ID
    guardrailVersion='DRAFT',  # Replace with the correct version number
    source='OUTPUT',  # Evaluating relevance by comparing input and output
    content=[
        {
            'text': {
                'text': 'What are the benefits of using AI in healthcare?',
                'qualifiers': ['query']
            }
        },
        {
            text: {
                'text': 'I like pink horses.',
                'qualifiers': ['guard_content']
            },
 
            text: {
                'text': " Ai is a great way to diagnose diseases, analyze medical data, and transcribe medical notes etc.", 
                "qualifiers": ['grounding_source']
            }
        }
    ]
) 
```

`"ResponseMetadata": {
        "RequestId": "01d05eaf-f62c-46da-bac3-8aea36a5c8f3",
        "HTTPStatusCode": 200,
        "HTTPHeaders": {
            "date": "Tue, 28 Jan 2025 00:09:02 GMT",
            "content-type": "application/json",
            "content-length": "615",
            "connection": "keep-alive",
            "x-amzn-requestid": "01d05eaf-f62c-46da-bac3-8aea36a5c8f3"
        },
        "RetryAttempts": 0
    },
    "usage": {
        "topicPolicyUnits": 0,
        "contentPolicyUnits": 0,
        "wordPolicyUnits": 0,
        "sensitiveInformationPolicyUnits": 0,
        "sensitiveInformationPolicyFreeUnits": 0,
        "contextualGroundingPolicyUnits": 1
    },
    **"action": "NONE",
    "outputs": [],
    "assessments": [`**

re:Post Agent · Answer

It appears that the relevance check in your Amazon Bedrock Guardrails configuration is not functioning as expected. There are a few potential reasons for this:

1. Configuration: Ensure that your guardrail is properly configured with the relevance check enabled. Double-check your guardrail settings in the Amazon Bedrock console.

2. Content structure: The way you're structuring the content in your API call might not be optimal for the relevance check. Make sure you're correctly designating the 'query', 'guard_content', and 'grounding_source' in your input.

3. Threshold sensitivity: The relevance check might not be sensitive enough to detect the discrepancy between your input and output. You may need to adjust the threshold settings for the relevance check in your guardrail configuration.

4. Guardrail version: You're using the 'DRAFT' version of the guardrail. Ensure that your changes are saved and that you're using the correct version of the guardrail.

5. API usage: Verify that you're using the correct API call for your use case. The ApplyGuardrail API is designed to assess text using pre-configured guardrails without invoking foundation models.

To troubleshoot this issue:

1. Review your guardrail configuration in the Amazon Bedrock console.
2. Ensure your content structure in the API call is correct.
3. Try adjusting the sensitivity of your relevance check.
4. Confirm you're using the intended version of your guardrail.
5. Double-check that you're using the appropriate API for your use case.

If the issue persists after these steps, you may want to contact AWS support for further assistance.  
**Sources**  
[Use the ApplyGuardrail API in your application - Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-use-independent-api.html)  
[How Amazon Bedrock Guardrails works - Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-how.html)

Bedrock relevance check not working

Add your answer

Relevant content