- Newest
- Most votes
- Most comments
Hi Deeba, thanks for posting your question! The detail helped me easily reproduce the issue.
I set up a test script to reproduce this with my own Bedrock Knowledge Base and Amazon OpenSearch Service (managed, not serverless) and the difference in behavior between the SDK and the AWS Management Console for both the 'in' and '=' operators and the field types you specified in your configuration.
I've identified two issues causing your filters to return 0 results:
Issue 1: Incorrect Field Type in OpenSearch
Your configuration shows:
Metadata field: metadata (type text in OpenSearch)
Problem: Bedrock's metadata filters require exact matching, which only works on keyword type fields. OpenSearch text fields are tokenized/analyzed for full-text search, breaking exact match filtering. This is why both console and SDK equals operators fail.
Fix: Recreate your OpenSearch index with metadata fields configured as type keyword. When you let Bedrock auto-create the index, it does this correctly. Alternatively, manually create an index with proper mappings where filterable metadata fields are type keyword, not text.
Issue 2: Incorrect Syntax for 'in' Operator
Your code shows:
metadata_filter = {"in": {"key": "program", "values": ["CAH"]}}
Problem: The in operator uses value (singular), not values (plural). The API will reject this with a parameter validation error.
Fix: Change to:
metadata_filter = {"in": {"key": "program", "value": ["CAH"]}} # "value" not "values"
Validation
I tested both issues with a working Knowledge Base using OpenSearch Managed Cluster:
- ✅ With
keywordfields and correct syntax: Bothequalsandinoperators returned 50 results via SDK - ❌ With
values(plural): API correctly rejects with "Unknown parameter" error
Documentation:
Fix both issues and your filters will work correctly in both console and SDK. I hope this helps. If it does not, let me know and I am happy to troubleshoot further!
Hello, thank you for reaching out. I'll address the two main questions you've raised.
[Q1. Confirm the expected JSON shape for filter in KnowledgeBaseVectorSearchConfiguration for equals and in, given the metadata structure above]
Let me confirm the correct KnowledgeBaseVectorSearchConfiguration filter JSON format for your metadata structure:
- equals operator (single value matching) :
retrieval_configuration = {
"vectorSearchConfiguration": {
"numberOfResults": 50,
"overrideSearchType": "HYBRID",
"filter": {
"equals": {
"key": "program",
"value": "CAH"
}
}
}
}
- in operator (multiple value matching) :
retrieval_configuration = {
"vectorSearchConfiguration": {
"numberOfResults": 50,
"overrideSearchType": "HYBRID",
"filter": {
"in": {
"key": "program",
"values": ["CAH"]
}
}
}
}
The JSON structure you used is completely correct in terms of format. However, to definitively verify the proper format and troubleshoot any differences, I recommend comparing the actual API calls using CloudTrail. This will reveal any differences in how the console and SDK are constructing the API requests.
The issue is very likely not with the filter structure itself, but rather with the field type mapping in your OpenSearch index.
For more details, please refer to documentation[1] and [2].
[Q2. Are there known differences in filter behavior between console and SDK, or any limitations?]
A2. Yes, there are several known differences and limitations.
a. Field Type Dependency (Most Critical in your case)
- For metadata filtering to work, the fields you want to filter on must be mapped as
keywordtype - Fields mapped as
texttype are tokenized during storage, making exact matching (equals,in) impossible
The Bedrock SDK internally uses OpenSearch term queries, which only work correctly with keyword types. This is the root cause of your issue: your metadata field is currently text type. For more details, please refer to documentation[3] and [4].
b. Manual Index Management Required
- Bedrock does not automatically create optimal mappings for managed OpenSearch
- You must explicitly specify metadata field types when creating the index
- If an index is created with incorrect mappings, re-indexing is required
c. Console vs SDK Behavior Discrepancy (the issue you're experiencing)
- The Bedrock console may use internal query transformations or fallback logic
- The SDK calls OpenSearch APIs more directly, so field type issues surface immediately
If you have additional questions or need help with specific configurations, please open a support case and we'll be happy to assist.
Thank you.
Reference [1] Bedrock Agent Runtime - https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base.html [2] Configure and customize queries and response generation - Amazon Bedro... - https://docs.aws.amazon.com/bedrock/latest/userguide/kb-test-config.html [3] Mappings - OpenSearch DocumentationMappings | OpenSearch Documentation - https://docs.opensearch.org/latest/mappings/ [4] Keyword - OpenSearch DocumentationKeyword | OpenSearch DocumentationLi... - https://docs.opensearch.org/latest/mappings/supported-field-types/keyword/
answered 10 months ago
Relevant content
asked 2 years ago
asked a year ago
