Clarification on Bedrock Retrieve API Handling for Long Posts and Incremental Summarization

Question

Clarification on Bedrock Retrieve API Handling for Long Posts and Incremental Summarization
 
Hi [Name],
 
I’m currently using an approach where I first use the Retrieve API to fetch chunks in batches, and then use InvokeModel to generate summaries.
 
For handling long posts, since Bedrock supports retrieving up to 100 results per call, I’ve implemented a loop that recursively calls the Retrieve API until all posts are retrieved and summarized.
 
I’m applying metadata filtering — retrieving posts by threadid (which is unique for each topic), and since each post has a different pid, I also use a PID-based filter for incremental summarization and batching. The filter looks like this:
 
my $filter = {
    andAll => [
        { equals => { key => "threadid", value => $threadid } },
        { greaterThanOrEquals => { key => "pid", value => $firstpid } },
        { greaterThan => { key => "pid", value => $lastpid + 0 } }
    ]
};

Here, $firstpid is static — it defines where the retrieval starts. This helps maintain the correct order of posts during summarization.
 
However, I’m facing an issue:
 
When there are 150 posts in a thread and I request 100 results (numberOfResults => 100), Bedrock sometimes returns posts across the entire range, including the post with pid = 150. As a result, when I try to fetch the “next batch” (e.g., posts 101–150 based on $lastpid), the second batch becomes empty or overlaps because Bedrock already included the last ones.
 
I’d like to understand the best way to retrieve posts in order and handle long threads (for example, 1,000 posts) during the first-time summarization.
 
Also, I want to confirm that this same flow will work correctly for incremental summarization (updates) — for example, if I initially summarize 50 posts and later 20 new posts are created under the same topic, the system should summarize the new posts and merge them accurately with the existing summary.
 
The main issue occurs during first-time summarization when there are more than 100 posts — it causes problems with retrieval and batching.
 
Could you please suggest how this situation can be handled?

Is there a recommended way to manage this, or is it a known limitation in Bedrock’s retrieval support?

re:Post Agent · Accepted Answer

When working with Amazon Bedrock Knowledge Bases and handling large numbers of posts, the behavior you're experiencing with the Retrieve API is related to how relevance ranking works rather than strict ordering by metadata.

The issue you're facing occurs because the Retrieve API is primarily designed to return the most relevant results for a query, not necessarily to return results in a specific order based on metadata. When you request 100 results, Bedrock is returning what it considers the most relevant chunks across the entire dataset that match your filter criteria, which may include posts with higher PIDs.

For your use case of processing posts in order, here are some recommended approaches:

1. Use smaller batch sizes: Instead of requesting 100 results at once, consider using smaller batch sizes (e.g., 20-30) to reduce the likelihood of getting posts from across the entire range.

2. Implement more specific filtering: Modify your filter to include an upper bound on the PIDs for each batch:
```
{
andAll => [
{ equals => { key => "threadid", value => $threadid } },
{ greaterThanOrEquals => { key => "pid", value => $firstpid } },
{ lessThan => { key => "pid", value => $firstpid + 100 } }
]
}
```
Then increment the range for each subsequent batch.

3. Post-process the results: After receiving results from the Retrieve API, sort them by PID on your side and track which PIDs you've already processed to avoid duplicates.

For incremental summarization, your approach should work well. When new posts are added, you can use a filter that only retrieves posts with PIDs greater than the last processed PID from your previous summarization. This ensures you're only processing the new content.

If you continue to face challenges with ordering and batching, you might consider implementing your own pagination logic by tracking the highest PID you've processed and using that as the starting point for subsequent batches, ensuring you don't miss any posts or process duplicates.  
**Sources**  
[Query a knowledge base and retrieve data - Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/kb-test-retrieve.html)  
[Multi-tenancy in RAG applications in a single Amazon Bedrock knowledge base with metadata filtering | AWS Machine Learning Blog](https://aws.amazon.com/blogs/machine-learning/multi-tenancy-in-rag-applications-in-a-single-amazon-bedrock-knowledge-base-with-metadata-filtering/)  
[Bedrock  Agent with knowledgebase metadata search | AWS re:Post](https://repost.aws/questions/QUbOIXPEWrQBuXlRckN3xs1Q/bedrock-agent-with-knowledgebase-metadata-search)

Clarification on Bedrock Retrieve API Handling for Long Posts and Incremental Summarization

Add your answer

Relevant content