Scoring mechanism in hierarchical chunking for Bedrock Knowledge Bases

When using hierarchical chunking in Amazon Bedrock Knowledge Bases, the scoring mechanism for retrieved parent chunks is not explicitly detailed in the available documentation. However, based on the information provided about how hierarchical chunking works, we can infer some aspects of the scoring process:

1. The system initially retrieves child chunks based on their relevance to the query. These child chunks are assigned individual relevancy scores.

2. After retrieving the relevant child chunks, the system replaces them with their corresponding parent chunks to provide more comprehensive context.

3. While the exact calculation method for the final parent chunk score is not specified, it's likely that the score is derived from the scores of its matching child chunks.

Given the nature of hierarchical chunking and its focus on providing comprehensive context, it's probable that the parent chunk score is influenced by the following factors:

• The number of child chunks that matched the query within that parent
• The individual scores of those matching child chunks
• The relevance of the overall parent chunk content to the query

Although the precise algorithm isn't disclosed, it's reasonable to assume that the system uses some form of aggregation of the child chunk scores, potentially giving more weight to higher-scoring children or using a combination of maximum and average scores.

It's important to note that the scoring mechanism is designed to balance precision (which smaller child chunks provide) with the need for comprehensive context (which larger parent chunks offer). This approach allows for efficient retrieval while preserving the contextual relationships between chunks, which is beneficial for generating coherent and contextually relevant responses in RAG (Retrieval-Augmented Generation) applications.

For the most accurate and up-to-date information on the scoring mechanism, it would be best to consult the latest AWS documentation or reach out to AWS support for detailed clarification.
Sources
How content chunking works for knowledge bases - Amazon Bedrock
Amazon Bedrock Knowledge Bases now supports advanced parsing, chunking, and query reformulation giving greater control of accuracy in RAG based applications | AWS Machine Learning Blog
Amazon Bedrock Knowledge Bases now supports advanced parsing, chunking, and query reformulation giving greater control of accuracy in RAG based applications | AWS Machine Learning Blog