Skip to content
By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post
About re:Post

Working of Citation coverage scores

0

Here is an example, where citation coverage is one: And i had 4 citations, so can i assume each response part corresponds to citations not retrived documents right? Citation coverage score: 1 To assess the citation coverage (how much of the information in the generated response that the context can support has been properly cited), we need to do the following for each generated response segment that contains citations: (1) determine whether the cited passages support all the information present in the generated response, and (2) determine whether the context supports all the information present in the generated response. Response part 1: . Response part 2: Response part 3: . Response part 4: Information coverage based on context: The candidate response accurately reflects the information provided in the context passages regarding the. It mentions the system's purpose, the use of...... All these points are covered in the passages.

Topics
Machine Learning & AIGenerative AI on AWS
Tags
Knowledge Bases for Amazon BedrockAmazon Bedrock
Language
English

asked 10 months ago157 views

1 Answer
0

Hi Safant. Thanks for the question in re:Post.

You are correct in that each response part corresponds to citations, not retrieved documents.

Citation coverage measures what percentage of your generated response has proper source attribution. A score of 1.0 means every part of the response is grounded in the retrieved context with no hallucinated content.

Your Example

With 4 citations and citation coverage score of 1.0:

  • Your response has 4 distinct text segments (response parts)
  • Each segment is properly supported by source documents
  • Score of 1.0 = 100% coverage (all response parts are cited)
  • The number of retrieved documents is likely higher than 4

How It Works

When using Knowledge Bases with RetrieveAndGenerate:

  • Retrieved documents are the source chunks fetched from your knowledge base
  • Citations link segments of the generated response to their supporting sources
  • Response parts are the text segments within each citation

References

AWS
EXPERT

answered 10 months ago

  • Safant
    9 months ago

    Thanks for the explanation! I’m trying to reconcile this with the AWS evaluation prompt i can see in invoke_model and definition for Citation Coverage. The AWS prompt says the evaluator checks: Candidate Response vs Related Passages And asks whether the information in the answer is “present in the context passages.” so does this mean its checking with citations or with retrived documents from the KB The official definition also states: “Citation coverage is approximately citation recall.” And I ran an eval, and in explation it says: “A response mostly unfaithful to the retrieved passages may still receive high citation coverage, if all the useful retrieved passages have been cited.” Because of this, I’m trying to understand: Does citation coverage score 1.0 mean: All response parts are grounded and have correct source attribution? or All information in the answer that exists in the retrieved context is covered, regardless of whether the entire answer is grounded. Both interpretations appear in different explanations, so I want to confirm how AWS actually computes the score based strictly on the evaluation prompt and definition.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Relevant content