trulens.feedback.templates.rag¶
RAG evaluation templates: groundedness, context relevance, answer relevance, answerability, comprehensiveness, etc.
Classes¶
Bases: Semantics
This evaluates the relevance of the LLM response to the given text by LLM prompting.
Relevance is available for any LLM provider.
dataclass
¶
Bases: Semantics, WithPrompt, CriteriaOutputSpaceMixin
Graded, format-agnostic citation quality.
Like CitationAttribution, this asks whether a response's citations are supported by the retrieved context. It differs in three ways that decide which one you want:
- Citation format.
CitationAttributionrequires explicit[N]markers and checks each marker against numbered passage N. This template is format-agnostic: it also covers inline or prose attributions ("according to the 2023 annual report..."), URLs, or footnotes, where there is no[N]marker to resolve. - Output space.
CitationAttributionis binary: any misattribution is a hard fail. This template is a 0-3 Likert, so a response with one bad citation out of ten is distinguishable from one where every citation is bad. Use it when you want a graded signal to track across runs. - Missing citations.
CitationAttributiondeliberately does not penalize a claim that carries no citation marker. This template does: a claim that the context supports but that goes uncited counts against the score. Use it when your pipeline requires that context-derived claims be cited, and useCitationAttributionwhen uncited claims are acceptable and only misattribution matters.
dataclass
¶
Bases: Semantics, WithPrompt, CriteriaOutputSpaceMixin
Citation-attribution faithfulness.
Unlike groundedness (does the source support the statement somewhere),
this checks attribution: does each [N] citation marker point to the
SOURCE passage that supports the specific claim it is attached to. It
catches misattribution: a claim cited to passage [A] that does not
support it, even though some other passage [B] would.