trulens.feedback.templates.base

trulens.feedback.templates.base

Base classes and shared scaffolding for feedback evaluation templates.

Contains

• FeedbackTemplate (base class, renamed from the former v2 Feedback)
• WithPrompt, NaturalLanguage, Semantics
• CriteriaOutputSpaceMixin, OutputSpace, EvalSchema
• FewShotExample / FewShotExamples
• Criteria enum and supported_criteria mapping
• Level 2/3 design-sketch abstractions (FeedbackOutputType, Model, …)

Classes

FeedbackTemplate

Bases: BaseModel

Base class for feedback template definitions.

Subclasses define system_prompt, user_prompt, criteria, and output_space as ClassVars. Providers use these to build LLM evaluation prompts.

Criteria

Bases: str, Enum

A Criteria to evaluate.

OutputSpace

Bases: Enum

Enum for valid output spaces of scores.

FewShotExamples

Bases: BaseModel

Functions

from_examples_list classmethod

from_examples_list(
    examples_list: List[Tuple[Dict[str, str], int]]
) -> FewShotExamples

Create a FewShotExamples instance from a list of examples.

PARAMETER	DESCRIPTION
examples_list	A list of tuples where the first element is the feedback_args and the second element is the score. TYPE: List[Tuple[Dict[str, str], int]]

RETURNS	DESCRIPTION
FewShotExamples	An instance of FewShotExamples with the provided
FewShotExamples	examples.

FeedbackOutput

Bases: BaseModel

Feedback functions produce at least a floating score.

ClassificationModel

Bases: Model

Functions

of_prompt staticmethod

of_prompt(model: CompletionModel, prompt: str) -> None

Define a classification model from a completion model, a prompt, and optional examples.