trulens.feedback.schema_validator¶

trulens.feedback.schema_validator ¶

Classes¶

SchemaValidator ¶

Bases: WithClassInfo, SerialModel

Non-LLM feedback functions for validating LLM output against a schema.

Accepts either a JSON schema dict (requires jsonschema) or a Pydantic model class. Each method returns 1.0 when the output is valid and 0.0 otherwise, along with a metadata dict that contains any validation errors.

Example — JSON schema dict:

from trulens.feedback.schema_validator import SchemaValidator
from trulens.core.metric.metric import Metric

schema = {
    "type": "object",
    "properties": {"answer": {"type": "string"}},
    "required": ["answer"],
}
validator = SchemaValidator(schema=schema)
f = Metric(validator.validate_json).on_output()

Example — Pydantic model:

import pydantic
from trulens.feedback.schema_validator import SchemaValidator
from trulens.core.metric.metric import Metric

class MyOutput(pydantic.BaseModel):
    answer: str
    score: float

validator = SchemaValidator(schema=MyOutput)
f = Metric(validator.validate_json).on_output()

Attributes¶

tru_class_info `instance-attribute` ¶

tru_class_info: Class

Class information of this pydantic object for use in deserialization.

Using this odd key to not pollute attribute names in whatever class we mix this into. Should be the same as CLASS_INFO.

Functions¶

init ¶

__init__(schema: _SchemaType, **kwargs)

Create a SchemaValidator.

PARAMETER	DESCRIPTION
`schema`	Either a JSON schema dict or a Pydantic `BaseModel` class (not an instance). TYPE: `_SchemaType`

validate_json ¶

validate_json(output: str) -> tuple[float, dict[str, str]]

Validate that output is valid JSON conforming to the schema.

Returns 1.0 when valid, 0.0 otherwise. The accompanying metadata dict always contains an "explanation" key describing the outcome (or the first validation error).

PARAMETER	DESCRIPTION
`output`	The string to validate. It must be parseable as JSON. TYPE: `str`

RETURNS	DESCRIPTION
`float`	A `(score, metadata)` tuple compatible with TruLens feedback
`dict[str, str]`	infrastructure.

validate_json_partial ¶

validate_json_partial(
    output: str, required_keys: list | None = None
) -> tuple[float, dict[str, str]]

Validate that output is valid JSON and optionally check for keys.

This is a lighter-weight check: it verifies that output parses as a JSON object and, when required_keys is provided, that each key is present at the top level. The full schema is not consulted, making this useful for streaming or partial outputs.

PARAMETER	DESCRIPTION
`output`	The string to validate. TYPE: `str`
`required_keys`	Optional list of keys that must exist at the top level of the parsed object. TYPE: `list \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`tuple[float, dict[str, str]]`	A `(score, metadata)` tuple.

repr ¶

__repr__() -> str

Safe repr that handles circular references.

Pydantic's default __repr__ does not guard against circular references among model instances, which leads to RecursionError (see GitHub issue #1862). This override uses the same formatted_objects context-variable that __rich_repr__ uses so that already-visited objects are replaced with a short placeholder instead of recursing infinitely.

__rich_repr__ ¶

__rich_repr__() -> Result

Requirement for pretty printing using the rich package.

load `staticmethod` ¶

load(obj, *args, **kwargs)

Deserialize/load this object using the class information in tru_class_info to lookup the actual class that will do the deserialization.

model_validate `classmethod` ¶

model_validate(*args, **kwargs) -> Any

Deserialized a jsonized version of the app into the instance of the class it was serialized from.

Note

This process uses extra information stored in the jsonized object and handled by WithClassInfo.

trulens.feedback.schema_validator¶