Serialization Utilities

trulens_eval.utils.serial

Serialization utilities.

TODO: Lens class: can we store just the python AST instead of building up our own "Step" classes to hold the same data? We are already using AST for parsing.

Attributes

JSON_BASES module-attribute

JSON_BASES: Tuple[type, ...] = (str, int, float, bytes, type(None))

Tuple of JSON-able base types.

Can be used in isinstance checks.

JSON_BASES_T module-attribute

JSON_BASES_T = Union[str, int, float, bytes, None]

Alias for JSON-able base types.

JSON module-attribute

JSON = Union[JSON_BASES_T, Sequence[Any], Dict[str, Any]]

Alias for (non-strict) JSON-able data (Any = JSON).

If used with type argument, that argument indicates what the JSON represents and can be desererialized into.

Formal JSON must be a dict at the root but non-strict here means that the root can be a basic type or a sequence as well.

JSON_STRICT module-attribute

JSON_STRICT = Dict[str, JSON]

Alias for (strictly) JSON-able data.

Python object that is directly mappable to JSON.

Classes

JSONized

Bases: dict, Generic[T]

JSON-encoded data the can be deserialized into a given type T.

This class is meant only for type annotations. Any serialization/deserialization logic is handled by different classes, usually subclasses of pydantic.BaseModel.

SerialModel

Bases: BaseModel

Trulens-specific additions on top of pydantic models. Includes utilities to help serialization mostly.

Step

Bases: BaseModel, Hashable

A step in a selection path.

Functions

get

get(obj: Any) -> Iterable[Any]

Get the element of obj, indexed by self.

set

set(obj: Any, val: Any) -> Any

Set the value(s) indicated by self in obj to value val.

Lens

Bases: BaseModel, Sized, Hashable

Lenses into python objects.

Example

path = Lens().record[5]['somekey']

obj = ... # some object that contains a value at `obj.record[5]['somekey]`

value_at_path = path.get(obj) # that value

new_obj = path.set(obj, 42) # updates the value to be 42 instead

collect and special attributes

Some attributes hold special meaning for lenses. Attempting to access them will produce a special lens instead of one that looks up that attribute.

Example

path = Lens().record[:]

obj = dict(record=[1, 2, 3])

value_at_path = path.get(obj) # generates 3 items: 1, 2, 3 (not a list)

path_collect = path.collect()

value_at_path = path_collect.get(obj) # generates a single item, [1, 2, 3] (a list)

Functions

existing_prefix

existing_prefix(obj: Any) -> Lens

Get the Lens representing the longest prefix of the path that exists in the given object.

exists

exists(obj: Any) -> bool

Check whether the path exists in the given object.

of_string staticmethod

of_string(s: str) -> Lens

Convert a string representing a python expression into a Lens.

set_or_append

set_or_append(obj: Any, val: Any) -> Any

If obj at path self is None or does not exist, sets it to a list containing only the given val. If it already exists as a sequence, appends val to that sequence as a list. If it is set but not a sequence, error is thrown.

set

set(obj: T, val: Union[Any, T]) -> T

In obj at path self exists, change it to val. Otherwise create a spot for it with Munch objects and then set it.

Functions

model_dump

model_dump(obj: Union[BaseModel, BaseModel]) -> dict

Return the dict/model_dump of the given pydantic instance regardless of it being v2 or v1.

leaf_queries

leaf_queries(obj_json: JSON, query: Lens = None) -> Iterable[Lens]

Get all queries for the given object that select all of its leaf values.

all_queries

all_queries(obj: Any, query: Lens = None) -> Iterable[Lens]

Get all queries for the given object.

all_objects

all_objects(obj: Any, query: Lens = None) -> Iterable[Tuple[Lens, Any]]

Get all queries for the given object.