Index

trulens_eval.database.base

Attributes

DEFAULT_DATABASE_PREFIX module-attribute

DEFAULT_DATABASE_PREFIX: str = 'trulens_'

Default prefix for table names for trulens_eval to use.

This includes alembic's version table.

DEFAULT_DATABASE_FILE module-attribute

DEFAULT_DATABASE_FILE: str = 'default.sqlite'

Filename for default sqlite database.

The sqlalchemy url for this default local sqlite database is sqlite:///default.sqlite.

DEFAULT_DATABASE_REDACT_KEYS module-attribute

DEFAULT_DATABASE_REDACT_KEYS: bool = False

Default value for option to redact secrets before writing out data to database.

Classes

DB

Bases: SerialModel, ABC

Abstract definition of databases used by trulens_eval.

SQLAlchemyDB is the main and default implementation of this interface.

Attributes

redact_keys class-attribute instance-attribute

redact_keys: bool = DEFAULT_DATABASE_REDACT_KEYS

Redact secrets before writing out data.

table_prefix class-attribute instance-attribute

table_prefix: str = DEFAULT_DATABASE_PREFIX

Prefix for table names for trulens_eval to use.

May be useful in some databases where trulens is not the only app.

Functions

reset_database abstractmethod

reset_database()

Delete all data.

migrate_database abstractmethod

migrate_database(prior_prefix: Optional[str] = None)

Migrade the stored data to the current configuration of the database.

PARAMETER	DESCRIPTION
prior_prefix	If given, the database is assumed to have been reconfigured from a database with the given prefix. If not given, it may be guessed if there is only one table in the database with the suffix alembic_version. TYPE: Optional[str] DEFAULT: None

check_db_revision abstractmethod

check_db_revision()

Check that the database is up to date with the current trulens_eval version.

RAISES	DESCRIPTION
ValueError	If the database is not up to date.

insert_record abstractmethod

insert_record(record: Record) -> RecordID

Upsert a record into the database.

PARAMETER	DESCRIPTION
record	The record to insert or update. TYPE: Record

RETURNS	DESCRIPTION
RecordID	The id of the given record.

insert_app abstractmethod

insert_app(app: AppDefinition) -> AppID

Upsert an app into the database.

PARAMETER	DESCRIPTION
app	The app to insert or update. Note that only the AppDefinition parts are serialized hence the type hint. TYPE: AppDefinition

RETURNS	DESCRIPTION
AppID	The id of the given app.

insert_feedback_definition abstractmethod

insert_feedback_definition(feedback_definition: FeedbackDefinition) -> FeedbackDefinitionID

Upsert a feedback_definition into the databaase.

PARAMETER	DESCRIPTION
feedback_definition	The feedback definition to insert or update. Note that only the FeedbackDefinition parts are serialized hence the type hint. TYPE: FeedbackDefinition

RETURNS	DESCRIPTION
FeedbackDefinitionID	The id of the given feedback definition.

get_feedback_defs abstractmethod

get_feedback_defs(feedback_definition_id: Optional[FeedbackDefinitionID] = None) -> DataFrame

Retrieve feedback definitions from the database.

PARAMETER	DESCRIPTION
feedback_definition_id	if provided, only the feedback definition with the given id is returned. Otherwise, all feedback definitions are returned. TYPE: Optional[FeedbackDefinitionID] DEFAULT: None

RETURNS	DESCRIPTION
DataFrame	A dataframe with the feedback definitions.

insert_feedback abstractmethod

insert_feedback(feedback_result: FeedbackResult) -> FeedbackResultID

Upsert a feedback_result into the the database.

PARAMETER	DESCRIPTION
feedback_result	The feedback result to insert or update. TYPE: FeedbackResult

RETURNS	DESCRIPTION
FeedbackResultID	The id of the given feedback result.

get_feedback abstractmethod

get_feedback(record_id: Optional[RecordID] = None, feedback_result_id: Optional[FeedbackResultID] = None, feedback_definition_id: Optional[FeedbackDefinitionID] = None, status: Optional[Union[FeedbackResultStatus, Sequence[FeedbackResultStatus]]] = None, last_ts_before: Optional[datetime] = None, offset: Optional[int] = None, limit: Optional[int] = None, shuffle: Optional[bool] = None) -> DataFrame

Get feedback results matching a set of optional criteria:

PARAMETER	DESCRIPTION
record_id	Get only the feedback for the given record id. TYPE: Optional[RecordID] DEFAULT: None
feedback_result_id	Get only the feedback for the given feedback result id. TYPE: Optional[FeedbackResultID] DEFAULT: None
feedback_definition_id	Get only the feedback for the given feedback definition id. TYPE: Optional[FeedbackDefinitionID] DEFAULT: None
status	Get only the feedback with the given status. If a sequence of statuses is given, all feedback with any of the given statuses are returned. TYPE: Optional[Union[FeedbackResultStatus, Sequence[FeedbackResultStatus]]] DEFAULT: None
last_ts_before	get only results with last_ts before the given datetime. TYPE: Optional[datetime] DEFAULT: None
offset	index of the first row to return. TYPE: Optional[int] DEFAULT: None
limit	limit the number of rows returned. TYPE: Optional[int] DEFAULT: None
shuffle	shuffle the rows before returning them. TYPE: Optional[bool] DEFAULT: None

get_feedback_count_by_status abstractmethod

get_feedback_count_by_status(record_id: Optional[RecordID] = None, feedback_result_id: Optional[FeedbackResultID] = None, feedback_definition_id: Optional[FeedbackDefinitionID] = None, status: Optional[Union[FeedbackResultStatus, Sequence[FeedbackResultStatus]]] = None, last_ts_before: Optional[datetime] = None, offset: Optional[int] = None, limit: Optional[int] = None, shuffle: bool = False) -> Dict[FeedbackResultStatus, int]

Get count of feedback results matching a set of optional criteria grouped by their status.

See get_feedback for the meaning of the the arguments.

RETURNS	DESCRIPTION
Dict[FeedbackResultStatus, int]	A mapping of status to the count of feedback results of that status that match the given filters.

get_app abstractmethod

get_app(app_id: AppID) -> Optional[JSONized[App]]

Get the app with the given id from the database.

RETURNS	DESCRIPTION
Optional[JSONized[App]]	The jsonized version of the app with the given id. Deserialization can be done with App.model_validate.

get_apps abstractmethod

get_apps() -> Iterable[JSON]

Get all apps.

get_records_and_feedback abstractmethod

get_records_and_feedback(app_ids: Optional[List[AppID]] = None) -> Tuple[DataFrame, Sequence[str]]

Get records fom the database.

PARAMETER	DESCRIPTION
app_ids	If given, retrieve only the records for the given apps. Otherwise all apps are retrieved. TYPE: Optional[List[AppID]] DEFAULT: None

RETURNS	DESCRIPTION
DataFrame	A dataframe with the records.
Sequence[str]	A list of column names that contain feedback results.

Functions