🕸✨ Database Migration

When upgrading TruLens-Eval, it may sometimes be required to migrade the database to incorporate changes in existing database created from the previously installed version. The changes to database schemas is handled by Alembic while some data changes are handled by converters in the data module.

Upgrading to the latest schema revision

from trulens_eval import Tru

tru = Tru(
   database_url="<sqlalchemy_url>",
   database_prefix="trulens_" # default, may be ommitted
)
tru.migrate_database()

Changing database prefix

Since 0.28.0, all tables used by TruLens-Eval are prefixed with "trulens_" including the special alembic_version table used for tracking schema changes. Upgrading to 0.28.0 for the first time will require a migration as specified above. This migration assumes that the prefix in the existing database was blank.

If you need to change this prefix after migration, you may need to specify the old prefix when invoking migrate_database:

tru = Tru(
   database_url="<sqlalchemy_url>",
   database_prefix="new_prefix"
)
tru.migrate_database(prior_prefix="old_prefix")

Copying a database

Have a look at the help text for copy_database and take into account all the items under the section Important considerations:

from trulens_eval.database.utils import copy_database

help(copy_database)

Copy all data from the source database into an EMPTY target database:

from trulens_eval.database.utils import copy_database

copy_database(
    src_url="<source_db_url>",
    tgt_url="<target_db_url>",
    src_prefix="<source_db_prefix>",
    tgt_prefix="<target_db_prefix>"
)

trulens_eval.tru.Tru.migrate_database

migrate_database(**kwargs: Dict[str, Any])

Migrates the database.

This should be run whenever there are breaking changes in a database created with an older version of trulens_eval.

PARAMETER	DESCRIPTION
**kwargs	Keyword arguments to pass to migrate_database of the current database. TYPE: Dict[str, Any] DEFAULT: {}

See DB.migrate_database.

trulens_eval.database.utils.copy_database

copy_database(src_url: str, tgt_url: str, src_prefix: str, tgt_prefix: str)

Copy all data from a source database to an EMPTY target database.

Important considerations:

• All source data will be appended to the target tables, so it is important that the target database is empty.

• Will fail if the databases are not at the latest schema revision. That can be fixed with Tru(database_url="...", database_prefix="...").migrate_database()

• Might fail if the target database enforces relationship constraints, because then the order of inserting data matters.

• This process is NOT transactional, so it is highly recommended that the databases are NOT used by anyone while this process runs.

trulens_eval.database.migrations.data

Attributes

sql_alchemy_migration_versions module-attribute

sql_alchemy_migration_versions: List[str] = ['1']

DB versions that need data migration.

The most recent should be the first in the list.

sqlalchemy_upgrade_paths module-attribute

sqlalchemy_upgrade_paths = {}

A DAG of upgrade functions to get to most recent DB.

Classes

Functions

data_migrate

data_migrate(db: DB, from_version: str)

Makes any data changes needed for upgrading from the from_version to the current version.

PARAMETER	DESCRIPTION
db	The database instance. TYPE: DB
from_version	The version to migrate data from. TYPE: str

RAISES	DESCRIPTION
VersionException	Can raise a migration or validation upgrade error.