Skip to content

πŸš€ Getting Started

πŸ”¨ Installation

These installation instructions assume that you have conda installed and added to your path.

  1. Create a virtual environment (or modify an existing one).

    conda create -n "<my_name>" python=3  # Skip if using existing environment.
conda activate <my_name>

  2. [Pip installation] Install the trulens-eval pip package from PyPI.

    pip install trulens-eval

  3. [Local installation] If you would like to develop or modify TruLens, you can download the source code by cloning the TruLens repo.

    git clone https://github.com/truera/trulens.git

  4. [Local installation] Install the TruLens repo.

    cd trulens/trulens_eval
pip install -e .

🀿 Ready to dive in?

😍 Community

🏁 Releases

Releases

Releases are organized in <major>.<minor>.<patch> style. A release is made about every week around tuesday-thursday. Releases increment the minor version number. Occasionally bug-fix releases occur after a weekly release. Those increment only the patch number. No releases have yet made a major version increment. Those are expected to be major releases that introduce large number of breaking changes.

0.33.0

What's Changed
Documentation Updates
Bug Fixes

0.32.0

What's Changed
Documentation
Examples
Bug Fixes

0.31.0

What's Changed
Examples
Bug fixes

Full Changelog: https://github.com/truera/trulens/compare/trulens-eval-0.30.1...trulens-eval-0.31.0

0.30.1

What's Changed
Bug Fixes

Full Changelog: https://github.com/truera/trulens/compare/trulens-eval-0.29.0...trulens-eval-0.30.1

0.29.0

Breaking Changes

In this release, we re-aligned the groundedness feedback function with other LLM-based feedback functions. It's now faster and easier to define a groundedness feedback function, and can be done with a standard LLM provider rather than importing groundedness on its own. In addition, the custom groundedness aggregation required is now done by default.

Before: 

from trulens_eval.feedback.provider.openai import OpenAI
from trulens_eval.feedback import Groundedness

provider = OpenAI() # or any other LLM-based provider
grounded = Groundedness(groundedness_provider=provider)
f_groundedness = (
    Feedback(grounded.groundedness_measure_with_cot_reasons, name = "Groundedness")
    .on(Select.RecordCalls.retrieve.rets.collect())
    .on_output()
    .aggregate(grounded.grounded_statements_aggregator)
)

After: 

provider = OpenAI()
f_groundedness = (
    Feedback(provider.groundedness_measure_with_cot_reasons, name = "Groundedness")
    .on(Select.RecordCalls.retrieve.rets.collect())
    .on_output()
)

This change also applies to the NLI-based groundedness feedback function available from the Huggingface provider.

Before: 

from trulens_eval.feedback.provider.openai import Huggingface
from trulens_eval.feedback import Groundedness

from trulens_eval.feedback.provider import Huggingface
huggingface_provider = Huggingface()
grounded = Groundedness(groundedness_provider=huggingface_provider)

f_groundedness = (
    Feedback(grounded.groundedness_measure_with_cot_reasons, name = "Groundedness")
    .on(Select.RecordCalls.retrieve.rets.collect())
    .on_output()
    .aggregate(grounded.grounded_statements_aggregator)
)

After: 

from trulens_eval.feedback import Feedback
from trulens_eval.feedback.provider.hugs = Huggingface

huggingface_provider = Huggingface()

f_groundedness = (
    Feedback(huggingface_provider.groundedness_measure_with_nli, name = "Groundedness")
    .on(Select.RecordCalls.retrieve.rets.collect())
    .on_output()
)

In addition to the change described above, below you can find the full release description.

What's Changed

Bug Fixes

Examples

New Contributors

Full Changelog: https://github.com/truera/trulens/compare/trulens-eval-0.28.0...trulens-eval-0.29.0

0.28.1

Bug fixes
  • Fix for missing alembic.ini in package build.

0.28.0

What's Changed
Bug fixes
Examples

Full Changelog: https://github.com/truera/trulens/compare/trulens-eval-0.27.2...trulens-eval-0.28.0