Skip to content

Guardrails with Llama-Index

trulens_eval.guardrails.llama

Classes

WithFeedbackFilterNodes

Bases: RetrieverQueryEngine

Attributes
threshold instance-attribute
threshold: float = threshold

A BaseQueryEngine that filters documents using a minimum threshold on a feedback function before returning them.

PARAMETER DESCRIPTION
feedback

use this feedback function to score each document.

TYPE: Feedback

threshold

and keep documents only if their feedback value is at least this threshold.

TYPE: float

Using TruLens guardrail context filters with Llama-Index

from trulens_eval.guardrails.llama import WithFeedbackFilterNodes

# note: feedback function used for guardrail must only return a score, not also reasons
feedback = (
    Feedback(provider.context_relevance)
    .on_input()
    .on(context)
)

filtered_query_engine = WithFeedbackFilterNodes(query_engine, feedback=feedback, threshold=0.5)

tru_recorder = TruLlama(filtered_query_engine,
    app_id='LlamaIndex_App1_Filtered')

with tru_recorder as recording:
    llm_response = filtered_query_engine.query("What did the author do growing up?")
Functions
query
query(query: QueryBundle, **kwargs) -> List[NodeWithScore]

An extended query method that will:

  1. Query the engine with the given query bundle (like before).
  2. Evaluate nodes with a specified feedback function.
  3. Filter out nodes that do not meet the minimum threshold.
  4. Synthesize with only the filtered nodes.
PARAMETER DESCRIPTION
query

QueryBundle - the query bundle to search for relevant nodes.

TYPE: QueryBundle

**kwargs

additional keyword arguments.

DEFAULT: {}

RETURNS DESCRIPTION
List[NodeWithScore]

List[NodeWithScore]: a list of filtered, relevant nodes.