Skip to content

Guardrails with Langchain

trulens_eval.guardrails.langchain

Classes

WithFeedbackFilterDocuments

Bases: VectorStoreRetriever

Attributes
threshold instance-attribute
threshold: float

A VectorStoreRetriever that filters documents using a minimum threshold on a feedback function before returning them.

PARAMETER DESCRIPTION
feedback

use this feedback function to score each document.

TYPE: Feedback

threshold

and keep documents only if their feedback value is at least this threshold.

TYPE: float

Using TruLens guardrail context filters with Langchain

from trulens_eval.guardrails.langchain import WithFeedbackFilterDocuments

# note: feedback function used for guardrail must only return a score, not also reasons
feedback = Feedback(provider.context_relevance).on_input().on(context)

filtered_retriever = WithFeedbackFilterDocuments.of_retriever(
    retriever=retriever,
    feedback=feedback,
    threshold=0.5
)

rag_chain = {"context": filtered_retriever | format_docs, "question": RunnablePassthrough()} | prompt | llm | StrOutputParser()

tru_recorder = TruChain(rag_chain,
    app_id='Chain1_ChatApplication_Filtered')

with tru_recorder as recording:
    llm_response = rag_chain.invoke("What is Task Decomposition?")
Functions
of_retriever staticmethod
of_retriever(retriever: VectorStoreRetriever, **kwargs)

Create a new instance of WithFeedbackFilterDocuments based on an existing retriever.

The new instance will:

  1. Get relevant documents (like the existing retriever its based on).
  2. Evaluate documents with a specified feedback function.
  3. Filter out documents that do not meet the minimum threshold.
PARAMETER DESCRIPTION
retriever

VectorStoreRetriever - the base retriever to use.

TYPE: VectorStoreRetriever

**kwargs

additional keyword arguments.

DEFAULT: {}

Returns: - WithFeedbackFilterDocuments: a new instance of WithFeedbackFilterDocuments.

Functions