Google Quickstart¶

In this quickstart, you will learn to build and evaluate a RAG (Retrieval-Augmented Generation) application using Google's powerful Gemini models, with a focus on getting started quickly in Google AI Studio.

Building and evaluating RAG applications with Google Gemini gives you access to a suite of state-of-the-art, multi-modal models.

In this example, we will use Gemini models for embedding, generation, and as the LLM judge to power TruLens feedback functions. You can start building immediately in Google AI Studio with just a Google account.

In [ ]:

# !pip install trulens trulens-providers-google chromadb

# !pip install trulens trulens-providers-google chromadb

In [ ]:

import os

os.environ["TRULENS_OTEL_TRACING"] = "1"

os.environ["GOOGLE_API_KEY"] = "..."

import os os.environ["TRULENS_OTEL_TRACING"] = "1" os.environ["GOOGLE_API_KEY"] = "..."

Get Data¶

In this case, we'll just initialize some simple text in the notebook.

In [ ]:

university_info = """
The University of Washington, founded in 1861 in Seattle, is a public research university
with over 45,000 students across three campuses in Seattle, Tacoma, and Bothell.
As the flagship institution of the six public universities in Washington state,
UW encompasses over 500 buildings and 20 million square feet of space,
including one of the largest library systems in the world.
"""

university_info = """ The University of Washington, founded in 1861 in Seattle, is a public research university with over 45,000 students across three campuses in Seattle, Tacoma, and Bothell. As the flagship institution of the six public universities in Washington state, UW encompasses over 500 buildings and 20 million square feet of space, including one of the largest library systems in the world. """

In [ ]:

import chromadb

chroma_client = chromadb.Client()
vector_store = chroma_client.get_or_create_collection(name="Universities")

import chromadb chroma_client = chromadb.Client() vector_store = chroma_client.get_or_create_collection(name="Universities")

Build RAG from scratch¶

Build a custom RAG from scratch, and add TruLens custom instrumentation.

In [ ]:

from trulens.apps.app import instrument
from trulens.core import TruSession
from google import genai

session = TruSession()
session.reset_database()

# NOW create the google_client AFTER TruSession is initialized
# This ensures the client gets instrumented for token tracking
google_client = genai.Client()

# Create embeddings and add to vector store
document_embeddings = google_client.models.embed_content(
    model="gemini-embedding-001",
    contents=university_info
).embeddings

vector_store.add(
    "uni_info", documents=university_info, embeddings=document_embeddings[0].values
)

from trulens.apps.app import instrument from trulens.core import TruSession from google import genai session = TruSession() session.reset_database() # NOW create the google_client AFTER TruSession is initialized # This ensures the client gets instrumented for token tracking google_client = genai.Client() # Create embeddings and add to vector store document_embeddings = google_client.models.embed_content( model="gemini-embedding-001", contents=university_info ).embeddings vector_store.add( "uni_info", documents=university_info, embeddings=document_embeddings[0].values )

In [ ]:

class RAG_from_scratch:
    @instrument
    def retrieve(self, query: str) -> list:
        """
        Retrieve relevant text from vector store.
        """
        results = vector_store.query(
            query_embeddings=google_client.models.embed_content(
                model="gemini-embedding-001", contents=query
            )
            .embeddings[0]
            .values,
            n_results=2,
        )
        return results["documents"]

    @instrument
    def generate_completion(self, query: str, context_str: list) -> str:
        """
        Generate answer from context.
        """
        prompt = f"""
         We have provided context information below. 
            {context_str}
            Given this information, please answer the question: {query}
        """
        resp = google_client.models.generate_content(
            model="gemini-2.5-flash", contents=prompt
        )

        return resp.text if resp else ""

    @instrument
    def query(self, query: str) -> str:
        context_str = self.retrieve(query)
        completion = self.generate_completion(query, context_str)
        return completion


rag = RAG_from_scratch()

class RAG_from_scratch: @instrument def retrieve(self, query: str) -> list: """ Retrieve relevant text from vector store. """ results = vector_store.query( query_embeddings=google_client.models.embed_content( model="gemini-embedding-001", contents=query ) .embeddings[0] .values, n_results=2, ) return results["documents"] @instrument def generate_completion(self, query: str, context_str: list) -> str: """ Generate answer from context. """ prompt = f""" We have provided context information below. {context_str} Given this information, please answer the question: {query} """ resp = google_client.models.generate_content( model="gemini-2.5-flash", contents=prompt ) return resp.text if resp else "" @instrument def query(self, query: str) -> str: context_str = self.retrieve(query) completion = self.generate_completion(query, context_str) return completion rag = RAG_from_scratch()

Set up feedback functions.¶

Here we'll use groundedness, answer relevance and context relevance to detect hallucination.

In [ ]:

import numpy as np
from trulens.core import Feedback
from trulens.core.feedback.selector import Selector
from trulens.providers.google import Google

provider = Google()

# Define a groundedness feedback function
# groundedness_measure_with_cot_reasons(source, statement)
f_groundedness = Feedback(
    provider.groundedness_measure_with_cot_reasons, name="Groundedness"
).on({
    "source": Selector(function_name="retrieve", function_attribute="return", collect_list=True),
    "statement": Selector.select_record_output()
})

# Question/answer relevance between overall question and answer.
# relevance_with_cot_reasons(prompt, response)
f_answer_relevance = Feedback(
    provider.relevance_with_cot_reasons, name="Answer Relevance"
).on({
    "prompt": Selector(function_name="retrieve", function_attribute="query"),
    "response": Selector.select_record_output()
})

# Question/statement relevance between question and each context chunk.
# context_relevance_with_cot_reasons(question, context)
f_context_relevance = (
    Feedback(
        provider.context_relevance_with_cot_reasons, name="Context Relevance"
    )
    .on({
        "question": Selector(function_name="retrieve", function_attribute="query"),
        "context": Selector(function_name="retrieve", function_attribute="return", collect_list=True)
    })
    .aggregate(np.mean)
)
                      
# coherence_with_cot_reasons(text)
f_coherence = Feedback(
    provider.coherence_with_cot_reasons, name="coherence"
).on({"text": Selector.select_record_output()})

import numpy as np from trulens.core import Feedback from trulens.core.feedback.selector import Selector from trulens.providers.google import Google provider = Google() # Define a groundedness feedback function # groundedness_measure_with_cot_reasons(source, statement) f_groundedness = Feedback( provider.groundedness_measure_with_cot_reasons, name="Groundedness" ).on({ "source": Selector(function_name="retrieve", function_attribute="return", collect_list=True), "statement": Selector.select_record_output() }) # Question/answer relevance between overall question and answer. # relevance_with_cot_reasons(prompt, response) f_answer_relevance = Feedback( provider.relevance_with_cot_reasons, name="Answer Relevance" ).on({ "prompt": Selector(function_name="retrieve", function_attribute="query"), "response": Selector.select_record_output() }) # Question/statement relevance between question and each context chunk. # context_relevance_with_cot_reasons(question, context) f_context_relevance = ( Feedback( provider.context_relevance_with_cot_reasons, name="Context Relevance" ) .on({ "question": Selector(function_name="retrieve", function_attribute="query"), "context": Selector(function_name="retrieve", function_attribute="return", collect_list=True) }) .aggregate(np.mean) ) # coherence_with_cot_reasons(text) f_coherence = Feedback( provider.coherence_with_cot_reasons, name="coherence" ).on({"text": Selector.select_record_output()})

Construct the app¶

Wrap the custom RAG with TruApp, add list of feedbacks for eval

In [ ]:

from trulens.apps.app import TruApp

tru_rag = TruApp(
    rag,
    app_name="RAG",
    app_version="v1",
    feedbacks=[
        f_groundedness,
        f_answer_relevance,
        f_context_relevance,
        f_coherence,
    ],
)

from trulens.apps.app import TruApp tru_rag = TruApp( rag, app_name="RAG", app_version="v1", feedbacks=[ f_groundedness, f_answer_relevance, f_context_relevance, f_coherence, ], )

Run the app¶

Use tru_rag as a context manager for the custom RAG-from-scratch app.

In [ ]:

with tru_rag as recording:
    resp = rag.query("What do you know about the University of Washington? Please write a 600 word paragraph")

with tru_rag as recording: resp = rag.query("What do you know about the University of Washington? Please write a 600 word paragraph")

In [ ]:

resp

resp

In [ ]:

session.get_leaderboard(app_ids=[])

session.get_leaderboard(app_ids=[])

In [ ]:

# To view the dashboard, run this in your terminal:
# python -m trulens.dashboard

# Or force it to open in browser instead of notebook widget:
from trulens.dashboard import run_dashboard

run_dashboard(session)

# To view the dashboard, run this in your terminal: # python -m trulens.dashboard # Or force it to open in browser instead of notebook widget: from trulens.dashboard import run_dashboard run_dashboard(session)