Logging to a Snowflake Cortex AI Gateway¶

This notebook demonstrates how to configure TruLens to export traces to a Snowflake Cortex AI Gateway via OTLP, instead of the default SQLite.

An AI Gateway is a governed inference endpoint in your Snowflake account: LLM requests are routed through it, and the gateway writes server-side observability spans (prompt, completion, tokens, cost, latency) to its AGENT_TRACE_TABLE automatically. This example additionally exports the client-side application spans from your instrumented app, so a full trace — your app's logic plus the gateway's server spans — lands in one table.

Two export paths are shown:

1. TruSession with an OTLP exporter — for instrumented Python apps (TruApp and friends).
2. Client hooks destination — for coding agents like OpenCode and Claude Code, using TRULENS_DESTINATION=ai_gateway.

Prerequisites¶

1. An AI Gateway with client telemetry enabled¶

-- ACCOUNTADMIN required
CREATE OR ALTER AI GATEWAY my_gateway SET SETTINGS = {
    'enable_client_telemetry': true,
    'capture_payload': {}
}

2. A programmatic access token (PAT) for authentication¶

The gateway authenticates OTLP exports with a bearer token. Store the PAT in a file and keep it out of source control.

3. Install the OTLP exporters¶

pip install "trulens[otlp]"

Note: AI Gateways serve OTLP over HTTP/protobuf (the OTLP spec's default transport), not gRPC. TruLens selects the transport via otlp_protocol="http/protobuf" (or the standard OTEL_EXPORTER_OTLP_PROTOCOL environment variable).

Configure the Gateway¶

In [ ]:

import os

# Path to your programmatic access token file
PAT_FILE = "~/.snowflake/my_gateway.pat"

# Gateway URL, up to and including the gateway name
GATEWAY_URL = "https://<account-host>/api/v2/aigateways/my_gateway"

with open(os.path.expanduser(PAT_FILE)) as f:
    token = f.read().strip()

import os # Path to your programmatic access token file PAT_FILE = "~/.snowflake/my_gateway.pat" # Gateway URL, up to and including the gateway name GATEWAY_URL = "https://<account-host>/api/v2/aigateways/my_gateway" with open(os.path.expanduser(PAT_FILE)) as f: token = f.read().strip()

Create an Instrumented App¶

Let's create a simple RAG-style application whose LLM calls are routed through the gateway. The gateway records server-side spans for these calls automatically; the @instrument decorators below add the client-side spans that make up the rest of the trace.

In [ ]:

from openai import OpenAI
from trulens.core.otel.instrument import instrument
from trulens.otel.semconv.trace import SpanAttributes

# Point the OpenAI client at the gateway; requests to any model
# the gateway serves are logged server-side by the gateway.
client = OpenAI(
    base_url=f"{GATEWAY_URL}/v1",
    api_key=token,
)


class SimpleRAGApp:
    """A simple RAG-style application for demonstration."""

    @instrument(span_type=SpanAttributes.SpanType.RETRIEVAL)
    def retrieve(self, query: str) -> list:
        """Retrieve relevant contexts for a query."""
        # Simulated retrieval - in practice, this would query a vector store
        return ["TruLens is a framework for tracing and evaluating LLM applications."]

    @instrument(span_type=SpanAttributes.SpanType.GENERATION)
    def generate_completion(self, query: str, context_str: list) -> str:
        """Generate an answer from the context."""
        completion = client.chat.completions.create(
            model="my-model",  # any model the gateway serves
            messages=[
                {"role": "system", "content": "Answer the question using the context."},
                {"role": "user", "content": f"Context: {context_str}\n\nQuestion: {query}"},
            ],
        )
        return completion.choices[0].message.content

    @instrument()
    def query(self, query: str) -> str:
        """Retrieve context and generate an answer."""
        context_str = self.retrieve(query=query)
        return self.generate_completion(query=query, context_str=context_str)


app = SimpleRAGApp()

from openai import OpenAI from trulens.core.otel.instrument import instrument from trulens.otel.semconv.trace import SpanAttributes # Point the OpenAI client at the gateway; requests to any model # the gateway serves are logged server-side by the gateway. client = OpenAI( base_url=f"{GATEWAY_URL}/v1", api_key=token, ) class SimpleRAGApp: """A simple RAG-style application for demonstration.""" @instrument(span_type=SpanAttributes.SpanType.RETRIEVAL) def retrieve(self, query: str) -> list: """Retrieve relevant contexts for a query.""" # Simulated retrieval - in practice, this would query a vector store return ["TruLens is a framework for tracing and evaluating LLM applications."] @instrument(span_type=SpanAttributes.SpanType.GENERATION) def generate_completion(self, query: str, context_str: list) -> str: """Generate an answer from the context.""" completion = client.chat.completions.create( model="my-model", # any model the gateway serves messages=[ {"role": "system", "content": "Answer the question using the context."}, {"role": "user", "content": f"Context: {context_str}\n\nQuestion: {query}"}, ], ) return completion.choices[0].message.content @instrument() def query(self, query: str) -> str: """Retrieve context and generate an answer.""" context_str = self.retrieve(query=query) return self.generate_completion(query=query, context_str=context_str) app = SimpleRAGApp()

Connect TruSession to the Gateway¶

AI Gateways serve OTLP over HTTP/protobuf at <gateway>/telemetry/v1/traces (traces only), so pass an explicit span_exporter to export traces without wiring a metrics pipeline.

In [ ]:

from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from trulens.core import TruSession

session = TruSession(
    span_exporter=OTLPSpanExporter(
        endpoint=f"{GATEWAY_URL}/telemetry/v1/traces",
        headers={"Authorization": f"Bearer {token}"},
    ),
)

# Alternatively, the stock OTLP option with the protocol override. Note this
# also wires an OTLP metrics pipeline, and gateways do not serve one, so
# metric exports will fail (harmlessly) with a 400:
#
# import os
# os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = f"Authorization=Bearer {token}"
# session = TruSession(
#     otel_exporter="otlp",
#     otlp_endpoint=f"{GATEWAY_URL}/telemetry/v1/traces",
#     otlp_protocol="http/protobuf",
# )

from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter from trulens.core import TruSession session = TruSession( span_exporter=OTLPSpanExporter( endpoint=f"{GATEWAY_URL}/telemetry/v1/traces", headers={"Authorization": f"Bearer {token}"}, ), ) # Alternatively, the stock OTLP option with the protocol override. Note this # also wires an OTLP metrics pipeline, and gateways do not serve one, so # metric exports will fail (harmlessly) with a 400: # # import os # os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = f"Authorization=Bearer {token}" # session = TruSession( # otel_exporter="otlp", # otlp_endpoint=f"{GATEWAY_URL}/telemetry/v1/traces", # otlp_protocol="http/protobuf", # )

Wrap with TruApp and Record¶

In [ ]:

from trulens.apps.app import TruApp

tru_app = TruApp(
    app,
    app_name="AIGatewayExampleApp",
    app_version="v1",
)

# Run the app - client-side spans are exported to the gateway via OTLP,
# and the gateway records its own server-side spans for the LLM calls.
with tru_app as recording:
    result = app.query("What is TruLens?")
    print(f"Answer: {result}")

session.force_flush()

from trulens.apps.app import TruApp tru_app = TruApp( app, app_name="AIGatewayExampleApp", app_version="v1", ) # Run the app - client-side spans are exported to the gateway via OTLP, # and the gateway records its own server-side spans for the LLM calls. with tru_app as recording: result = app.query("What is TruLens?") print(f"Answer: {result}") session.force_flush()

Query Traces from the Gateway's Trace Table¶

Client-side and server-side spans both land in the gateway's AGENT_TRACE_TABLE. Query it from any Snowflake session with the AGENT_TRACE_TABLE table function:

In [ ]:

# From a Snowflake session (e.g. snowpark) connected to the same account:
#
# spans_df = snowpark_session.sql(""
#     SELECT trace['trace_id'] AS trace_id,
#            record:name AS span_name,
#            record:kind AS span_kind,
#            record_attributes['ai.observability.app_name'] AS app_name
#     FROM TABLE(AGENT_TRACE_TABLE('my_gateway'))
#     ORDER BY start_timestamp DESC
# "").to_pandas()
# spans_df.head(10)

print("See the SQL above - run it from your Snowflake session.")

# From a Snowflake session (e.g. snowpark) connected to the same account: # # spans_df = snowpark_session.sql("" # SELECT trace['trace_id'] AS trace_id, # record:name AS span_name, # record:kind AS span_kind, # record_attributes['ai.observability.app_name'] AS app_name # FROM TABLE(AGENT_TRACE_TABLE('my_gateway')) # ORDER BY start_timestamp DESC # "").to_pandas() # spans_df.head(10) print("See the SQL above - run it from your Snowflake session.")

Coding Agents: Client Hooks Destination¶

For coding agents (OpenCode, Claude Code), install the TruLens client hooks and set the ai_gateway destination. The hook journal exports agent turns as traces to the same gateway table:

# Install the plugin (writes ~/.config/opencode/plugins/trulens-client-hooks.js)
trulens-client-hooks install opencode

# Export environment variables for the hook worker
export TRULENS_DESTINATION=ai_gateway
export TRULENS_AI_GATEWAY_URL="https://<account-host>/api/v2/aigateways/my_gateway"
export TRULENS_AI_GATEWAY_PAT_FILE="$HOME/.snowflake/my_gateway.pat"

Each agent turn is assembled into a trace (for OpenCode: opencode.agent, chat.message, opencode.request_response, session.idle) and exported to the gateway's AGENT_TRACE_TABLE with conversation attribution intact.

The generic otlp destination also works against a gateway:

export TRULENS_DESTINATION=otlp
export TRULENS_OTLP_ENDPOINT="https://<account-host>/api/v2/aigateways/my_gateway/telemetry/v1/traces"
export TRULENS_OTLP_PROTOCOL=http/protobuf
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer $TOKEN"

Troubleshooting¶

• ModuleNotFoundError: opentelemetry.exporter.otlp.proto.grpc — the default OTLP transport is gRPC. Gateways serve HTTP/protobuf, so pass otlp_protocol="http/protobuf" (or set OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf).
• 401 from the telemetry endpoint — the bearer token is required; check the PAT is valid and not expired.
• Client hooks export nothing, exit 0 — export failures are logged by the worker (check ~/.trulens/client-hooks/worker.log); the most common cause is a missing TRULENS_AI_GATEWAY_URL or unset TRULENS_DESTINATION in the environment the worker was started in.
• Spans not visible immediately — the gateway ingests traces asynchronously; allow a short delay before querying AGENT_TRACE_TABLE.