trulens.core.sampling¶

trulens.core.sampling ¶

Sampling configuration and controller for online evaluation.

Controls which records are evaluated during automatic post-ingest evaluation. Does not affect explicit compute_metrics / compute_now calls.

.. note:: Counters (throttle window, daily cost accumulator) are per-process. Multiple workers each enforce their own limits, so effective maximums scale with the number of processes. The daily cost budget resets at UTC midnight and enforcement lags by roughly concurrency * cost_per_eval, making it a soft cap.

Classes¶

EvalDecisionReason ¶

Bases: str, Enum

Why a record was or was not evaluated.

Attributes¶

EVALUATED `class-attribute` `instance-attribute` ¶

EVALUATED = 'evaluated'

Record was sampled and evaluated normally.

NOT_CONFIGURED `class-attribute` `instance-attribute` ¶

NOT_CONFIGURED = 'not_configured'

Sampling does not apply to this record (app not in per-app config).

The record is evaluated normally — this is not a skip reason.

NOT_SAMPLED `class-attribute` `instance-attribute` ¶

NOT_SAMPLED = 'not_sampled'

Record was skipped by the probabilistic sampler.

THROTTLED `class-attribute` `instance-attribute` ¶

THROTTLED = 'throttled'

Record was skipped because the throttle window is saturated.

OVER_BUDGET `class-attribute` `instance-attribute` ¶

OVER_BUDGET = 'over_budget'

Record was skipped because the daily cost budget is exhausted.

SamplingConfig ¶

Bases: BaseModel

Immutable configuration for online-evaluation sampling.

ATTRIBUTE	DESCRIPTION
`sample_rate`	Probability (0.0 -- 1.0) that a record is evaluated. Can also be a `{app_name: rate}` mapping for per-app rates. Default `1.0` (evaluate everything). TYPE: `Union[float, Dict[str, float]]`
`throttle`	Maximum evaluations per minute. `None` means unlimited. TYPE: `Optional[int]`
`cost_budget`	Daily USD cap for evaluation cost. `None` means unlimited. Only enforceable for providers whose `reports_costs` property is `True`. TYPE: `Optional[float]`

Attributes¶

sample_rate `class-attribute` `instance-attribute` ¶

sample_rate: Union[float, Dict[str, float]] = 1.0

Probability of evaluating a record.

float -- a single global rate applied to every app.
dict[str, float] -- per-app rates keyed by app name. Records from apps not in the dict are not evaluated.

throttle `class-attribute` `instance-attribute` ¶

throttle: Optional[int] = None

Max evaluations per minute (None = unlimited).

cost_budget `class-attribute` `instance-attribute` ¶

cost_budget: Optional[float] = None

Daily USD cap (None = unlimited).

Resets at UTC midnight. Per-process, so N workers -> up to N * budget.

SamplingController ¶

Mutable state manager for sampling decisions.

Holds the token-bucket (throttle), daily cost accumulator, and evaluated/skipped counters. All public methods are thread-safe: the lock is held only for bookkeeping, never across an evaluation call.

This class is designed to be substitutable in tests -- construct one directly and pass it wherever a controller is expected.

Attributes¶

counters `property` ¶

counters: Dict[str, int]

Return a snapshot of decision counters (reason -> count).

Functions¶

should_evaluate ¶

should_evaluate(
    record_id: str, app_name: Optional[str] = None
) -> tuple

Decide whether record_id should be evaluated.

Order of checks: app scope -> sample -> throttle -> budget.

RETURNS	DESCRIPTION
`tuple`	`(should_eval, meta)` where meta is a dict with keys
`tuple`	`sample_rate`, `eval_decision_reason`, and `sampled`
`tuple`	(a convenience boolean).

record_cost ¶

record_cost(cost: float) -> None

Record evaluation cost for daily budget tracking.

trulens.core.sampling¶