trulens.core.feedback.custom_metric¶

trulens.core.feedback.custom_metric ¶

DEPRECATED: Client-side custom metrics functionality.

This module is deprecated. Use trulens.core.metric.Metric instead.

The MetricConfig class is maintained for backward compatibility but will be removed in a future version.

Classes¶

MetricConfig ¶

DEPRECATED: Use Metric instead.

This class is maintained for backward compatibility. All functionality has been merged into the Metric class.

Example of migrating to Metric

# Old way (deprecated):
from trulens.core.feedback import MetricConfig
config = MetricConfig(
    metric_name="my_metric",
    metric_implementation=my_fn,
    selectors={"arg": Selector.select_record_input()},
)
feedback = config.create_feedback_definition()

# New way (recommended):
from trulens.core import Metric, Selector
metric = Metric(
    name="my_metric",
    implementation=my_fn,
).on({"arg": Selector.select_record_input()})

Functions¶

init ¶

__init__(
    metric_name: str,
    metric_implementation: Callable,
    metric_type: Optional[str] = None,
    selectors: Optional[Dict[str, Selector]] = None,
    computation_type: str = "client",
    higher_is_better: bool = True,
    description: Optional[str] = None,
)

Initialize a metric configuration (deprecated, use Metric instead).

PARAMETER	DESCRIPTION
`metric_name`	Unique semantic identifier for this specific metric usage (e.g., "text2sql_accuracy_v1", "custom_relevance_for_qa") TYPE: `str`
`metric_implementation`	The metric function to execute TYPE: `Callable`
`metric_type`	Implementation identifier of the custom metric (e.g., "text2sql", "accuracy", "relevance"). If not provided, defaults to the function name. TYPE: `Optional[str]` DEFAULT: `None`
`selectors`	Dictionary mapping parameter names to Selectors TYPE: `Optional[Dict[str, Selector]]` DEFAULT: `None`
`computation_type`	Where to compute ("client" or "server") TYPE: `str` DEFAULT: `'client'`
`higher_is_better`	Whether higher scores are better TYPE: `bool` DEFAULT: `True`
`description`	Optional description of the metric TYPE: `Optional[str]` DEFAULT: `None`

validate_selectors ¶

validate_selectors() -> None

Validate that selectors match the function signature.

Only checks required parameters (those without default values).

RAISES	DESCRIPTION
`ValueError`	If selectors don't match function parameters

create_feedback_definition ¶

create_feedback_definition() -> Metric

Create a Metric instance from this metric configuration.

DEPRECATED: This method returns a Metric instance (formerly Feedback).

RETURNS	DESCRIPTION
`Metric`	A Metric instance configured for this metric

RAISES	DESCRIPTION
`ValueError`	If selectors don't match function parameters

to_dict ¶

to_dict() -> Dict[str, Any]

Convert the metric config to a dictionary.

RETURNS	DESCRIPTION
`Dict[str, Any]`	Dictionary representation of the config

trulens.core.feedback.custom_metric¶