Skip to content

Distributions of Interest

trulens_explain.trulens.nn.distributions

The distribution of interest lets us specify the set of samples over which we want our explanations to be faithful. In some cases, we may want to explain the modelโ€™s behavior on a particular record, whereas other times we may be interested in a more general behavior over a distribution of samples.

Classes

DoiCutSupportError

Bases: ValueError

Exception raised if the distribution of interest is called on a cut whose output is not supported by the distribution of interest.

DoI

Bases: ABC

Interface for distributions of interest. The Distribution of Interest (DoI) specifies the samples over which an attribution method is aggregated.

Functions
__init__
__init__(cut: Cut = None)

"Initialize DoI

PARAMETER DESCRIPTION
cut

The Cut in which the DoI will be applied. If None, the DoI will be applied to the input. otherwise, the distribution should be applied to the latent space defined by the cut.

TYPE: Cut DEFAULT: None

__call__ abstractmethod
__call__(z: OM[Inputs, TensorLike], *, model_inputs: Optional[ModelInputs] = None) -> OM[Inputs, Uniform[TensorLike]]

Computes the distribution of interest from an initial point. If z: TensorLike is given, we assume there is only 1 input to the DoI layer. If z: List[TensorLike] is given, it provides all of the inputs to the DoI layer.

Either way, we always return List[List[TensorLike]] (alias Inputs[Uniform[TensorLike]]) with outer list spanning layer inputs, and inner list spanning a distribution's instance.

PARAMETER DESCRIPTION
z

Input point from which the distribution is derived. If list/tuple, the point is defined by multiple tensors.

TYPE: OM[Inputs, TensorLike]

model_inputs

Optional wrapped model input arguments that produce value z at cut.

TYPE: Optional[ModelInputs] DEFAULT: None

RETURNS DESCRIPTION
OM[Inputs, Uniform[TensorLike]]

List of points which are all assigned equal probability mass in the

OM[Inputs, Uniform[TensorLike]]

distribution of interest, i.e., the distribution of interest is a

OM[Inputs, Uniform[TensorLike]]

discrete, uniform distribution over the list of returned points. If

OM[Inputs, Uniform[TensorLike]]

z is multi-input, returns a distribution for each input.

cut
cut() -> Cut
RETURNS DESCRIPTION
Cut

The Cut in which the DoI will be applied. If None, the DoI will be

Cut

applied to the input. otherwise, the distribution should be applied

Cut

to the latent space defined by the cut.

get_activation_multiplier
get_activation_multiplier(activation: OM[Inputs, TensorLike], *, model_inputs: Optional[ModelInputs] = None) -> OM[Inputs, TensorLike]

Returns a term to multiply the gradient by to convert from "influence space" to "attribution space". Conceptually, "influence space" corresponds to the potential effect of a slight increase in each feature, while "attribution space" corresponds to an approximation of the net marginal contribution to the quantity of interest of each feature.

PARAMETER DESCRIPTION
activation

The activation of the layer the DoI is applied to. DoI may be multi-input in which case activation will be a list.

TYPE: OM[Inputs, TensorLike]

model_inputs

Optional wrapped model input arguments that produce activation at cut.

TYPE: Optional[ModelInputs] DEFAULT: None

RETURNS DESCRIPTION
OM[Inputs, TensorLike]

An array with the same shape as activation that will be

OM[Inputs, TensorLike]

multiplied by the gradient to obtain the attribution. The default

OM[Inputs, TensorLike]

implementation of this method simply returns activation. If

OM[Inputs, TensorLike]

activation is multi-input, returns one multiplier for each.

PointDoi

Bases: DoI

Distribution that puts all probability mass on a single point.

Functions
__init__
__init__(cut: Cut = None)

"Initialize PointDoI

PARAMETER DESCRIPTION
cut

The Cut in which the DoI will be applied. If None, the DoI will be applied to the input. otherwise, the distribution should be applied to the latent space defined by the cut.

TYPE: Cut DEFAULT: None

LinearDoi

Bases: DoI

Distribution representing the linear interpolation between a baseline and the given point. Used by Integrated Gradients.

Functions
__init__
__init__(baseline: BaselineLike = None, resolution: int = 10, *, cut: Cut = None)

The DoI for point, z, will be a uniform distribution over the points on the line segment connecting z to baseline, approximated by a sample of resolution points equally spaced along this segment.

PARAMETER DESCRIPTION
cut

The Cut in which the DoI will be applied. If None, the DoI will be applied to the input. otherwise, the distribution should be applied to the latent space defined by the cut.

TYPE: Cut, optional, from DoI DEFAULT: None

baseline

The baseline to interpolate from. Must be same shape as the space the distribution acts over, i.e., the shape of the points, z, eventually passed to __call__. If cut is None, this must be the same shape as the input, otherwise this must be the same shape as the latent space defined by the cut. If None is given, baseline will be the zero vector in the appropriate shape. If the baseline is callable, it is expected to return the baseline, given z and optional model arguments.

TYPE: BaselineLike DEFAULT: None

resolution

Number of points returned by each call to this DoI. A higher resolution is more computationally expensive, but gives a better approximation of the DoI this object mathematically represents.

TYPE: int DEFAULT: 10

get_activation_multiplier
get_activation_multiplier(activation: OM[Inputs, TensorLike], *, model_inputs: Optional[ModelInputs] = None) -> Inputs[TensorLike]

Returns a term to multiply the gradient by to convert from "influence space" to "attribution space". Conceptually, "influence space" corresponds to the potential effect of a slight increase in each feature, while "attribution space" corresponds to an approximation of the net marginal contribution to the quantity of interest of each feature.

PARAMETER DESCRIPTION
activation

The activation of the layer the DoI is applied to.

TYPE: OM[Inputs, TensorLike]

RETURNS DESCRIPTION
Inputs[TensorLike]

The activation adjusted by the baseline passed to the constructor.

GaussianDoi

Bases: DoI

Distribution representing a Gaussian ball around the point. Used by Smooth Gradients.

Functions
__init__
__init__(var: float, resolution: int, cut: Cut = None)
PARAMETER DESCRIPTION
var

The variance of the Gaussian noise to be added around the point.

TYPE: float

resolution

Number of samples returned by each call to this DoI.

TYPE: int

cut

The Cut in which the DoI will be applied. If None, the DoI will be applied to the input. otherwise, the distribution should be applied to the latent space defined by the cut.

TYPE: Cut DEFAULT: None