Distributions of Interest¶

The distribution of interest lets us specify the set of samples over which we want our explanations to be faithful. In some cases, we may want to explain the model’s behavior on a particular record, whereas other times we may be interested in a more general behavior over a distribution of samples.

DoI¶

Interface for distributions of interest. The Distribution of Interest (DoI) specifies the samples over which an attribution method is aggregated.

__call__(self, z) special ¶

Computes the distribution of interest from an initial point.

Parameters:

Name Type Description Default
z Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]]

Input point from which the distribution is derived.

required

Returns:

Type Description
List[Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]]]

List of points which are all assigned equal probability mass in the distribution of interest, i.e., the distribution of interest is a discrete, uniform distribution over the list of returned points. Each point in the list shares the same type and shape as z.

Source code in trulens/nn/distributions.py
@abstractmethod
def __call__(self, z: ArrayLike) -> List[ArrayLike]:
"""
Computes the distribution of interest from an initial point.

Parameters:
z:
Input point from which the distribution is derived.

Returns:
List of points which are all assigned equal probability mass in the
distribution of interest, i.e., the distribution of interest is
a discrete, uniform distribution over the list of returned points.
Each point in the list shares the same type and shape as z.
"""
raise NotImplementedError

__init__(self, cut=None) special ¶

"Initialize DoI

Parameters:

Name Type Description Default
cut Cut

The Cut in which the DoI will be applied. If None, the DoI will be applied to the input. otherwise, the distribution should be applied to the latent space defined by the cut.

None
Source code in trulens/nn/distributions.py
def __init__(self, cut: Cut = None):
""""Initialize DoI

Parameters:
cut (Cut, optional):
The Cut in which the DoI will be applied. If None, the DoI will be
applied to the input. otherwise, the distribution should be applied
to the latent space defined by the cut.
"""
self._cut = cut

cut(self) ¶

Returns:

Type Description
Cut

The Cut in which the DoI will be applied. If None, the DoI will be applied to the input. otherwise, the distribution should be applied to the latent space defined by the cut.

Source code in trulens/nn/distributions.py
def cut(self) -> Cut:
"""
Returns:
The Cut in which the DoI will be applied. If None, the DoI will be
applied to the input. otherwise, the distribution should be applied
to the latent space defined by the cut.
"""
return self._cut

get_activation_multiplier(self, activation) ¶

Returns a term to multiply the gradient by to convert from "influence space" to "attribution space". Conceptually, "influence space" corresponds to the potential effect of a slight increase in each feature, while "attribution space" corresponds to an approximation of the net marginal contribution to the quantity of interest of each feature.

Parameters:

Name Type Description Default
activation Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]]

The activation of the layer the DoI is applied to.

required

Returns:

Type Description
Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]]

An array with the same shape as activation that will be multiplied by the gradient to obtain the attribution. The default implementation of this method simply returns activation.

Source code in trulens/nn/distributions.py
def get_activation_multiplier(self, activation: ArrayLike) -> ArrayLike:
"""
Returns a term to multiply the gradient by to convert from "*influence
space*" to "*attribution space*". Conceptually, "influence space"
corresponds to the potential effect of a slight increase in each
feature, while "attribution space" corresponds to an approximation of
the net marginal contribution to the quantity of interest of each
feature.

Parameters:
activation:
The activation of the layer the DoI is applied to.

Returns:
An array with the same shape as activation that will be
implementation of this method simply returns activation.
"""
return activation

DoiCutSupportError¶

Exception raised if the distribution of interest is called on a cut whose output is not supported by the distribution of interest.

GaussianDoi¶

Distribution representing a Gaussian ball around the point. Used by Smooth Gradients.

__init__(self, var, resolution, cut=None) special ¶

Parameters:

Name Type Description Default
var float

The variance of the Gaussian noise to be added around the point.

required
resolution int

Number of samples returned by each call to this DoI.

required
cut Cut

The Cut in which the DoI will be applied. If None, the DoI will be applied to the input. otherwise, the distribution should be applied to the latent space defined by the cut.

None
Source code in trulens/nn/distributions.py
def __init__(self, var: float, resolution: int, cut: Cut = None):
"""
Parameters:
var:
The variance of the Gaussian noise to be added around the point.

resolution:
Number of samples returned by each call to this DoI.
cut (Cut, optional):
The Cut in which the DoI will be applied. If None, the DoI will be
applied to the input. otherwise, the distribution should be applied
to the latent space defined by the cut.
"""
super(GaussianDoi, self).__init__(cut)
self._var = var
self._resolution = resolution

LinearDoi¶

Distribution representing the linear interpolation between a baseline and the given point. Used by Integrated Gradients.

__init__(self, baseline=None, resolution=10, cut=None) special ¶

The DoI for point, z, will be a uniform distribution over the points on the line segment connecting z to baseline, approximated by a sample of resolution points equally spaced along this segment.

Parameters:

Name Type Description Default
baseline Optional[Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]]]

The baseline to interpolate from. Must be same shape as the space the distribution acts over, i.e., the shape of the points, z, eventually passed to __call__. If cut is None, this must be the same shape as the input, otherwise this must be the same shape as the latent space defined by the cut. If None is given, baseline will be the zero vector in the appropriate shape.

None
resolution int

Number of points returned by each call to this DoI. A higher resolution is more computationally expensive, but gives a better approximation of the DoI this object mathematically represents.

10
cut Cut

The Cut in which the DoI will be applied. If None, the DoI will be applied to the input. otherwise, the distribution should be applied to the latent space defined by the cut.

None
Source code in trulens/nn/distributions.py
def __init__(
self,
baseline: Optional[ArrayLike] = None,
resolution: int = 10,
cut: Cut = None):
"""
The DoI for point, z, will be a uniform distribution over the points
on the line segment connecting z to baseline, approximated by a
sample of resolution points equally spaced along this segment.

Parameters:
baseline:
The baseline to interpolate from. Must be same shape as the
space the distribution acts over, i.e., the shape of the points,
z, eventually passed to __call__. If cut is None, this
must be the same shape as the input, otherwise this must be the
same shape as the latent space defined by the cut. If None is
given, baseline will be the zero vector in the appropriate
shape.

resolution:
Number of points returned by each call to this DoI. A higher
resolution is more computationally expensive, but gives a better
approximation of the DoI this object mathematically represents.

cut (Cut, optional):
The Cut in which the DoI will be applied. If None, the DoI will be
applied to the input. otherwise, the distribution should be applied
to the latent space defined by the cut.
"""
super(LinearDoi, self).__init__(cut)
self._baseline = baseline
self._resolution = resolution

get_activation_multiplier(self, activation) ¶

Returns a term to multiply the gradient by to convert from "influence space" to "attribution space". Conceptually, "influence space" corresponds to the potential effect of a slight increase in each feature, while "attribution space" corresponds to an approximation of the net marginal contribution to the quantity of interest of each feature.

Parameters:

Name Type Description Default
activation Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]]

The activation of the layer the DoI is applied to.

required

Returns:

Type Description
Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]]

The activation adjusted by the baseline passed to the constructor.

Source code in trulens/nn/distributions.py
def get_activation_multiplier(self, activation: ArrayLike) -> ArrayLike:
"""
Returns a term to multiply the gradient by to convert from "*influence
space*" to "*attribution space*". Conceptually, "influence space"
corresponds to the potential effect of a slight increase in each
feature, while "attribution space" corresponds to an approximation of
the net marginal contribution to the quantity of interest of each
feature.

Parameters:
activation:
The activation of the layer the DoI is applied to.

Returns:
The activation adjusted by the baseline passed to the constructor.
"""
return (
activation if self._baseline is None else activation -
self._baseline)

PointDoi¶

Distribution that puts all probability mass on a single point.

__init__(self, cut=None) special ¶

"Initialize PointDoI

Parameters:

Name Type Description Default
cut Cut

The Cut in which the DoI will be applied. If None, the DoI will be applied to the input. otherwise, the distribution should be applied to the latent space defined by the cut.

None
Source code in trulens/nn/distributions.py
def __init__(self, cut: Cut = None):
""""Initialize PointDoI

Parameters:
cut (Cut, optional):
The Cut in which the DoI will be applied. If None, the DoI will be
applied to the input. otherwise, the distribution should be applied
to the latent space defined by the cut.
"""
super(PointDoi, self).__init__(cut)