Skip to content

Distributions of Interest

The distribution of interest lets us specify the set of samples over which we want our explanations to be faithful. In some cases, we may want to explain the model’s behavior on a particular record, whereas other times we may be interested in a more general behavior over a distribution of samples.

DoI

Interface for distributions of interest. The Distribution of Interest (DoI) specifies the samples over which an attribution method is aggregated.

__call__(self, z) special

Computes the distribution of interest from an initial point.

Parameters:

Name Type Description Default
z Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]]

Input point from which the distribution is derived.

required

Returns:

Type Description
List[Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]]]

List of points which are all assigned equal probability mass in the distribution of interest, i.e., the distribution of interest is a discrete, uniform distribution over the list of returned points. Each point in the list shares the same type and shape as z.

Source code in trulens/nn/distributions.py
@abstractmethod
def __call__(self, z: ArrayLike) -> List[ArrayLike]:
    """
    Computes the distribution of interest from an initial point.

    Parameters:
        z:
            Input point from which the distribution is derived.

    Returns:
        List of points which are all assigned equal probability mass in the
        distribution of interest, i.e., the distribution of interest is
        a discrete, uniform distribution over the list of returned points.
        Each point in the list shares the same type and shape as `z`.
    """
    raise NotImplementedError

__init__(self, cut=None) special

"Initialize DoI

Parameters:

Name Type Description Default
cut Cut

The Cut in which the DoI will be applied. If None, the DoI will be applied to the input. otherwise, the distribution should be applied to the latent space defined by the cut.

None
Source code in trulens/nn/distributions.py
def __init__(self, cut: Cut = None):
    """"Initialize DoI

    Parameters:
        cut (Cut, optional): 
            The Cut in which the DoI will be applied. If `None`, the DoI will be
            applied to the input. otherwise, the distribution should be applied
            to the latent space defined by the cut. 
    """
    self._cut = cut

cut(self)

Returns:

Type Description
Cut

The Cut in which the DoI will be applied. If None, the DoI will be applied to the input. otherwise, the distribution should be applied to the latent space defined by the cut.

Source code in trulens/nn/distributions.py
def cut(self) -> Cut:
    """
    Returns:
        The Cut in which the DoI will be applied. If `None`, the DoI will be
        applied to the input. otherwise, the distribution should be applied
        to the latent space defined by the cut. 
    """
    return self._cut

get_activation_multiplier(self, activation)

Returns a term to multiply the gradient by to convert from "influence space" to "attribution space". Conceptually, "influence space" corresponds to the potential effect of a slight increase in each feature, while "attribution space" corresponds to an approximation of the net marginal contribution to the quantity of interest of each feature.

Parameters:

Name Type Description Default
activation Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]]

The activation of the layer the DoI is applied to.

required

Returns:

Type Description
Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]]

An array with the same shape as activation that will be multiplied by the gradient to obtain the attribution. The default implementation of this method simply returns activation.

Source code in trulens/nn/distributions.py
def get_activation_multiplier(self, activation: ArrayLike) -> ArrayLike:
    """
    Returns a term to multiply the gradient by to convert from "*influence 
    space*" to "*attribution space*". Conceptually, "influence space"
    corresponds to the potential effect of a slight increase in each 
    feature, while "attribution space" corresponds to an approximation of
    the net marginal contribution to the quantity of interest of each 
    feature.

    Parameters:
        activation:
            The activation of the layer the DoI is applied to.

    Returns:
        An array with the same shape as ``activation`` that will be 
        multiplied by the gradient to obtain the attribution. The default 
        implementation of this method simply returns ``activation``.
    """
    return activation

DoiCutSupportError

Exception raised if the distribution of interest is called on a cut whose output is not supported by the distribution of interest.

GaussianDoi

Distribution representing a Gaussian ball around the point. Used by Smooth Gradients.

__init__(self, var, resolution, cut=None) special

Parameters:

Name Type Description Default
var float

The variance of the Gaussian noise to be added around the point.

required
resolution int

Number of samples returned by each call to this DoI.

required
cut Cut

The Cut in which the DoI will be applied. If None, the DoI will be applied to the input. otherwise, the distribution should be applied to the latent space defined by the cut.

None
Source code in trulens/nn/distributions.py
def __init__(self, var: float, resolution: int, cut: Cut = None):
    """
    Parameters:
        var:
            The variance of the Gaussian noise to be added around the point.

        resolution:
            Number of samples returned by each call to this DoI.
        cut (Cut, optional): 
            The Cut in which the DoI will be applied. If `None`, the DoI will be
            applied to the input. otherwise, the distribution should be applied
            to the latent space defined by the cut. 
    """
    super(GaussianDoi, self).__init__(cut)
    self._var = var
    self._resolution = resolution

LinearDoi

Distribution representing the linear interpolation between a baseline and the given point. Used by Integrated Gradients.

__init__(self, baseline=None, resolution=10, cut=None) special

The DoI for point, z, will be a uniform distribution over the points on the line segment connecting z to baseline, approximated by a sample of resolution points equally spaced along this segment.

Parameters:

Name Type Description Default
baseline Optional[Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]]]

The baseline to interpolate from. Must be same shape as the space the distribution acts over, i.e., the shape of the points, z, eventually passed to __call__. If cut is None, this must be the same shape as the input, otherwise this must be the same shape as the latent space defined by the cut. If None is given, baseline will be the zero vector in the appropriate shape.

None
resolution int

Number of points returned by each call to this DoI. A higher resolution is more computationally expensive, but gives a better approximation of the DoI this object mathematically represents.

10
cut Cut

The Cut in which the DoI will be applied. If None, the DoI will be applied to the input. otherwise, the distribution should be applied to the latent space defined by the cut.

None
Source code in trulens/nn/distributions.py
def __init__(
        self,
        baseline: Optional[ArrayLike] = None,
        resolution: int = 10,
        cut: Cut = None):
    """
    The DoI for point, `z`, will be a uniform distribution over the points
    on the line segment connecting `z` to `baseline`, approximated by a
    sample of `resolution` points equally spaced along this segment.

    Parameters:
        baseline:
            The baseline to interpolate from. Must be same shape as the 
            space the distribution acts over, i.e., the shape of the points, 
            `z`, eventually passed to `__call__`. If `cut` is `None`, this
            must be the same shape as the input, otherwise this must be the
            same shape as the latent space defined by the cut. If `None` is
            given, `baseline` will be the zero vector in the appropriate 
            shape.

        resolution:
            Number of points returned by each call to this DoI. A higher 
            resolution is more computationally expensive, but gives a better
            approximation of the DoI this object mathematically represents.

        cut (Cut, optional): 
            The Cut in which the DoI will be applied. If `None`, the DoI will be
            applied to the input. otherwise, the distribution should be applied
            to the latent space defined by the cut. 
    """
    super(LinearDoi, self).__init__(cut)
    self._baseline = baseline
    self._resolution = resolution

get_activation_multiplier(self, activation)

Returns a term to multiply the gradient by to convert from "influence space" to "attribution space". Conceptually, "influence space" corresponds to the potential effect of a slight increase in each feature, while "attribution space" corresponds to an approximation of the net marginal contribution to the quantity of interest of each feature.

Parameters:

Name Type Description Default
activation Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]]

The activation of the layer the DoI is applied to.

required

Returns:

Type Description
Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]]

The activation adjusted by the baseline passed to the constructor.

Source code in trulens/nn/distributions.py
def get_activation_multiplier(self, activation: ArrayLike) -> ArrayLike:
    """
    Returns a term to multiply the gradient by to convert from "*influence 
    space*" to "*attribution space*". Conceptually, "influence space"
    corresponds to the potential effect of a slight increase in each 
    feature, while "attribution space" corresponds to an approximation of
    the net marginal contribution to the quantity of interest of each 
    feature.

    Parameters:
        activation:
            The activation of the layer the DoI is applied to.

    Returns:
        The activation adjusted by the baseline passed to the constructor.
    """
    return (
        activation if self._baseline is None else activation -
        self._baseline)

PointDoi

Distribution that puts all probability mass on a single point.

__init__(self, cut=None) special

"Initialize PointDoI

Parameters:

Name Type Description Default
cut Cut

The Cut in which the DoI will be applied. If None, the DoI will be applied to the input. otherwise, the distribution should be applied to the latent space defined by the cut.

None
Source code in trulens/nn/distributions.py
def __init__(self, cut: Cut = None):
    """"Initialize PointDoI

    Parameters:
        cut (Cut, optional): 
            The Cut in which the DoI will be applied. If `None`, the DoI will be
            applied to the input. otherwise, the distribution should be applied
            to the latent space defined by the cut. 
    """
    super(PointDoi, self).__init__(cut)