Distributions of Interest¶
¶
The distribution of interest lets us specify the set of samples over which we want our explanations to be faithful. In some cases, we may want to explain the modelâ€™s behavior on a particular record, whereas other times we may be interested in a more general behavior over a distribution of samples.
DoI
¶
Interface for distributions of interest. The Distribution of Interest (DoI) specifies the samples over which an attribution method is aggregated.
__call__(self, z)
special
¶
Computes the distribution of interest from an initial point.
Parameters:
Name  Type  Description  Default 

z 
Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]] 
Input point from which the distribution is derived. 
required 
Returns:
Type  Description 

List[Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]]] 
List of points which are all assigned equal probability mass in the
distribution of interest, i.e., the distribution of interest is
a discrete, uniform distribution over the list of returned points.
Each point in the list shares the same type and shape as 
Source code in trulens/nn/distributions.py
@abstractmethod
def __call__(self, z: ArrayLike) > List[ArrayLike]:
"""
Computes the distribution of interest from an initial point.
Parameters:
z:
Input point from which the distribution is derived.
Returns:
List of points which are all assigned equal probability mass in the
distribution of interest, i.e., the distribution of interest is
a discrete, uniform distribution over the list of returned points.
Each point in the list shares the same type and shape as `z`.
"""
raise NotImplementedError
__init__(self, cut=None)
special
¶
"Initialize DoI
Parameters:
Name  Type  Description  Default 

cut 
Cut 
The Cut in which the DoI will be applied. If 
None 
Source code in trulens/nn/distributions.py
def __init__(self, cut: Cut = None):
""""Initialize DoI
Parameters:
cut (Cut, optional):
The Cut in which the DoI will be applied. If `None`, the DoI will be
applied to the input. otherwise, the distribution should be applied
to the latent space defined by the cut.
"""
self._cut = cut
cut(self)
¶
Returns:
Type  Description 

Cut 
The Cut in which the DoI will be applied. If 
Source code in trulens/nn/distributions.py
def cut(self) > Cut:
"""
Returns:
The Cut in which the DoI will be applied. If `None`, the DoI will be
applied to the input. otherwise, the distribution should be applied
to the latent space defined by the cut.
"""
return self._cut
get_activation_multiplier(self, activation)
¶
Returns a term to multiply the gradient by to convert from "influence space" to "attribution space". Conceptually, "influence space" corresponds to the potential effect of a slight increase in each feature, while "attribution space" corresponds to an approximation of the net marginal contribution to the quantity of interest of each feature.
Parameters:
Name  Type  Description  Default 

activation 
Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]] 
The activation of the layer the DoI is applied to. 
required 
Returns:
Type  Description 

Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]] 
An array with the same shape as 
Source code in trulens/nn/distributions.py
def get_activation_multiplier(self, activation: ArrayLike) > ArrayLike:
"""
Returns a term to multiply the gradient by to convert from "*influence
space*" to "*attribution space*". Conceptually, "influence space"
corresponds to the potential effect of a slight increase in each
feature, while "attribution space" corresponds to an approximation of
the net marginal contribution to the quantity of interest of each
feature.
Parameters:
activation:
The activation of the layer the DoI is applied to.
Returns:
An array with the same shape as ``activation`` that will be
multiplied by the gradient to obtain the attribution. The default
implementation of this method simply returns ``activation``.
"""
return activation
DoiCutSupportError
¶
Exception raised if the distribution of interest is called on a cut whose output is not supported by the distribution of interest.
GaussianDoi
¶
Distribution representing a Gaussian ball around the point. Used by Smooth Gradients.
__init__(self, var, resolution, cut=None)
special
¶
Parameters:
Name  Type  Description  Default 

var 
float 
The variance of the Gaussian noise to be added around the point. 
required 
resolution 
int 
Number of samples returned by each call to this DoI. 
required 
cut 
Cut 
The Cut in which the DoI will be applied. If 
None 
Source code in trulens/nn/distributions.py
def __init__(self, var: float, resolution: int, cut: Cut = None):
"""
Parameters:
var:
The variance of the Gaussian noise to be added around the point.
resolution:
Number of samples returned by each call to this DoI.
cut (Cut, optional):
The Cut in which the DoI will be applied. If `None`, the DoI will be
applied to the input. otherwise, the distribution should be applied
to the latent space defined by the cut.
"""
super(GaussianDoi, self).__init__(cut)
self._var = var
self._resolution = resolution
LinearDoi
¶
Distribution representing the linear interpolation between a baseline and the given point. Used by Integrated Gradients.
__init__(self, baseline=None, resolution=10, cut=None)
special
¶
The DoI for point, z
, will be a uniform distribution over the points
on the line segment connecting z
to baseline
, approximated by a
sample of resolution
points equally spaced along this segment.
Parameters:
Name  Type  Description  Default 

baseline 
Optional[Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]]] 
The baseline to interpolate from. Must be same shape as the
space the distribution acts over, i.e., the shape of the points,

None 
resolution 
int 
Number of points returned by each call to this DoI. A higher resolution is more computationally expensive, but gives a better approximation of the DoI this object mathematically represents. 
10 
cut 
Cut 
The Cut in which the DoI will be applied. If 
None 
Source code in trulens/nn/distributions.py
def __init__(
self,
baseline: Optional[ArrayLike] = None,
resolution: int = 10,
cut: Cut = None):
"""
The DoI for point, `z`, will be a uniform distribution over the points
on the line segment connecting `z` to `baseline`, approximated by a
sample of `resolution` points equally spaced along this segment.
Parameters:
baseline:
The baseline to interpolate from. Must be same shape as the
space the distribution acts over, i.e., the shape of the points,
`z`, eventually passed to `__call__`. If `cut` is `None`, this
must be the same shape as the input, otherwise this must be the
same shape as the latent space defined by the cut. If `None` is
given, `baseline` will be the zero vector in the appropriate
shape.
resolution:
Number of points returned by each call to this DoI. A higher
resolution is more computationally expensive, but gives a better
approximation of the DoI this object mathematically represents.
cut (Cut, optional):
The Cut in which the DoI will be applied. If `None`, the DoI will be
applied to the input. otherwise, the distribution should be applied
to the latent space defined by the cut.
"""
super(LinearDoi, self).__init__(cut)
self._baseline = baseline
self._resolution = resolution
get_activation_multiplier(self, activation)
¶
Returns a term to multiply the gradient by to convert from "influence space" to "attribution space". Conceptually, "influence space" corresponds to the potential effect of a slight increase in each feature, while "attribution space" corresponds to an approximation of the net marginal contribution to the quantity of interest of each feature.
Parameters:
Name  Type  Description  Default 

activation 
Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]] 
The activation of the layer the DoI is applied to. 
required 
Returns:
Type  Description 

Union[numpy.ndarray, Any, List[Union[numpy.ndarray, Any]]] 
The activation adjusted by the baseline passed to the constructor. 
Source code in trulens/nn/distributions.py
def get_activation_multiplier(self, activation: ArrayLike) > ArrayLike:
"""
Returns a term to multiply the gradient by to convert from "*influence
space*" to "*attribution space*". Conceptually, "influence space"
corresponds to the potential effect of a slight increase in each
feature, while "attribution space" corresponds to an approximation of
the net marginal contribution to the quantity of interest of each
feature.
Parameters:
activation:
The activation of the layer the DoI is applied to.
Returns:
The activation adjusted by the baseline passed to the constructor.
"""
return (
activation if self._baseline is None else activation 
self._baseline)
PointDoi
¶
Distribution that puts all probability mass on a single point.
__init__(self, cut=None)
special
¶
"Initialize PointDoI
Parameters:
Name  Type  Description  Default 

cut 
Cut 
The Cut in which the DoI will be applied. If 
None 
Source code in trulens/nn/distributions.py
def __init__(self, cut: Cut = None):
""""Initialize PointDoI
Parameters:
cut (Cut, optional):
The Cut in which the DoI will be applied. If `None`, the DoI will be
applied to the input. otherwise, the distribution should be applied
to the latent space defined by the cut.
"""
super(PointDoi, self).__init__(cut)