Quantities of Interest

trulens.nn.quantities

A Quantity of Interest (QoI) is a function of the output that determines the network output behavior that the attributions describe.

The quantity of interest lets us specify what we want to explain. Often, this is the output of the network corresponding to a particular class, addressing, e.g., "Why did the model classify a given image as a car?" However, we could also consider various combinations of outputs, allowing us to ask more specific questions, such as, "Why did the model classify a given image as a sedan and not a convertible?" The former may highlight general “car features,” such as tires, while the latter (called a comparative explanation) might focus on the roof of the car, a “car feature” not shared by convertibles.

Classes

QoiCutSupportError

Bases: ValueError

Exception raised if the quantity of interest is called on a cut whose output is not supported by the quantity of interest.

QoI

Bases: ABC

Interface for quantities of interest. The Quantity of Interest (QoI) is a function of the output specified by the slice that determines the network output behavior that the attributions describe.

Functions

__call__ abstractmethod

__call__(y: OM[Outputs, Tensor]) -> OM[Outputs, Tensor]

Computes the distribution of interest from an initial point.

PARAMETER	DESCRIPTION
y	Output point from which the quantity is derived. Must be a differentiable tensor. TYPE: OM[Outputs, Tensor]

RETURNS	DESCRIPTION
OM[Outputs, Tensor]	A differentiable batched scalar tensor representing the QoI.

MaxClassQoI

Bases: QoI

Quantity of interest for attributing output towards the maximum-predicted class.

Functions

__init__

__init__(axis: int = 1, activation: Union[Callable, str, None] = None)

PARAMETER	DESCRIPTION
axis	Output dimension over which max operation is taken. TYPE: int DEFAULT: 1
activation	Activation function to be applied to the output before taking the max. If activation is a string, use the corresponding named activation function implemented by the backend. The following strings are currently supported as shorthands for the respective standard activation functions: 'sigmoid' 'softmax' If activation is None, no activation function is applied to the input. TYPE: Union[Callable, str, None] DEFAULT: None

InternalChannelQoI

Bases: QoI

Quantity of interest for attributing output towards the output of an internal convolutional layer channel, aggregating using a specified operation.

Also works for non-convolutional dense layers, where the given neuron's activation is returned.

Functions

__init__

__init__(channel: Union[int, List[int]], channel_axis: Optional[int] = None, agg_fn: Optional[Callable] = None)

PARAMETER	DESCRIPTION
channel	Channel to return. If a list is provided, then the quantity sums over each of the channels in the list. TYPE: Union[int, List[int]]
channel_axis	Channel dimension index, if relevant, e.g., for 2D convolutional layers. If channel_axis is None, then the channel axis of the relevant backend will be used. This argument is not used when the channels are scalars, e.g., for dense layers. TYPE: Optional[int] DEFAULT: None
agg_fn	Function with which to aggregate the remaining dimensions (except the batch dimension) in order to get a single scalar value for each channel. If agg_fn is None then a sum over each neuron in the channel will be taken. This argument is not used when the channels are scalars, e.g., for dense layers. TYPE: Optional[Callable] DEFAULT: None

ClassQoI

Bases: QoI

Quantity of interest for attributing output towards a specified class.

Functions

__init__

__init__(cl: int)

PARAMETER	DESCRIPTION
cl	The index of the class the QoI is for. TYPE: int

ComparativeQoI

Bases: QoI

Quantity of interest for attributing network output towards a given class, relative to another.

Functions

__init__

__init__(cl1: int, cl2: int)

PARAMETER	DESCRIPTION
cl1	The index of the class the QoI is for. TYPE: int
cl2	The index of the class to compare against. TYPE: int

LambdaQoI

Bases: QoI

Generic quantity of interest allowing the user to specify a function of the model's output as the QoI.

Functions

__init__

__init__(function: Callable)

PARAMETER	DESCRIPTION
function	A callable that takes a single argument representing the model's tensor output and returns a differentiable batched scalar tensor representing the QoI. TYPE: Callable

ThresholdQoI

Bases: QoI

Quantity of interest for attributing network output toward the difference between two regions seperated by a given threshold. I.e., the quantity of interest is the "high" elements minus the "low" elements, where the high elements have activations above the threshold and the low elements have activations below the threshold.

Use case: bianry segmentation.

Functions

__init__

__init__(threshold: float, low_minus_high: bool = False, activation: Union[Callable, str, None] = None)

PARAMETER	DESCRIPTION
threshold	A threshold to determine the element-wise sign of the input tensor. The elements with activations higher than the threshold will retain their sign, while the elements with activations lower than the threshold will have their sign flipped (or vice versa if low_minus_high is set to True). TYPE: float
low_minus_high	If True, substract the output with activations above the threshold from the output with activations below the threshold. If False, substract the output with activations below the threshold from the output with activations above the threshold. TYPE: bool DEFAULT: False
activation	str or function, optional Activation function to be applied to the quantity before taking the threshold. If activation is a string, use the corresponding activation function implemented by the backend (currently supported: 'sigmoid' and 'softmax'). Otherwise, if activation is not None, it will be treated as a callable. If activation is None, do not apply an activation function to the quantity. TYPE: Union[Callable, str, None] DEFAULT: None

ClassSeqQoI

Bases: QoI

Quantity of interest for attributing output towards a sequence of classes for each input.

Functions

__init__

__init__(seq_labels: List[int])

PARAMETER	DESCRIPTION
seq_labels	A sequence of classes corresponding to each input. TYPE: List[int]

Functions