# Quantities of Interest¶

## ¶

A *Quantity of Interest* (QoI) is a function of the output that determines the
network output behavior that the attributions describe.

The quantity of interest lets us specify what we want to explain. Often, this is
the output of the network corresponding to a particular class, addressing, e.g.,
"Why did the model classify a given image as a car?" However, we could also
consider various combinations of outputs, allowing us to ask more specific
questions, such as, "Why did the model classify a given image as a sedan *and
not a convertible*?" The former may highlight general “car features,” such as
tires, while the latter (called a comparative explanation) might focus on the
roof of the car, a “car feature” not shared by convertibles.

##
`ClassQoI`

¶

Quantity of interest for attributing output towards a specified class.

###
```
__init__(self, cl)
```

`special`

¶

**Parameters:**

Name | Type | Description | Default |
---|---|---|---|

`cl` |
`int` |
The index of the class the QoI is for. |
required |

## Source code in `trulens/nn/quantities.py`

```
def __init__(self, cl: int):
"""
Parameters:
cl:
The index of the class the QoI is for.
"""
self.cl = cl
```

##
`ClassSeqQoI`

¶

Quantity of interest for attributing output towards a sequence of classes for each input.

###
```
__init__(self, seq_labels)
```

`special`

¶

**Parameters:**

Name | Type | Description | Default |
---|---|---|---|

`seq_labels` |
`List[int]` |
A sequence of classes corresponding to each input. |
required |

## Source code in `trulens/nn/quantities.py`

```
def __init__(self, seq_labels: List[int]):
"""
Parameters:
seq_labels:
A sequence of classes corresponding to each input.
"""
self.seq_labels = seq_labels
```

##
`ComparativeQoI`

¶

Quantity of interest for attributing network output towards a given class, relative to another.

###
```
__init__(self, cl1, cl2)
```

`special`

¶

**Parameters:**

Name | Type | Description | Default |
---|---|---|---|

`cl1` |
`int` |
The index of the class the QoI is for. |
required |

`cl2` |
`int` |
The index of the class to compare against. |
required |

## Source code in `trulens/nn/quantities.py`

```
def __init__(self, cl1: int, cl2: int):
"""
Parameters:
cl1:
The index of the class the QoI is for.
cl2:
The index of the class to compare against.
"""
self.cl1 = cl1
self.cl2 = cl2
```

##
`InternalChannelQoI`

¶

Quantity of interest for attributing output towards the output of an internal convolutional layer channel, aggregating using a specified operation.

Also works for non-convolutional dense layers, where the given neuron's activation is returned.

###
```
__init__(self, channel, channel_axis=None, agg_fn=None)
```

`special`

¶

**Parameters:**

Name | Type | Description | Default |
---|---|---|---|

`channel` |
`Union[int, List[int]]` |
Channel to return. If a list is provided, then the quantity sums over each of the channels in the list. |
required |

`channel_axis` |
`Optional[int]` |
Channel dimension index, if relevant, e.g., for 2D convolutional
layers. If |
`None` |

`agg_fn` |
`Optional[Callable]` |
Function with which to aggregate the remaining dimensions
(except the batch dimension) in order to get a single scalar
value for each channel. If |
`None` |

## Source code in `trulens/nn/quantities.py`

```
def __init__(
self,
channel: Union[int, List[int]],
channel_axis: Optional[int] = None,
agg_fn: Optional[Callable] = None):
"""
Parameters:
channel:
Channel to return. If a list is provided, then the quantity sums
over each of the channels in the list.
channel_axis:
Channel dimension index, if relevant, e.g., for 2D convolutional
layers. If `channel_axis` is `None`, then the channel axis of
the relevant backend will be used. This argument is not used
when the channels are scalars, e.g., for dense layers.
agg_fn:
Function with which to aggregate the remaining dimensions
(except the batch dimension) in order to get a single scalar
value for each channel. If `agg_fn` is `None` then a sum over
each neuron in the channel will be taken. This argument is not
used when the channels are scalars, e.g., for dense layers.
"""
if channel_axis is None:
channel_axis = get_backend().channel_axis
if agg_fn is None:
agg_fn = InternalChannelQoI._batch_sum
self._channel_ax = channel_axis
self._agg_fn = agg_fn
self._channels = channel if isinstance(channel, list) else [channel]
```

##
`LambdaQoI`

¶

Generic quantity of interest allowing the user to specify a function of the model's output as the QoI.

###
```
__init__(self, function)
```

`special`

¶

**Parameters:**

Name | Type | Description | Default |
---|---|---|---|

`function` |
`Callable` |
A callable that takes a single argument representing the model's tensor output and returns a differentiable batched scalar tensor representing the QoI. |
required |

## Source code in `trulens/nn/quantities.py`

```
def __init__(self, function: Callable):
"""
Parameters:
function:
A callable that takes a single argument representing the model's
tensor output and returns a differentiable batched scalar tensor
representing the QoI.
"""
if len(signature(function).parameters) != 1:
raise ValueError(
'QoI function must take exactly 1 argument, but provided '
'function takes {} arguments'.format(
len(signature(function).parameters)))
self.function = function
```

##
`MaxClassQoI`

¶

Quantity of interest for attributing output towards the maximum-predicted class.

###
```
__init__(self, axis=1, activation=None)
```

`special`

¶

**Parameters:**

Name | Type | Description | Default |
---|---|---|---|

`axis` |
`int` |
Output dimension over which max operation is taken. |
`1` |

`activation` |
`Optional[Union[Callable, str]]` |
Activation function to be applied to the output before taking
the max. If `'sigmoid'` `'softmax'`
If |
`None` |

## Source code in `trulens/nn/quantities.py`

```
def __init__(
self, axis: int = 1, activation: Union[Callable, str, None] = None):
"""
Parameters:
axis:
Output dimension over which max operation is taken.
activation:
Activation function to be applied to the output before taking
the max. If `activation` is a string, use the corresponding
named activation function implemented by the backend. The
following strings are currently supported as shorthands for the
respective standard activation functions:
- `'sigmoid'`
- `'softmax'`
If `activation` is `None`, no activation function is applied to
the input.
"""
self._axis = axis
self.activation = activation
```

##
`QoI`

¶

Interface for quantities of interest. The *Quantity of Interest* (QoI) is a
function of the output specified by the slice that determines the network
output behavior that the attributions describe.

###
```
__call__(self, y)
```

`special`

¶

Computes the distribution of interest from an initial point.

**Parameters:**

Name | Type | Description | Default |
---|---|---|---|

`y` |
`Union[Any, List[Any]]` |
Output point from which the quantity is derived. Must be a differentiable tensor. |
required |

**Returns:**

Type | Description |
---|---|

`Union[Any, List[Any]]` |
A differentiable batched scalar tensor representing the QoI. |

## Source code in `trulens/nn/quantities.py`

```
@abstractmethod
def __call__(self, y: TensorLike) -> TensorLike:
"""
Computes the distribution of interest from an initial point.
Parameters:
y:
Output point from which the quantity is derived. Must be a
differentiable tensor.
Returns:
A differentiable batched scalar tensor representing the QoI.
"""
raise NotImplementedError
```

##
`QoiCutSupportError`

¶

Exception raised if the quantity of interest is called on a cut whose output is not supported by the quantity of interest.

##
`ThresholdQoI`

¶

Quantity of interest for attributing network output toward the difference between two regions seperated by a given threshold. I.e., the quantity of interest is the "high" elements minus the "low" elements, where the high elements have activations above the threshold and the low elements have activations below the threshold.

Use case: bianry segmentation.

###
```
__init__(self, threshold, low_minus_high=False, activation=None)
```

`special`

¶

**Parameters:**

Name | Type | Description | Default |
---|---|---|---|

`threshold` |
`float` |
A threshold to determine the element-wise sign of the input
tensor. The elements with activations higher than the threshold
will retain their sign, while the elements with activations
lower than the threshold will have their sign flipped (or vice
versa if |
required |

`low_minus_high` |
`bool` |
If |
`False` |

`activation` |
`Optional[Union[Callable, str]]` |
str or function, optional
Activation function to be applied to the quantity before taking
the threshold. If |
`None` |

## Source code in `trulens/nn/quantities.py`

```
def __init__(
self,
threshold: float,
low_minus_high: bool = False,
activation: Union[Callable, str, None] = None):
"""
Parameters:
threshold:
A threshold to determine the element-wise sign of the input
tensor. The elements with activations higher than the threshold
will retain their sign, while the elements with activations
lower than the threshold will have their sign flipped (or vice
versa if `low_minus_high` is set to `True`).
low_minus_high:
If `True`, substract the output with activations above the
threshold from the output with activations below the threshold.
If `False`, substract the output with activations below the
threshold from the output with activations above the threshold.
activation: str or function, optional
Activation function to be applied to the quantity before taking
the threshold. If `activation` is a string, use the
corresponding activation function implemented by the backend
(currently supported: `'sigmoid'` and `'softmax'`). Otherwise,
if `activation` is not `None`, it will be treated as a callable.
If `activation` is `None`, do not apply an activation function
to the quantity.
"""
# TODO(klas):should this support an aggregation function? By default
# this is a sum, but it could, for example, subtract the greatest
# positive element from the least negative element.
self.threshold = threshold
self.low_minus_high = low_minus_high
self.activation = activation
```