Sensitivity index

Classification error probability $p_{e}$, and effective and approximate discriminability indices of two univariate normal distributions with unequal variances (top), and of two bivariate normal distributions with unequal covariance matrices (bottom, ellipses are 1 sd error ellipses). The classification boundary is in black. These are computed by numerical methods.

The sensitivity index or discriminability index or detectability index is a dimensionless statistic used in signal detection theory. A higher index indicates that the signal can be more readily detected.

Definition

The discriminability index is the separation between the means of two distributions (typically the signal and the noise distributions), in units of the standard deviation.

Equal variances/covariances

For two univariate distributions $a$ and $b$ with the same standard deviation, it is denoted by $d'$ ('dee-prime'):

$d'={\frac {\left\vert \mu _{a}-\mu _{b}\right\vert }{\sigma }}$.

In higher dimensions, i.e. with two multivariate distributions with the same variance-covariance matrix $\mathbf {\Sigma }$, (whose symmetric square-root, the standard deviation matrix, is $\mathbf {S}$), this generalizes to the Mahalanobis distance between the two distributions:

$d'={\sqrt {({\boldsymbol {\mu }}_{a}-{\boldsymbol {\mu }}_{b})'\mathbf {\Sigma } ^{-1}({\boldsymbol {\mu }}_{a}-{\boldsymbol {\mu }}_{b})}}=\lVert \mathbf {S} ^{-1}({\boldsymbol {\mu }}_{a}-{\boldsymbol {\mu }}_{b})\rVert =\lVert {\boldsymbol {\mu }}_{a}-{\boldsymbol {\mu }}_{b}\rVert /\sigma _{\hat {\boldsymbol {\mu }}}$,

where $\sigma _{\hat {\boldsymbol {\mu }}}=1/\lVert \mathbf {S} ^{-1}{\hat {\boldsymbol {\mu }}}\rVert$ is the 1d slice of the sd along the axis ${\hat {\boldsymbol {\mu }}}$ through the means, i.e. the $d'$ equals the $d'$ along the 1d slice through the means.

Unequal variances/covariances

When the two distributions have different standard deviations (or in general dimensions, different covariance matrices), there exist several contending indices, all of which reduce to $d'$ for equal variance/covariance. The Bayes discriminability index for two normal distributions is based on the amount of their overlap, i.e. the optimal (Bayes) error of classification $e_{b}$, or its complement, the optimal accuracy $a_{b}$:

$d_{b}=-2Z\left({\text{Bayes error rate }}e_{b}\right)=2Z\left({\text{best accuracy rate }}a_{b}\right)$,

where $Z$ is the inverse cumulative distribution function of the standard normal. This index measures the best possible (Bayes-optimal) discriminability (classification accuracy) between two univariate or multivariate normals, but may also be used when the distributions are close to normal. $e_{b}$ and $d_{b}$ do not have closed-form expressions, but are numerically computed (Matlab code).

In particular, for a yes/no task between two univariate normal distributions with means $\mu _{a},\mu _{b}$ and variances $v_{a}>v_{b}$, the Bayes-optimal classification accuracies are:

$p(a|a)=p({\chi '}_{1,v_{a}\lambda }^{2}>v_{b}c),\;\;p(b|b)=p({\chi '}_{1,v_{b}\lambda }^{2}<v_{a}c)$,

where $\chi '^{2}$ denotes the non-central chi-squared distribution, $\lambda =\left({\frac {\mu _{a}-\mu _{b}}{v_{a}-v_{b}}}\right)^{2}$, and $c=\lambda +{\frac {\ln v_{a}-\ln v_{b}}{v_{a}-v_{b}}}$. The Bayes discriminability $d_{b}=2Z\left({\frac {p\left(a|a\right)+p\left(b|b\right)}{2}}\right).$

For a two-interval task between these distributions, the optimal accuracy is $a_{b}=p\left({\tilde {\chi }}_{{\boldsymbol {w}},{\boldsymbol {k}},{\boldsymbol {\lambda }},0,0}^{2}>0\right)$ (${\tilde {\chi }}^{2}$ denotes the generalized chi-squared distribution), where ${\boldsymbol {w}}={\begin{bmatrix}\sigma _{s}^{2}&-\sigma _{n}^{2}\end{bmatrix}},\;{\boldsymbol {k}}={\begin{bmatrix}1&1\end{bmatrix}},\;{\boldsymbol {\lambda }}={\frac {\mu _{s}-\mu _{n}}{\sigma _{s}^{2}-\sigma _{n}^{2}}}{\begin{bmatrix}\sigma _{s}^{2}&\sigma _{n}^{2}\end{bmatrix}}$. The Bayes discriminability $d_{b}=2Z\left(a_{b}\right)$.

A sub-optimal discriminability index (since it uses a single criterion) is ${\sqrt {2}}$ times the $z$-score of the area under the receiver operating characteristic curve, or AUC . A common closed-form approximation related to this is to take the average of the variances, i.e. the rms of the two standard deviations: $d_{a}=\left\vert \mu _{a}-\mu _{b}\right\vert /\sigma _{\text{rms}}$, extended to general dimensions as the Mahalanobis distance using the pooled covariance, i.e. with $\mathbf {S} _{\text{rms}}=\left[\left(\mathbf {\Sigma } _{a}+\mathbf {\Sigma } _{b}\right)/2\right]^{\frac {1}{2}}$ as the common sd matrix. Another index is $d'_{e}=\left\vert \mu _{a}-\mu _{b}\right\vert /\sigma _{\text{avg}}$, extended to general dimensions using $\mathbf {S} _{\text{avg}}=\left(\mathbf {S} _{a}+\mathbf {S} _{b}\right)/2$ as the common sd matrix. Another common index is $d_{z}=Z({\text{hit rate}})-Z({\text{false alarm rate}})$ .

Comparison of the indices

It has been shown that:

$d_{a}\leq d'_{e}\leq d_{o}\leq d_{z}.$

Thus, $d_{a}$ and $d'_{e}$ underestimate the optimal discriminability of normal distributions, whereas $d_{z}$ overestimates it. Simpson and Fitter promoted $d_{a}$ as the best index, particularly for two-interval tasks, but Das and Geisler have shown that $d_{o}$ is the optimal discriminability in all cases, and $d'_{e}$ is a better approximation than $d_{a}$, even for two-interval tasks.

The approximate index $d_{gm}$, which uses the geometric mean of the sd's, is less than $d_{o}$ at small discriminability, but greater at large discriminability.

See also

• Receiver operating characteristic (ROC)
• Summary statistics
• Effect size

References

• Wickens, Thomas D. (2001). Elementary Signal Detection Theory. OUP USA. ch. 2, p. 20. ISBN 0-19-509250-3.

External links

• Interactive signal detection theory tutorial including calculation of d′.