Mahalanobis Distance
Mahalanobis distance is a Euclidean distance that is transformed through a change of basis to normalize variance.
Contents
Description
Recall that Euclidean distance is conventionally calculated as |a-b|. In the notation of linear algebra, squared Euclidean distance is calculated as (a-b)T(a-b). (Or in the case that the origin is the implicit comparison point, |a| and ata are the respective formulations.)
There is an implicit change of basis in the linear algebra formulation: aTIa where I is the identity matrix.
Mahalanobis distance is a Euclidean distance with a meaningful change in basis. Conventionally that basis is notated as the inverse of some matrix; the motivation is that measurements x and y have been linearly transformed by A, and this is accomplished with A-1.
Alternatively: imagine a two-dimensional graph of the unit circle. A change of basis stretches and squeezes the graph, such that the circle becomes an ellipsoid. This transformation must be undone to correctly calculate distance in the space. Note that if the change of basis matrix is diagonal, the ellipsoid will be axis-aligned.
Properties
Mahalanobis distance is invariant under non-singular linear transformations. Given a (non-singular) constant matrix A, let Y1 = AX1 and Y2 = AX2. The Mahalanobis distance between Y1 and Y2 is equal to that between X1 and X2.
Usage
Standardization
For certain test statistics, a random vector must be standardized; that is, de-meaned and variance set to one. This corresponds to:
where the change of basis is established by the precision matrix Σ-1.
Normalized Euclidean distance
Similar to standardization, a precision matrix is used as the change of basis, but with all off-diagonal terms set to zero. This is effectively an assumption of independence. While not as generalized as a true Mahalanobis distance, there are some applications.
The mahascore documentation calls this normalized Euclidean distance.