|
Size: 2248
Comment: Link
|
← Revision 15 as of 2026-08-12 14:08:22 ⇥
Size: 2721
Comment: Rewrite
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 3: | Line 3: |
| '''Mahalanobis distance''' is a [[LinearAlgebra/Distance|Euclidean distance]] that is transformed through a [[LinearAlgebra/Basis#Change_of_Basis|change of basis]] to normalize variance. | '''Mahalanobis distance''' is a [[Calculus/Distance#Euclidean_distance|Euclidean distance]] that is transformed through a [[LinearAlgebra/Basis#Change_of_Basis|change of basis]] to normalize [[Analysis/Variance|variance]]. |
| Line 11: | Line 11: |
| == Definition == | == Description == |
| Line 13: | Line 13: |
| [[LinearAlgebra/Distance|Euclidean distance]] is typically described as ''x^T^x'', but an equivalent formulation is ''x^T^'''I'''^T^'''I'''x''. In a two-dimensional graph, plotting the points with a Euclidean distance of 1 around the origin results in a unit circle. | Recall that [[Calculus/Distance#Euclidean_distance|Euclidean distance]] is conventionally calculated as ''|a-b|''. In the notation of [[LinearAlgebra|linear algebra]], '''squared''' Euclidean distance is calculated as ''(a-b)^T^(a-b)''. (Or in the case that the origin is the implicit comparison point, ''|a|'' and ''a^t^a'' are the respective formulations.) |
| Line 15: | Line 15: |
| The distance can be transformed to a different [[LinearAlgebra/Basis|basis]] by swapping the [[LinearAlgebra/SpecialMatrices#Identity_Matrix|identity matrix]] with some other '''A''': ''x^T^'''A'''^T^'''A'''x''. The two-dimensional graph will now appear as an ellipsoid. This ellipsoid is '''axis-aligned''' (i.e. appears to be stretched along the ''x'' or ''y'' axes) if the '''A''' is diagonal. | There is an implicit [[LinearAlgebra/Basis|change of basis]] in the linear algebra formulation: ''a^T^Ia'' where ''I'' is the [[LinearAlgebra/SpecialMatrices#Identity_Matrix|identity matrix]]. |
| Line 17: | Line 17: |
| Of course distance can be calculated from any arbitrary point, not just the origin. Subtract the difference between the origin and the true reference point, leading to ''(x-m)^T^(x-m)'' or ''(x-m)^T^'''A'''^T^'''A'''(x-m)''. | Mahalanobis distance is a Euclidean distance with a meaningful change in basis. Conventionally that basis is notated as the [[LinearAlgebra/Invertibility|inverse]] of some matrix; the motivation is that measurements ''x'' and ''y'' have been linearly transformed by ''A'', and this is accomplished with ''A^-1^''. Alternatively: imagine a two-dimensional graph of the unit circle. A change of basis stretches and squeezes the graph, such that the circle becomes an ellipsoid. This transformation must be undone to correctly calculate distance in the space. Note that if the change of basis matrix is [[LinearAlgebra/Diagonalization|diagonal]], the ellipsoid will be axis-aligned. === Properties === Mahalanobis distance is invariant under [[LinearAlgebra/Invertibility|non-singular]] linear transformations. Given a (non-singular) constant matrix ''A'', let ''Y,,1,, = AX,,1,,'' and ''Y,,2,, = AX,,2,,''. The Mahalanobis distance between ''Y,,1,,'' and ''Y,,2,,'' is equal to that between ''X,,1,,'' and ''X,,2,,''. |
| Line 23: | Line 31: |
| == Application == | == Usage == |
| Line 25: | Line 33: |
| For computing the variance-normalized distance between two testable measurements, instead of using a simple [[LinearAlgebra/Distance|Euclidean distance]] (i.e. ''x^T^x''), use a Mahalanobis distance with the respective means and the covariance matrix (usually notated as '''''Σ'''''). | |
| Line 27: | Line 34: |
| The measurement must be normalized to the mean: ''(x-μ)''. | |
| Line 29: | Line 35: |
| Given the normalized measurement, the covariance matrix describes how the unit variance was transformed into some other variances. Therefore the inverse of the covariance matrix ('''''Σ'''^-1^'') describes the inverse transformation. Specifically, '''''A''''' is substituted with '''''Σ'''^-0.5^''. A covariance matrix is always [[LinearAlgebra/MatrixProperties#Positive_Semi-definite|positive semi-definite]] so it can always be [[LinearAlgebra/MatrixInversion|inverted]] and can always have the square root taken. '''''A'''^T^'''A''''' then evaluates to '''''Σ'''^-1^''. | === Standardization === |
| Line 31: | Line 37: |
| The Mahalanobis distance is thus implemented as ''(x-μ)^T^'''Σ'''^-1^(x-μ)''. | For certain [[Statistics/TestStatistic|test statistics]], a random vector must be standardized; that is, de-meaned and variance set to one. This corresponds to: {{attachment:standard.svg}} where the [[LinearAlgebra/Basis|change of basis]] is established by the [[Analysis/Covariance|precision matrix]] ''Σ^-1^''. === Normalized Euclidean distance === Similar to standardization, a precision matrix is used as the change of basis, but with all off-diagonal terms set to zero. This is effectively an assumption of independence. While not as generalized as a true Mahalanobis distance, there are some applications. The [[Stata/Mahapick|mahascore]] documentation calls this '''normalized Euclidean distance'''. |
Mahalanobis Distance
Mahalanobis distance is a Euclidean distance that is transformed through a change of basis to normalize variance.
Contents
Description
Recall that Euclidean distance is conventionally calculated as |a-b|. In the notation of linear algebra, squared Euclidean distance is calculated as (a-b)T(a-b). (Or in the case that the origin is the implicit comparison point, |a| and ata are the respective formulations.)
There is an implicit change of basis in the linear algebra formulation: aTIa where I is the identity matrix.
Mahalanobis distance is a Euclidean distance with a meaningful change in basis. Conventionally that basis is notated as the inverse of some matrix; the motivation is that measurements x and y have been linearly transformed by A, and this is accomplished with A-1.
Alternatively: imagine a two-dimensional graph of the unit circle. A change of basis stretches and squeezes the graph, such that the circle becomes an ellipsoid. This transformation must be undone to correctly calculate distance in the space. Note that if the change of basis matrix is diagonal, the ellipsoid will be axis-aligned.
Properties
Mahalanobis distance is invariant under non-singular linear transformations. Given a (non-singular) constant matrix A, let Y1 = AX1 and Y2 = AX2. The Mahalanobis distance between Y1 and Y2 is equal to that between X1 and X2.
Usage
Standardization
For certain test statistics, a random vector must be standardized; that is, de-meaned and variance set to one. This corresponds to:
where the change of basis is established by the precision matrix Σ-1.
Normalized Euclidean distance
Similar to standardization, a precision matrix is used as the change of basis, but with all off-diagonal terms set to zero. This is effectively an assumption of independence. While not as generalized as a true Mahalanobis distance, there are some applications.
The mahascore documentation calls this normalized Euclidean distance.