|
Size: 3257
Comment: Relink
|
← Revision 15 as of 2026-08-12 14:08:22 ⇥
Size: 2721
Comment: Rewrite
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 13: | Line 13: |
| Mahalanobis distance is equivalent to [[Calculus/Distance#Euclidean_distance|Euclidean distance]] with a change in [[LinearAlgebra/Basis|basis]]. | Recall that [[Calculus/Distance#Euclidean_distance|Euclidean distance]] is conventionally calculated as ''|a-b|''. In the notation of [[LinearAlgebra|linear algebra]], '''squared''' Euclidean distance is calculated as ''(a-b)^T^(a-b)''. (Or in the case that the origin is the implicit comparison point, ''|a|'' and ''a^t^a'' are the respective formulations.) |
| Line 15: | Line 15: |
| ''Squared'' Euclidean distance is commonly formulated as... * given a vector x⃗ and the origin as a reference point, ''x⃗^T^x⃗''. * given two vectors x⃗ and y⃗, ''(x⃗-y⃗)^T^(x⃗-y⃗)''. * Let ''z⃗ = x⃗ - y⃗'', so ''(x⃗-y⃗)^T^(x⃗-y⃗) = z⃗^T^z⃗''. * given a column ''x'' and a column of population means as ''μ'', ''(x-μ)^T^(x-μ)''. |
There is an implicit [[LinearAlgebra/Basis|change of basis]] in the linear algebra formulation: ''a^T^Ia'' where ''I'' is the [[LinearAlgebra/SpecialMatrices#Identity_Matrix|identity matrix]]. |
| Line 21: | Line 17: |
| Never forget to take the square root! | Mahalanobis distance is a Euclidean distance with a meaningful change in basis. Conventionally that basis is notated as the [[LinearAlgebra/Invertibility|inverse]] of some matrix; the motivation is that measurements ''x'' and ''y'' have been linearly transformed by ''A'', and this is accomplished with ''A^-1^''. |
| Line 23: | Line 19: |
| Note that this is equivalent to ''x^T^'''I'''x''. A change of basis can be affected by swapping the [[LinearAlgebra/SpecialMatrices#Identity_Matrix|identity matrix]] with some other '''''A'''^-1^'' (so notated because the motivation is generally that there is some other linear transformation '''''A''''' that pre-exists, and needs to be undone). The ''squared'' Mahalanobis distance is then calculated as... * given a vector x⃗ and the origin as a reference point, ''x⃗^T^'''A'''^-1^x⃗''. * given two vectors x⃗ and y⃗, ''(x⃗-y⃗)^T^'''A'''^-1^(x⃗-y⃗)''. * Let ''z⃗ = x⃗ - y⃗'', so ''(x⃗-y⃗)^T^'''A'''^-1^(x⃗-y⃗) = z⃗^T^'''A'''^-1^z⃗''. * given a column ''x'' and a column of population means as ''μ'', ''(x-μ)^T^'''A'''^-1^(x-μ)''. Again, never forget to take the square root! |
Alternatively: imagine a two-dimensional graph of the unit circle. A change of basis stretches and squeezes the graph, such that the circle becomes an ellipsoid. This transformation must be undone to correctly calculate distance in the space. Note that if the change of basis matrix is [[LinearAlgebra/Diagonalization|diagonal]], the ellipsoid will be axis-aligned. |
| Line 37: | Line 25: |
| Mahalanobis distance is invariant under [[LinearAlgebra/Invertibility|non-singular]] linear transformations. Let ''Y,,1,, = a + '''b'''X,,1,,'' and ''Y,,2,, = a + '''b'''X,,2,,'', and suppose that '''''b''''' is non-singular. Then ''d,,M,,(Y,,1,,,Y,,2,,) = d,,M,,(X,,1,,,X,,2,,)''. === Geometry === In a two-dimensional graph, plotting the points with a Euclidean distance of 1 around the origin results in a unit circle. The change of basis described by '''''A''''' transforms the circle into an ellipsoid. Note that if '''''A''''' is [[LinearAlgebra/Diagonalization|diagonal]], the ellipsoid will be '''axis-aligned''' (i.e., appear to be stretched along the ''x'' or ''y'' axes). |
Mahalanobis distance is invariant under [[LinearAlgebra/Invertibility|non-singular]] linear transformations. Given a (non-singular) constant matrix ''A'', let ''Y,,1,, = AX,,1,,'' and ''Y,,2,, = AX,,2,,''. The Mahalanobis distance between ''Y,,1,,'' and ''Y,,2,,'' is equal to that between ''X,,1,,'' and ''X,,2,,''. |
| Line 53: | Line 33: |
| Mahalanobis distances are appropriate for calculating [[Analysis/Variance|variance]]-normalized distance under a multivariate distribution, as for [[Statistics/TestStatistic|test statistics]]. The change of [[LinearAlgebra/Basis|basis]] is established by the [[Analysis/Covariance|precision matrix]], notated as '''''Σ'''^-1^''. | === Standardization === For certain [[Statistics/TestStatistic|test statistics]], a random vector must be standardized; that is, de-meaned and variance set to one. This corresponds to: {{attachment:standard.svg}} where the [[LinearAlgebra/Basis|change of basis]] is established by the [[Analysis/Covariance|precision matrix]] ''Σ^-1^''. |
| Line 61: | Line 49: |
| Using a diagonal matrix of variance terms ignores correlations between the terms. It is effectively an assumption of [[Statistics/JointProbability#Independence|independence]]. Despite not being true Mahalanobis distance, there are still some utilities to this calculation. | Similar to standardization, a precision matrix is used as the change of basis, but with all off-diagonal terms set to zero. This is effectively an assumption of independence. While not as generalized as a true Mahalanobis distance, there are some applications. |
| Line 63: | Line 51: |
| The [[Stata/Mahapick|mahascore]] documentation calls this metric 'normalized Euclidean distance'. | The [[Stata/Mahapick|mahascore]] documentation calls this '''normalized Euclidean distance'''. |
Mahalanobis Distance
Mahalanobis distance is a Euclidean distance that is transformed through a change of basis to normalize variance.
Contents
Description
Recall that Euclidean distance is conventionally calculated as |a-b|. In the notation of linear algebra, squared Euclidean distance is calculated as (a-b)T(a-b). (Or in the case that the origin is the implicit comparison point, |a| and ata are the respective formulations.)
There is an implicit change of basis in the linear algebra formulation: aTIa where I is the identity matrix.
Mahalanobis distance is a Euclidean distance with a meaningful change in basis. Conventionally that basis is notated as the inverse of some matrix; the motivation is that measurements x and y have been linearly transformed by A, and this is accomplished with A-1.
Alternatively: imagine a two-dimensional graph of the unit circle. A change of basis stretches and squeezes the graph, such that the circle becomes an ellipsoid. This transformation must be undone to correctly calculate distance in the space. Note that if the change of basis matrix is diagonal, the ellipsoid will be axis-aligned.
Properties
Mahalanobis distance is invariant under non-singular linear transformations. Given a (non-singular) constant matrix A, let Y1 = AX1 and Y2 = AX2. The Mahalanobis distance between Y1 and Y2 is equal to that between X1 and X2.
Usage
Standardization
For certain test statistics, a random vector must be standardized; that is, de-meaned and variance set to one. This corresponds to:
where the change of basis is established by the precision matrix Σ-1.
Normalized Euclidean distance
Similar to standardization, a precision matrix is used as the change of basis, but with all off-diagonal terms set to zero. This is effectively an assumption of independence. While not as generalized as a true Mahalanobis distance, there are some applications.
The mahascore documentation calls this normalized Euclidean distance.