|
Size: 2232
Comment: Initial commit
|
Size: 3255
Comment: Link
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 3: | Line 3: |
| '''Mahalanobis distance''' is a [[LinearAlgebra/Distance|Euclidean distance]] that is transformed through a [[LinearAlgebra/Basis|change of basis]] to normalize variance. | '''Mahalanobis distance''' is a [[Calculus/Distance#Euclidean_distance|Euclidean distance]] that is transformed through a [[LinearAlgebra/Basis#Change_of_Basis|change of basis]] to normalize [[Statistics/Variance|variance]]. |
| Line 11: | Line 11: |
| == Definition == | == Description == |
| Line 13: | Line 13: |
| [[LinearAlgebra/Distance|Euclidean distance]] is typically described as ''x^T^x'', but an equivalent formulation is ''x^T^'''I'''^T^'''I'''x''. In a two-dimensional graph, plotting the points with a Euclidean distance of 1 around the origin results in a unit circle. | Mahalanobis distance is equivalent to [[Calculus/Distance#Euclidean_distance|Euclidean distance]] with a change in [[LinearAlgebra/Basis|basis]]. |
| Line 15: | Line 15: |
| The distance can be transformed to a different [[LinearAlgebra/Basis|basis]] by swapping the [[LinearAlgebra/SpecialMatrices#Identity_Matrix|identity matrix]] with some other '''A''': ''x^T^'''A'''^T^'''A'''x''. The two-dimensional graph will now appear as an ellipsoid. This ellipsoid is '''axis-aligned''' (i.e. appears to be stretched along the ''x'' or ''y'' axes) if the '''A''' is diagonal. | ''Squared'' Euclidean distance is commonly formulated as... * given a vector x⃗ and the origin as a reference point, ''x⃗^T^x⃗''. * given two vectors x⃗ and y⃗, ''(x⃗-y⃗)^T^(x⃗-y⃗)''. * Let ''z⃗ = x⃗ - y⃗'', so ''(x⃗-y⃗)^T^(x⃗-y⃗) = z⃗^T^z⃗''. * given a column ''x'' and a column of population means as ''μ'', ''(x-μ)^T^(x-μ)''. |
| Line 17: | Line 21: |
| Of course distance can be calculated from any arbitrary point, not just the origin. Subtract the difference between the origin and the true reference point, leading to ''(x-m)^T^(x-m)'' or ''(x-m)^T^'''A'''^T^'''A'''(x-m)''. | Never forget to take the square root! Note that this is equivalent to ''x^T^'''I'''x''. A change of basis can be affected by swapping the [[LinearAlgebra/SpecialMatrices#Identity_Matrix|identity matrix]] with some other '''''A'''^-1^'' (so notated because the motivation is generally that there is some other linear transformation '''''A''''' that pre-exists, and needs to be undone). The ''squared'' Mahalanobis distance is then calculated as... * given a vector x⃗ and the origin as a reference point, ''x⃗^T^'''A'''^-1^x⃗''. * given two vectors x⃗ and y⃗, ''(x⃗-y⃗)^T^'''A'''^-1^(x⃗-y⃗)''. * Let ''z⃗ = x⃗ - y⃗'', so ''(x⃗-y⃗)^T^'''A'''^-1^(x⃗-y⃗) = z⃗^T^'''A'''^-1^z⃗''. * given a column ''x'' and a column of population means as ''μ'', ''(x-μ)^T^'''A'''^-1^(x-μ)''. Again, never forget to take the square root! === Properties === Mahalanobis distance is invariant under [[LinearAlgebra/Invertibility|non-singular]] linear transformations. Let ''Y,,1,, = a + '''b'''X,,1,,'' and ''Y,,2,, = a + '''b'''X,,2,,'', and suppose that '''''b''''' is non-singular. Then ''d,,M,,(Y,,1,,,Y,,2,,) = d,,M,,(X,,1,,,X,,2,,)''. === Geometry === In a two-dimensional graph, plotting the points with a Euclidean distance of 1 around the origin results in a unit circle. The change of basis described by '''''A''''' transforms the circle into an ellipsoid. Note that if '''''A''''' is [[LinearAlgebra/Diagonalization|diagonal]], the ellipsoid will be '''axis-aligned''' (i.e., appear to be stretched along the ''x'' or ''y'' axes). |
| Line 23: | Line 51: |
| == Application == | == Usage == |
| Line 25: | Line 53: |
| For computing the variance-normalized distance between two testable measurements, instead of using a simple [[LinearAlgebra/Distance|Euclidean distance]] (i.e. ''x^T^x''), use a Mahalanobis distance with the respective means and the covariance matrix (usually notated as '''''Σ'''''). | Mahalanobis distances are appropriate for calculating variance-normalized distance under a multivariate distribution, as for [[Statistics/TestStatistic|test statistics]]. The change of [[LinearAlgebra/Basis|basis]] is established by the [[Statistics/Covariance#Matrix|inverse covariance matrix]], notated as '''''Σ'''^-1^''. |
| Line 27: | Line 55: |
| The measurement must be normalized to the mean: ''(x-μ)''. | |
| Line 29: | Line 56: |
| Given the normalized measurement, the covariance matrix describes how the unit variance was transformed into some other variances. Therefore the inverse of the covariance matrix ('''''Σ'''^-1^'') describes the inverse transformation. Specifically, '''''A''''' is substituted with '''''Σ'''^-0.5^''. A covariance matrix is always [[LinearAlgebra/MatrixProperties#Positive_Semi-definite|positive semi-definite]] so it can always be [[LinearAlgebra/MatrixInversion|inverted]] and can always have the square root taken. '''''A'''^T^'''A''''' then evaluates to '''''Σ'''^-1^''. | |
| Line 31: | Line 57: |
| The Mahalanobis distance is thus implemented as ''(x-μ)^T^'''Σ'''^-1^(x-μ)''. | === Normalized Euclidean distance === Using a diagonal matrix of variance terms ignores correlations between the terms. It is effectively an assumption of [[Statistics/JointProbability#Independence|independence]]. Despite not being true Mahalanobis distance, there are still some utilities to this calculation. The [[Stata/Mahapick|mahascore]] documentation calls this metric 'normalized Euclidean distance'. |
Mahalanobis Distance
Mahalanobis distance is a Euclidean distance that is transformed through a change of basis to normalize variance.
Description
Mahalanobis distance is equivalent to Euclidean distance with a change in basis.
Squared Euclidean distance is commonly formulated as...
given a vector x⃗ and the origin as a reference point, x⃗Tx⃗.
given two vectors x⃗ and y⃗, (x⃗-y⃗)T(x⃗-y⃗).
Let z⃗ = x⃗ - y⃗, so (x⃗-y⃗)T(x⃗-y⃗) = z⃗Tz⃗.
given a column x and a column of population means as μ, (x-μ)T(x-μ).
Never forget to take the square root!
Note that this is equivalent to xTIx. A change of basis can be affected by swapping the identity matrix with some other A-1 (so notated because the motivation is generally that there is some other linear transformation A that pre-exists, and needs to be undone).
The squared Mahalanobis distance is then calculated as...
given a vector x⃗ and the origin as a reference point, x⃗TA-1x⃗.
given two vectors x⃗ and y⃗, (x⃗-y⃗)TA-1(x⃗-y⃗).
Let z⃗ = x⃗ - y⃗, so (x⃗-y⃗)TA-1(x⃗-y⃗) = z⃗TA-1z⃗.
given a column x and a column of population means as μ, (x-μ)TA-1(x-μ).
Again, never forget to take the square root!
Properties
Mahalanobis distance is invariant under non-singular linear transformations. Let Y1 = a + bX1 and Y2 = a + bX2, and suppose that b is non-singular. Then dM(Y1,Y2) = dM(X1,X2).
Geometry
In a two-dimensional graph, plotting the points with a Euclidean distance of 1 around the origin results in a unit circle. The change of basis described by A transforms the circle into an ellipsoid.
Note that if A is diagonal, the ellipsoid will be axis-aligned (i.e., appear to be stretched along the x or y axes).
Usage
Mahalanobis distances are appropriate for calculating variance-normalized distance under a multivariate distribution, as for test statistics. The change of basis is established by the inverse covariance matrix, notated as Σ-1.
Normalized Euclidean distance
Using a diagonal matrix of variance terms ignores correlations between the terms. It is effectively an assumption of independence. Despite not being true Mahalanobis distance, there are still some utilities to this calculation.
The mahascore documentation calls this metric 'normalized Euclidean distance'.