|
Size: 3133
Comment: Moving vector pages
|
← Revision 15 as of 2026-08-12 14:08:22 ⇥
Size: 2721
Comment: Rewrite
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 3: | Line 3: |
| '''Mahalanobis distance''' is a [[Calculus/Distance#Euclidean_distance|Euclidean distance]] that is transformed through a [[LinearAlgebra/Basis#Change_of_Basis|change of basis]] to normalize variance. | '''Mahalanobis distance''' is a [[Calculus/Distance#Euclidean_distance|Euclidean distance]] that is transformed through a [[LinearAlgebra/Basis#Change_of_Basis|change of basis]] to normalize [[Analysis/Variance|variance]]. |
| Line 13: | Line 13: |
| Mahalanobis distance is equivalent to [[Calculus/Distance#Euclidean_distance|Euclidean distance]] with a change in [[LinearAlgebra/Basis|basis]]. | Recall that [[Calculus/Distance#Euclidean_distance|Euclidean distance]] is conventionally calculated as ''|a-b|''. In the notation of [[LinearAlgebra|linear algebra]], '''squared''' Euclidean distance is calculated as ''(a-b)^T^(a-b)''. (Or in the case that the origin is the implicit comparison point, ''|a|'' and ''a^t^a'' are the respective formulations.) |
| Line 15: | Line 15: |
| Euclidean distance is commonly formulated as ''(x-y)^T^(x-y)'' (or if the reference point is the origin, just ''x^T^x''), but an equivalent formulation looks like ''x^T^'''I'''^T^'''I'''x''. | There is an implicit [[LinearAlgebra/Basis|change of basis]] in the linear algebra formulation: ''a^T^Ia'' where ''I'' is the [[LinearAlgebra/SpecialMatrices#Identity_Matrix|identity matrix]]. |
| Line 17: | Line 17: |
| A change of basis can be effected by swapping the [[LinearAlgebra/SpecialMatrices#Identity_Matrix|identity matrix]] with some other '''''A''''': ''x^T^'''A'''^T^'''A'''x''. | Mahalanobis distance is a Euclidean distance with a meaningful change in basis. Conventionally that basis is notated as the [[LinearAlgebra/Invertibility|inverse]] of some matrix; the motivation is that measurements ''x'' and ''y'' have been linearly transformed by ''A'', and this is accomplished with ''A^-1^''. Alternatively: imagine a two-dimensional graph of the unit circle. A change of basis stretches and squeezes the graph, such that the circle becomes an ellipsoid. This transformation must be undone to correctly calculate distance in the space. Note that if the change of basis matrix is [[LinearAlgebra/Diagonalization|diagonal]], the ellipsoid will be axis-aligned. |
| Line 21: | Line 23: |
| === Graphing === | === Properties === |
| Line 23: | Line 25: |
| In a two-dimensional graph, plotting the points with a Euclidean distance of 1 around the origin results in a unit circle. The change of basis described by '''''A''''' transforms the circle into an ellipsoid. Note that if '''''A''''' is [[LinearAlgebra/SpecialMatrices#Diagonal_Matrices|diagonal]], the ellipsoid will be '''axis-aligned''' (i.e., appear to be stretched along the ''x'' or ''y'' axes). |
Mahalanobis distance is invariant under [[LinearAlgebra/Invertibility|non-singular]] linear transformations. Given a (non-singular) constant matrix ''A'', let ''Y,,1,, = AX,,1,,'' and ''Y,,2,, = AX,,2,,''. The Mahalanobis distance between ''Y,,1,,'' and ''Y,,2,,'' is equal to that between ''X,,1,,'' and ''X,,2,,''. |
| Line 33: | Line 33: |
| Mahalanobis distances are appropriate for calculating variance-normalized distances, as for [[Statistics/TestStatistic|test statistics]]. The change of [[LinearAlgebra/Basis|basis]] is established by the '''covariance matrix''', notated as '''''Σ'''''. More specifically, using the '''standard deviation matrix''' (''√'''Σ''' = '''Σ'''^0.5^''). | |
| Line 35: | Line 34: |
| The variance-normalized distance from a distribution to an estimate in a single dimension can be calculated with, e.g., the Z-statistic: ''(x̂-μ,,X,,)/σ,,X,,''. (Henceforward measurements are normalized: ''x = x̂-μ,,X,,''.) This can be repeated for any number of dimensions. If variance is unit and independent across dimensions, then the joint distance from the multivariate distribution can be calculated (for two dimensions) like: ''√(x^T^x + y^T^y) = √((x-y)^T^(x-y))''. But variances are not unit and do correlate, as described by the covariate matrix. The change of basis must 'undo' this distribution, ergo the inverse of the standard deviation matrix (''√('''Σ'''^-1^) = '''Σ'''^-0.5^'') should be used for '''''A'''''. | |
| Line 37: | Line 35: |
| Note that a covariance matrix is... * always square [[LinearAlgebra/MatrixProperties#Symmetry|symmetric]], so '''''Σ'''^T^ = '''Σ''''' * always [[LinearAlgebra/MatrixProperties#Positive_Semi-definite|positive semi-definite]], so... * '''''Σ'''^0.5^'' can always be evaluated * the [[LinearAlgebra/Determinants|determinant]] is bound by ''|'''Σ'''| >= 0'', so... * either ''|'''Σ'''| = 0'' or '''''Σ''''' is [[LinearAlgebra/MatrixProperties#Invertible|invertible]] |
=== Standardization === |
| Line 44: | Line 37: |
| After substitution, using the symmetric rule, and simplifying exponents through the product rule, '''''A'''^T^'''A''''' becomes '''''Σ'''^-1^''. In summary, the variance-normalized distance is calculated like: ''√((x-y)^T^'''Σ'''^-1^(x-y))'' | For certain [[Statistics/TestStatistic|test statistics]], a random vector must be standardized; that is, de-meaned and variance set to one. This corresponds to: {{attachment:standard.svg}} where the [[LinearAlgebra/Basis|change of basis]] is established by the [[Analysis/Covariance|precision matrix]] ''Σ^-1^''. === Normalized Euclidean distance === Similar to standardization, a precision matrix is used as the change of basis, but with all off-diagonal terms set to zero. This is effectively an assumption of independence. While not as generalized as a true Mahalanobis distance, there are some applications. The [[Stata/Mahapick|mahascore]] documentation calls this '''normalized Euclidean distance'''. |
Mahalanobis Distance
Mahalanobis distance is a Euclidean distance that is transformed through a change of basis to normalize variance.
Contents
Description
Recall that Euclidean distance is conventionally calculated as |a-b|. In the notation of linear algebra, squared Euclidean distance is calculated as (a-b)T(a-b). (Or in the case that the origin is the implicit comparison point, |a| and ata are the respective formulations.)
There is an implicit change of basis in the linear algebra formulation: aTIa where I is the identity matrix.
Mahalanobis distance is a Euclidean distance with a meaningful change in basis. Conventionally that basis is notated as the inverse of some matrix; the motivation is that measurements x and y have been linearly transformed by A, and this is accomplished with A-1.
Alternatively: imagine a two-dimensional graph of the unit circle. A change of basis stretches and squeezes the graph, such that the circle becomes an ellipsoid. This transformation must be undone to correctly calculate distance in the space. Note that if the change of basis matrix is diagonal, the ellipsoid will be axis-aligned.
Properties
Mahalanobis distance is invariant under non-singular linear transformations. Given a (non-singular) constant matrix A, let Y1 = AX1 and Y2 = AX2. The Mahalanobis distance between Y1 and Y2 is equal to that between X1 and X2.
Usage
Standardization
For certain test statistics, a random vector must be standardized; that is, de-meaned and variance set to one. This corresponds to:
where the change of basis is established by the precision matrix Σ-1.
Normalized Euclidean distance
Similar to standardization, a precision matrix is used as the change of basis, but with all off-diagonal terms set to zero. This is effectively an assumption of independence. While not as generalized as a true Mahalanobis distance, there are some applications.
The mahascore documentation calls this normalized Euclidean distance.