|
Size: 2232
Comment: Initial commit
|
← Revision 3 as of 2025-03-27 19:23:52 ⇥
Size: 3105
Comment: Rewrite for clarity
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 3: | Line 3: |
| '''Mahalanobis distance''' is a [[LinearAlgebra/Distance|Euclidean distance]] that is transformed through a [[LinearAlgebra/Basis|change of basis]] to normalize variance. | '''Mahalanobis distance''' is a [[LinearAlgebra/Distance|Euclidean distance]] that is transformed through a [[LinearAlgebra/Basis#Change_of_Basis|change of basis]] to normalize variance. |
| Line 11: | Line 11: |
| == Definition == | == Description == |
| Line 13: | Line 13: |
| [[LinearAlgebra/Distance|Euclidean distance]] is typically described as ''x^T^x'', but an equivalent formulation is ''x^T^'''I'''^T^'''I'''x''. In a two-dimensional graph, plotting the points with a Euclidean distance of 1 around the origin results in a unit circle. | Mahalanobis distance is equivalent to [[LinearAlgebra/Distance|Euclidean distance]] with a change in [[LinearAlgebra/Basis|basis]]. |
| Line 15: | Line 15: |
| The distance can be transformed to a different [[LinearAlgebra/Basis|basis]] by swapping the [[LinearAlgebra/SpecialMatrices#Identity_Matrix|identity matrix]] with some other '''A''': ''x^T^'''A'''^T^'''A'''x''. The two-dimensional graph will now appear as an ellipsoid. This ellipsoid is '''axis-aligned''' (i.e. appears to be stretched along the ''x'' or ''y'' axes) if the '''A''' is diagonal. | Euclidean distance is commonly formulated as ''(x-y)^T^(x-y)'' (or if the reference point is the origin, just ''x^T^x''), but an equivalent formulation looks like ''x^T^'''I'''^T^'''I'''x''. |
| Line 17: | Line 17: |
| Of course distance can be calculated from any arbitrary point, not just the origin. Subtract the difference between the origin and the true reference point, leading to ''(x-m)^T^(x-m)'' or ''(x-m)^T^'''A'''^T^'''A'''(x-m)''. | A change of basis can be effected by swapping the [[LinearAlgebra/SpecialMatrices#Identity_Matrix|identity matrix]] with some other '''''A''''': ''x^T^'''A'''^T^'''A'''x''. === Graphing === In a two-dimensional graph, plotting the points with a Euclidean distance of 1 around the origin results in a unit circle. The change of basis described by '''''A''''' transforms the circle into an ellipsoid. Note that if '''''A''''' is [[LinearAlgebra/SpecialMatrices#Diagonal_Matrices|diagonal]], the ellipsoid will be '''axis-aligned''' (i.e., appear to be stretched along the ''x'' or ''y'' axes). |
| Line 23: | Line 31: |
| == Application == | == Usage == |
| Line 25: | Line 33: |
| For computing the variance-normalized distance between two testable measurements, instead of using a simple [[LinearAlgebra/Distance|Euclidean distance]] (i.e. ''x^T^x''), use a Mahalanobis distance with the respective means and the covariance matrix (usually notated as '''''Σ'''''). | Mahalanobis distances are appropriate for calculating variance-normalized distances, as for [[Statistics/TestStatistic|test statistics]]. The change of [[LinearAlgebra/Basis|basis]] is established by the '''covariance matrix''', notated as '''''Σ'''''. More specifically, using the '''standard deviation matrix''' (''√'''Σ''' = '''Σ'''^0.5^''). |
| Line 27: | Line 35: |
| The measurement must be normalized to the mean: ''(x-μ)''. | The variance-normalized distance from a distribution to an estimate in a single dimension can be calculated with, e.g., the Z-statistic: ''(x̂-μ,,X,,)/σ,,X,,''. (Henceforward measurements are normalized: ''x = x̂-μ,,X,,''.) This can be repeated for any number of dimensions. If variance is unit and independent across dimensions, then the joint distance from the multivariate distribution can be calculated (for two dimensions) like: ''√(x^T^x + y^T^y) = √((x-y)^T^(x-y))''. But variances are not unit and do correlate, as described by the covariate matrix. The change of basis must 'undo' this distribution, ergo the inverse of the standard deviation matrix (''√('''Σ'''^-1^) = '''Σ'''^-0.5^'') should be used for '''''A'''''. |
| Line 29: | Line 37: |
| Given the normalized measurement, the covariance matrix describes how the unit variance was transformed into some other variances. Therefore the inverse of the covariance matrix ('''''Σ'''^-1^'') describes the inverse transformation. Specifically, '''''A''''' is substituted with '''''Σ'''^-0.5^''. A covariance matrix is always [[LinearAlgebra/MatrixProperties#Positive_Semi-definite|positive semi-definite]] so it can always be [[LinearAlgebra/MatrixInversion|inverted]] and can always have the square root taken. '''''A'''^T^'''A''''' then evaluates to '''''Σ'''^-1^''. | Note that a covariance matrix is... * always square [[LinearAlgebra/MatrixProperties#Symmetry|symmetric]], so '''''Σ'''^T^ = '''Σ''''' * always [[LinearAlgebra/MatrixProperties#Positive_Semi-definite|positive semi-definite]], so... * '''''Σ'''^0.5^'' can always be evaluated * the [[LinearAlgebra/Determinants|determinant]] is bound by ''|'''Σ'''| >= 0'', so... * either ''|'''Σ'''| = 0'' or '''''Σ''''' is [[LinearAlgebra/MatrixProperties#Invertible|invertible]] |
| Line 31: | Line 44: |
| The Mahalanobis distance is thus implemented as ''(x-μ)^T^'''Σ'''^-1^(x-μ)''. | After substitution, using the symmetric rule, and simplifying exponents through the product rule, '''''A'''^T^'''A''''' becomes '''''Σ'''^-1^''. In summary, the variance-normalized distance is calculated like: ''√((x-y)^T^'''Σ'''^-1^(x-y))'' |
Mahalanobis Distance
Mahalanobis distance is a Euclidean distance that is transformed through a change of basis to normalize variance.
Contents
Description
Mahalanobis distance is equivalent to Euclidean distance with a change in basis.
Euclidean distance is commonly formulated as (x-y)T(x-y) (or if the reference point is the origin, just xTx), but an equivalent formulation looks like xTITIx.
A change of basis can be effected by swapping the identity matrix with some other A: xTATAx.
Graphing
In a two-dimensional graph, plotting the points with a Euclidean distance of 1 around the origin results in a unit circle. The change of basis described by A transforms the circle into an ellipsoid.
Note that if A is diagonal, the ellipsoid will be axis-aligned (i.e., appear to be stretched along the x or y axes).
Usage
Mahalanobis distances are appropriate for calculating variance-normalized distances, as for test statistics. The change of basis is established by the covariance matrix, notated as Σ. More specifically, using the standard deviation matrix (√Σ = Σ0.5).
The variance-normalized distance from a distribution to an estimate in a single dimension can be calculated with, e.g., the Z-statistic: (x̂-μX)/σX. (Henceforward measurements are normalized: x = x̂-μX.) This can be repeated for any number of dimensions. If variance is unit and independent across dimensions, then the joint distance from the multivariate distribution can be calculated (for two dimensions) like: √(xTx + yTy) = √((x-y)T(x-y)). But variances are not unit and do correlate, as described by the covariate matrix. The change of basis must 'undo' this distribution, ergo the inverse of the standard deviation matrix (√(Σ-1) = Σ-0.5) should be used for A.
Note that a covariance matrix is...
always square symmetric, so ΣT = Σ
always positive semi-definite, so...
Σ0.5 can always be evaluated
the determinant is bound by |Σ| >= 0, so...
either |Σ| = 0 or Σ is invertible
After substitution, using the symmetric rule, and simplifying exponents through the product rule, ATA becomes Σ-1. In summary, the variance-normalized distance is calculated like: √((x-y)TΣ-1(x-y))