|
Size: 3164
Comment: Relink
|
← Revision 11 as of 2026-08-11 21:30:30 ⇥
Size: 0
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| = Covariance = '''Covariance''' is a measure of how much something varies with another. It is a generalization of [[Analysis/Variance|variance]]: ''Var(X) = Cov(X,X)''. <<TableOfContents>> ---- == Description == Covariance is calculated as: ''Cov(X,Y) = E[(X - E[X])(Y - E[y])]'' Covariance is related to '''correlation''' as: ''Corr(X,Y) = Cov(X,Y)/σ,,X,,σ,,Y,,'' Letting ''X̅'' be the mean of ''X'', and letting ''Y̅'' be the mean of ''Y'', the calculation becomes: ''Cov(X,Y) = E[(X - X̅)(Y - Y̅)]'' ''E[XY - X̅Y - XY̅ + X̅Y̅]'' ''E[XY] - X̅E[Y] - E[X]Y̅ + X̅Y̅'' ''E[XY] - X̅Y̅ - X̅Y̅ + X̅Y̅'' ''E[XY] - X̅Y̅'' This gives a trivial proof that [[Statistics/JointProbability#Independence|independent]] variables have zero correlation and zero covariance. Necessarily ''E[XY] = E[X]E[Y]'', so ''E[XY] - X̅Y̅ = 0'' === Properties === Covariance is symmetric: ''Cov(X,Y) = Cov(Y,X)'' ---- == Transformations == Covariance linearly transforms with scalars. ''Cov(aX,Y) = E[aXY] - E[aX]E[Y]'' ''a E[XY] - a E[X]E[Y]'' ''a (E[XY] - E[X]E[Y])'' ''a Cov(X,Y)'' Covariance is linear with inputs. ''Cov(X+Y,Z) = E[(X+Y)Z] - E[X+Y]E[Z]'' ''E[XZ+YZ] - E[X+Y]E[Z]'' ''(E[XZ] + E[YZ]) - (E[X] + E[Y]) E[Z]'' ''(E[XZ] + E[YZ]) - (E[X]E[Z] + E[Y]E[Z])'' ''(E[XZ] - E[X]E[Z] + E[YZ] - E[Y]E[Z]'' ''Cov(X,Z) + Cov(Y,Z)'' This gives a trivial proof that constant additions cancel out. ''Cov(a+X,Y) = Cov(X,Y) + Cov(a,Y) = Cov(X,Y) + 0'' '''Altogether''': ''Cov(a+bX,c+dY) = b d Cov(X,Y)'' ---- == Matrix == A '''covariance matrix''' describes multivariate covariances. Consider a column ''x'': the covariance matrix reflects ''Cov(x,x)''. Cell ''(i,j)'' is the covariance of the ''i''th termwith the ''j''th term. On the diagonal are [[Statistics/Variance|variances]] (i.e., covariance of a term with itself). The matrix is usually notated as '''''Σ'''''. The inverse covariance matrix, '''''Σ'''^-1^'', is also called the '''precision matrix'''. The covariance matrix is calculated as: '''''Σ''' = E[(x - E[x])(x - E[x])^T^]'' Letting ''x̅'' be the mean vector of ''x'', the calculation becomes: '''''Σ''' = E[(X - x̅)(X - x̅)^T^]'' Alternatively: {{attachment:summation.svg}} === Properties === A covariance matrix is necessarily square, [[LinearAlgebra/SpecialMatrices#Symmetric_Matrices|symmetric]], and [[LinearAlgebra/PositiveDefiniteness|positive semi-definite]]. * '''''Σ''' = '''Σ'''^T^'' * the [[LinearAlgebra/Determinant|determinant]] is bound by ''|'''Σ'''| >= 0'' * '''''Σ'''^0.5^'' can always be evaluated === Linear Algebra === The covariance matrix linearly transforms with the inputs. ''Cov('''A'''x,'''A'''x) = E[('''A'''X - '''A'''x̅)('''A'''X - '''A'''x̅)^T^]'' ''E['''A'''(X - x̅)(X - x̅)^T^'''A'''^T^]'' '''''A'''E[(X - x̅)(X - x̅)^T^]'''A'''^T^'' '''''AΣA'''^T^'' Trivially, if the transformation is a scalar like ''a'''I''''': ''a'''IΣ'''a'''I'''^T^'' ''a'''Σ'''a'' ''a^2^'''Σ''''' ---- CategoryRicottone |
