= Covariance =

'''Covariance''' is a measure of how much something varies with another. It is a generalization of [[Statistics/Variance|variance]]: ''Var(X) = Cov(X,X)''.

<<TableOfContents>>

----




== Description ==

Covariance is calculated as:

''Cov(X,Y) = E[(X - E[X])(Y - E[y])]''

Covariance is related to '''correlation''' as:

''Corr(X,Y) = Cov(X,Y)/σ,,X,,σ,,Y,,''

Letting ''X̅'' be the mean of ''X'', and letting ''Y̅'' be the mean of ''Y'', the calculation becomes:

''Cov(X,Y) = E[(X - X̅)(Y - Y̅)]''

''E[XY - X̅Y - XY̅ + X̅Y̅]''

''E[XY] - X̅E[Y] - E[X]Y̅ + X̅Y̅''

''E[XY] - X̅Y̅ - X̅Y̅ + X̅Y̅''

''E[XY] - X̅Y̅''

This gives a trivial proof that [[Statistics/JointProbability#Independence|independent]] variables have zero correlation and zero covariance. Necessarily ''E[XY] = E[X]E[Y]'', so ''E[XY] - X̅Y̅ = 0''



=== Properties ===

Covariance is symmetric: ''Cov(X,Y) = Cov(Y,X)''

----



== Transformations ==

Covariance linearly transforms with scalars.

''Cov(aX,Y) = E[aXY] - E[aX]E[Y]''

''a E[XY] - a E[X]E[Y]''

''a (E[XY] - E[X]E[Y])''

''a Cov(X,Y)''

Covariance is linear with inputs.

''Cov(X+Y,Z) = E[(X+Y)Z] - E[X+Y]E[Z]''

''E[XZ+YZ] - E[X+Y]E[Z]''

''(E[XZ] + E[YZ]) - (E[X] + E[Y]) E[Z]''

''(E[XZ] + E[YZ]) - (E[X]E[Z] + E[Y]E[Z])''

''(E[XZ] - E[X]E[Z] + E[YZ] - E[Y]E[Z]''

''Cov(X,Z) + Cov(Y,Z)''

This gives a trivial proof that constant additions cancel out.

''Cov(a+X,Y) = Cov(X,Y) + Cov(a,Y) = Cov(X,Y) + 0''

'''Altogether''': ''Cov(a+bX,c+dY) = b d Cov(X,Y)''

----



== Matrix ==

A '''covariance matrix''' describes multivariate covariances. Consider a column ''x'': the covariance matrix reflects ''Cov(x,x)''. Cell ''(i,j)'' is the covariance of the ''i''th termwith the ''j''th term. On the diagonal are [[Statistics/Variance|variances]] (i.e., covariance of a term with itself). The matrix is usually notated as '''''Σ'''''.

The inverse covariance matrix, '''''Σ'''^-1^'', is also called the '''precision matrix'''.

The covariance matrix is calculated as:

'''''Σ''' = E[(x - E[x])(x - E[x])^T^]''

Letting ''x̅'' be the mean vector of ''x'', the calculation becomes:

'''''Σ''' = E[(X - x̅)(X - x̅)^T^]''

Alternatively:

{{attachment:summation.svg}}



=== Properties ===

A covariance matrix is necessarily square, [[LinearAlgebra/SpecialMatrices#Symmetric_Matrices|symmetric]], and [[LinearAlgebra/PositiveDefiniteness|positive semi-definite]].
 * '''''Σ''' = '''Σ'''^T^''
 * the [[LinearAlgebra/Determinant|determinant]] is bound by ''|'''Σ'''| >= 0''
 * '''''Σ'''^0.5^'' can always be evaluated


=== Linear Algebra ===

The covariance matrix linearly transforms with the inputs.

''Cov('''A'''x,'''A'''x) = E[('''A'''X - '''A'''x̅)('''A'''X - '''A'''x̅)^T^]''

''E['''A'''(X - x̅)(X - x̅)^T^'''A'''^T^]''

'''''A'''E[(X - x̅)(X - x̅)^T^]'''A'''^T^''

'''''AΣA'''^T^''

Trivially, if the transformation is a scalar like ''a'''I''''':

''a'''IΣ'''a'''I'''^T^''

''a'''Σ'''a''

''a^2^'''Σ'''''



----
CategoryRicottone