|
Size: 732
Comment: Simplifying matrix page names
|
Size: 2740
Comment: Rewrite
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| = Covariance Matrices = | = Covariance = |
| Line 3: | Line 3: |
| '''Covariance matrices''' are specially constricted matrices that are useful for various procedures. | '''Covariance''' is a measure of how much something varies with another. It is a generalization of '''variance''': ''Var(X) = Cov(X,X)''. |
| Line 11: | Line 11: |
| Line 13: | Line 14: |
| The matrix is usually notated as '''''Σ'''''. | Covariance is calculated as: |
| Line 15: | Line 16: |
| Each cell identified by ''(i,j)'' carries a value of the covariance between term ''i'' and term ''j''. The diagonal is therefore each term's variance. If the terms are independently distributed, their covariances are 0, and the matrix is fully specified as the [[LinearAlgebra/Diagonalization|diagonal matrix]] of variances. | ''Cov(X,Y) = E[(X - E[X])(Y - E[y])]'' Covariance is related to '''correlation''' as: ''Corr(X,Y) = Cov(X,Y)/σ,,X,,σ,,Y,,'' Letting ''X̅'' be the mean of ''X'', and letting ''Y̅'' be the mean of ''Y'', the calculation becomes: ''Cov(X,Y) = E[(X - X̅)(Y - Y̅)]'' ''E[XY - X̅Y - XY̅ + X̅Y̅]'' ''E[XY] - X̅E[Y] - E[X]Y̅ + X̅Y̅'' ''E[XY] - X̅Y̅ - X̅Y̅ + X̅Y̅'' ''E[XY] - X̅Y̅'' This gives a trivial proof that [[Statistics/JointProbability#Independence|independent]] variables have zero correlation and zero covariance. Necessarily ''E[XY] = E[X]E[Y]'', so ''E[XY] - X̅Y̅ = 0'' In the context of [[LinearAlgebra|linear algebra]], the calculation is notated as: ''Cov(X,Y) = E[(X - E[X])(Y - E[y])^T^]'' Letting ''m,,X,,'' be the mean vector of ''X'' and ''m,,Y,,'' be the mean vector of ''Y'', the calculation becomes: ''Cov(X,Y) = E[(X - m,,X,,)(Y - m,,Y,,)^T^]'' |
| Line 19: | Line 46: |
| === Precision Matrices === | === Properties === |
| Line 21: | Line 48: |
| The inverse of a covariance matrix, notated as '''''Σ'''^-1^'', is called a '''precision matrix'''. | Covariance is symmetric: ''Cov(X,Y) = Cov(Y,X)'' ---- == Transformations == Covariance linearly transforms with scalars. ''Cov(aX,Y) = E[aXY] - E[aX]E[Y]'' ''a E[XY] - a E[X]E[Y]'' ''a (E[XY] - E[X]E[Y])'' ''a Cov(X,Y)'' Covariance is linear with inputs. ''Cov(X+Y,Z) = E[(X+Y)Z] - E[X+Y]E[Z]'' ''E[XZ+YZ] - E[X+Y]E[Z]'' ''(E[XZ] + E[YZ]) - (E[X] + E[Y]) E[Z]'' ''(E[XZ] + E[YZ]) - (E[X]E[Z] + E[Y]E[Z])'' ''(E[XZ] - E[X]E[Z] + E[YZ] - E[Y]E[Z]'' ''Cov(X,Z) + Cov(Y,Z)'' This gives a trivial proof that constant additions cancel out. ''Cov(a+X,Y) = Cov(X,Y) + Cov(a,Y) = Cov(X,Y) + 0'' '''Altogether''': ''Cov(a+bX,c+dY) = b d Cov(X,Y)'' ---- == Matrix == A '''covariance matrix''' describes multivariate covariances. Cell ''(i,j)'' is the covariance of the ''i''th variable with the ''j''th variable. On the diagonal are variances (i.e., covariance of a variable with itself). The matrix is usually notated as '''''Σ'''''. The inverse covariance matrix, '''''Σ'''^-1^'', is also called the '''precision matrix'''. The covariance matrix linearly transforms with the inputs. ''Cov('''A'''X,'''A'''Y) = E[('''A'''X - '''A'''m,,X,,)('''A'''Y - '''A'''m,,Y,,)^T^]'' ''E['''A'''(X - m,,X,,)(Y - m,,Y,,)^T^'''A'''^T^]'' '''''A'''E[(X - m,,X,,)(Y - m,,Y,,)^T^]'''A'''^T^'' '''''AΣA'''^T^'' Trivially, if the transformation is a scalar like ''a'''I''''': ''a'''IΣ'''a'''I'''^T^'' ''a'''Σ'''a'' ''a^2^'''Σ''''' |
Covariance
Covariance is a measure of how much something varies with another. It is a generalization of variance: Var(X) = Cov(X,X).
Contents
Description
Covariance is calculated as:
Cov(X,Y) = E[(X - E[X])(Y - E[y])]
Covariance is related to correlation as:
Corr(X,Y) = Cov(X,Y)/σXσY
Letting X̅ be the mean of X, and letting Y̅ be the mean of Y, the calculation becomes:
Cov(X,Y) = E[(X - X̅)(Y - Y̅)]
E[XY - X̅Y - XY̅ + X̅Y̅]
E[XY] - X̅E[Y] - E[X]Y̅ + X̅Y̅
E[XY] - X̅Y̅ - X̅Y̅ + X̅Y̅
E[XY] - X̅Y̅
This gives a trivial proof that independent variables have zero correlation and zero covariance. Necessarily E[XY] = E[X]E[Y], so E[XY] - X̅Y̅ = 0
In the context of linear algebra, the calculation is notated as:
Cov(X,Y) = E[(X - E[X])(Y - E[y])T]
Letting mX be the mean vector of X and mY be the mean vector of Y, the calculation becomes:
Cov(X,Y) = E[(X - mX)(Y - mY)T]
Properties
Covariance is symmetric: Cov(X,Y) = Cov(Y,X)
Transformations
Covariance linearly transforms with scalars.
Cov(aX,Y) = E[aXY] - E[aX]E[Y]
a E[XY] - a E[X]E[Y]
a (E[XY] - E[X]E[Y])
a Cov(X,Y)
Covariance is linear with inputs.
Cov(X+Y,Z) = E[(X+Y)Z] - E[X+Y]E[Z]
E[XZ+YZ] - E[X+Y]E[Z]
(E[XZ] + E[YZ]) - (E[X] + E[Y]) E[Z]
(E[XZ] + E[YZ]) - (E[X]E[Z] + E[Y]E[Z])
(E[XZ] - E[X]E[Z] + E[YZ] - E[Y]E[Z]
Cov(X,Z) + Cov(Y,Z)
This gives a trivial proof that constant additions cancel out.
Cov(a+X,Y) = Cov(X,Y) + Cov(a,Y) = Cov(X,Y) + 0
Altogether: Cov(a+bX,c+dY) = b d Cov(X,Y)
Matrix
A covariance matrix describes multivariate covariances. Cell (i,j) is the covariance of the ith variable with the jth variable. On the diagonal are variances (i.e., covariance of a variable with itself). The matrix is usually notated as Σ.
The inverse covariance matrix, Σ-1, is also called the precision matrix.
The covariance matrix linearly transforms with the inputs.
Cov(AX,AY) = E[(AX - AmX)(AY - AmY)T]
E[A(X - mX)(Y - mY)TAT]
AE[(X - mX)(Y - mY)T]AT
AΣAT
Trivially, if the transformation is a scalar like aI:
aIΣaIT
aΣa
a2Σ