Variance
Variance is a measure of dispersion or spread. It is the second central moment.
Contents
Description
Variance is measure of how much observations vary from the expected value. If the expected value is expressed in 'units', then variance is expressed in 'units squared'. This can pose challenges to interpretation, so a closely related statistic is the standard deviation: the square root of variance.
For a random variable X, the variance is usually notated as Var(X) or Var[X]. In contexts where the distribution in question is understood, σ2 is commonly used.
If a particular distribution function can be assumed, there may be a simple formula for variance. For example:
for a Bernoulli distribution, p(1-p)
for a Binomial distribution, np(1-p)
for a Poisson distribution, λ
for a Uniform distribution, (b-a)^2/12
- and so on...
The variance of a random vector X is a covariance matrix.
Properties
If the variance of a random variable X is known, adding a constant to X does not change the variance. Multiplying X by a constant factor scales the known variance by the square of the factor. Succinctly: Var(aX + b) = a2 Var(X)
The inverse of variance is called precision.
Estimation
In most circumstances, sample variance (notated s2) is an unbiased estimator for the true variance σ2. The exceptions are distributions with an undefined expected value or infinite variance, like the Cauchy distribution. Furthermore there are circumstances where the estimate should not be considered good, as with small sample sizes.
When working with a set of observations X, where for each observation i there is an observed value xi, sample variance is calculated as:
Formulation
Formally, the second central moment (implicitly taking the expected value as the center) of a continuous quantitative (numeric) random variable is:
where Ω is the sample space and f is the probability density function for that distribution.
And for a discrete quantitative random variable:
where Ω is the set of possible values and f is the probability mass function for that distribution (although the distributions of discrete random variables are commonly given as tables, rather than mathematical functions).
Variance is therefore an expected value:
By completing the square and spreading the expected value function, a simpler formula is derived. Note that expectations are linear (i.e., E[a + b] = E[a] + E[b]) and that the expected value of an expected value is the expected value itself (i.e., E[E[a]] = E[a]).
