Variance

Variance is a measure of dispersion or spread. It is the second central moment.

Description

Variance is measure of how much observations vary from the expected value. If the expected value is expressed in 'units', then variance is expressed in 'units squared'. This can pose challenges to interpretation, so a closely related statistic is the standard deviation: the square root of variance.

For a random variable X, the variance is usually notated as Var(X) or Var[X]. In contexts where the distribution in question is understood, especially error when analyzing a model, more common notations are σ² for population variance or s² for sample variance. (This enables notation like s_i², for the variance of errors for observation i.)

Formulation

Formally, the second central moment (implicitly taking the expected value as the center) of a continuous quantitative (numeric) random variable is:

where Ω is the sample space and f is the probability density function for that distribution.

And for a discrete quantitative random variable:

where Ω is the set of possible values and f is the probability mass function for that distribution (although the distributions of discrete random variables are commonly given as tables, rather than mathematical functions).

But when working with a set of observations X, where for each observation i there is an observed value x_i, the most common formulas for variance are:

To get from the first to the second, the derivation for discrete variables is:

• Σ (x - μ)² P(x=X)

• Σ (x² - 2μx + μ²) P(x=X)

• Σ [x² P(x=X)] - 2μ Σ [x P(x=X)] + μ² Σ [P(x=X)]

• [E[X²]] - 2μ [μ] + μ² [1]

• E[X²] - 2μ² + μ²

• E[X²] - μ²

• E[X²] - (E[X])²

The derivation for continuous variables is:

• ∫ (x - μ)² f(x) dx

• ∫ (x² - 2μx + μ²) f(x) dx

• ∫ [x² f(x) dx] - 2μ ∫ [x f(x) dx] + μ² ∫ [f(x) dx]

• [E[X²]] - 2μ [μ] + μ² [1]

• E[X²] - 2μ² + μ²

• E[X²] - μ²

• E[X²] - (E[X])²

(Recall that definitionally, 1 = ∫ f(x) dx for any probability density function.)

It should also be clear that you can pick out the formulation as Var(X) = Σ(X²)/n - (E[X])², which can be useful in certain situations.

Variance as a Function

If the variance of a random variable X is known, adding a constant to X does not change the variance. Multiplying X by a constant factor scales the known variance by the square of the factor. Succinctly: Var(aX + b) = a² Var(X)

The variance of a ratio, i.e. Var(X/Y) where both X and Y are variable, is nonlinear.

CategoryRicottone