Moments
Moments are measures of a distribution's shape and density.
Contents
Description
The first raw moment is the mean: μ = E[X].
The second central moment is the variance: σ2 = E[(X - E[X])2 = E[(X - μ)2
The third central moment, skewness, measures lopsidedness of a distribution.
The fourth central moment, kurtosis, measures the heaviness of the tails on a distribution.
Errors
Models generally assume that individual errors average to zero, i.e. the first moment of errors is zero: E[Ŷ - Y] = 0. Nonetheless, higher order moments are important.
The mean square error (MSE) is the second moment of the error: MSE(ˆθ) = E[(ˆθ - E[ˆθ])2]. MSE can be decomposed into the variance of the estimator and bias: MSE(ˆθ) = Var(ˆθ) + Bias(ˆθ,θ)2 = Var(ˆθ) + (E[ˆθ]-θ)2.
Two important notes:
Bias, i.e. E[ˆθ] - θ, is not the same as the first moment of errors.
If there is no bias, then MSE is the variance of the estimator: MSE(ˆθ) = Var(ˆθ).