Residuals
Residuals are the difference between a prediction and an observed value.
Contents
Description
A residual is the difference between a predicted value (as by a model) and an actually observed value.
Note that there's substantial mixing of terms with error. Errors are a distinct concept: the difference between a true value and its measurement. Nonetheless, terminology is varied.
Sum of squared errors (SSE) actually means the sum of squared residuals.
Residual sum of squares (RSE) means the same thing.
Do not mix up with SSR (sum of squared regression), which often comes up in the same context. R2 = 1 - SSR/SST where SST is the total variation in the dependent variable.
Regarding mean squared error (MSE)...
In the context of regressions, MSE usually means SSE divided by the number of degrees of freedom (i.e., n - k - 1).
- In other contexts, MSE usually means a theoretical second moment. For example, when considering the properties of an estimator and how it differs from the true random variable.
Root mean squared error (RMSE) is the square root of MSE in any context.