Leverage
Leverage is a measure of influence on a prediction.
Contents
Description
Leverage is a measure of how much influence a case has in a regression. Remote combinations of predictors, or unique combinations of predictors, pull the regression line towards themselves. This inherently leads to a smaller residual.
The sum of leverage across all cases is equal to k + 1, the number of parameters (including the intercept) fit. Therefore the average leverage is (k+1)/n.
If a high level of influence for an observation is inappropriate, it can be considered an outlier and may be excluded from estimation. A common rule of thumb is to consider any case with leverage greater than twice the average (i.e., 2(k+1)/n) an outlier.
Hat Matrix
The hat matrix is notated and calculated as H = X(XTX)-1XT. The terms on the diagonal of H are leverage. It is called the 'hat' matrix because it puts a 'hat' on the dependent variable: ลท = Hy.
This is a projection matrices, and therefore is both symmetric and idempotent.