Collinearity Test

Independence is a critical assumption of regression, but there is no formal test for it. The following diagnostics are recommended as collinearity tests.

Correlation

In the context of a linear system, Pearson's r is a simple but appropriate method to check for collinearity.

Variance Inflation Factor

Every parameter in a regression model can be evaluated by it's variance inflation factor (VIF). This is the ratio of a parameter's variance under a full model to its variance under a model including only that parameter. This can be interpreted as how much a variance is inflated by collinearity, or as how much a variance is inflated compared to if a parameter had 0 collinearity with other parameters.

As a result, VIFs are a measure for multicollinearity in a regression model. Higher values indicate greater collinearity. A threshold of 10 can be a good benchmark.

Generally, VIF is calculated as the inverse of tolerance. (Therefore it is also possible to test multicollinearity with tolerance directly, as opposed to VIF, but this isn't as easily interpreted.)

Generally, tolerance for a parameter is calculated as 1 - R² where R² is the coefficient of determination from regressing all other independent variables onto the variable in question.

Example

for Stata, consider this example copied from here:

. use https://stats.idre.ucla.edu/stat/stata/notes/hsb2, clear

. generate rw = read*write  /* create interaction of read and write */

. svyset [pw=math], strata(ses)

            pweight: math
          VCE: linearized
  Single unit: missing
     Strata 1: ses
         SU 1: 
        FPC 1: 

. svy: regress rw write read
(running regress on estimation sample)

Survey: Linear regression

Number of strata   =         3                  Number of obs      =       200
Number of PSUs     =       200                  Population size    =     10529
                                                Design df          =       197
                                                F(   2,    196)    =   5258.91
                                                Prob > F           =    0.0000
                                                R-squared          =    0.9916

------------------------------------------------------------------------------
             |             Linearized
          rw |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       write |   49.77855    .948907    52.46   0.000     47.90723    51.64987
        read |    55.3573   .9117403    60.72   0.000     53.55928    57.15533
       _cons |  -2703.949   55.95981   -48.32   0.000    -2814.306   -2593.591
------------------------------------------------------------------------------

. display "tolerance = " 1-e(r2) " VIF = " 1/(1-e(r2))
tolerance = .00843133 VIF = 118.60521

For R, the olsrr package is recommended:

> library(olsrr)
> model <- lm(mpg ~ disp + hp + wt + qsec, data = mtcars)
> ols_vif_tol(model)
  variables Tolerance      VIF
1      disp 0.1252279 7.985439
2        hp 0.1935450 5.166758
3        wt 0.1445726 6.916942
4      qsec 0.3191708 3.133119

For SAS, see PROC REG.

proc reg data=LIBREF.TABLE;
  model DEPVAR = INDEPVARLIST / vif tol;
run;

CategoryRicottone