= Robust Standard Errors =

'''Robust standard errors''' are corrections to classical standard errors to account for failures of assumptions, mainly constant variance across all observations.

<<TableOfContents>>

----



== Background ==

A standard error is the [[Analysis/Variance|variance]] of a [[Statistics/SamplingDistribution|sample statistic]].



=== Classical Standard Error of an Estimated Coefficient in the Simple Linear Regression ===

In the univariate case, standard errors are classically specified as:

{{attachment:unispec1.svg}}

Supposing the population ''Var(ε)'' is known and errors are homoskedastic, i.e. they are constant across all cases, this can be simplified.

{{attachment:unispec2.svg}}

Lastly, rewrite the denominator in terms of ''Var(X)''.

{{attachment:unispec3.svg}}

''Var(ε)'' is unknown, so this term is estimated as:

{{attachment:uniest2.svg}}

{{attachment:uniest1.svg}}

1 degree of freedom is lost in assuming homoskedasticity of errors, i.e. {{attachment:homosked.svg}}; and ''k'' degrees of freedom are lost in assuming independence of errors and ''k'' independent variables, which is necessarily 1 in the univariate case, i.e.: {{attachment:ind.svg}}

This arrives at estimation as:

{{attachment:uniest3.svg}}



=== In General Linear Model ===

The classical multivariate specification is expressed in terms of ''('''b'''-β)'', as:

{{attachment:multspec1.svg}}

That term is rewritten as ''('''X'''^T^'''X''')^-1^'''Xε'''''.

{{attachment:multspec2.svg}}

{{attachment:multspec3.svg}}

''E['''εε'''^T^|'''X''']'' is not a practical matrix to work with, even if known. But if homoskedasticity and independence are assumed, i.e.: {{attachment:homosked_ind.svg}}, then this simplifies to:

{{attachment:multspec4.svg}}

''s^2^'' is unknown, so this term is estimated as:

{{attachment:multspec5.svg}}

This arrives at estimation as:

{{attachment:multspec6.svg}}

----



== Robust ==

In the presence of heteroskedasticity of errors, the above simplifications cannot apply. In the univariate case, use the original estimator.

This is mostly interesting in the multivariate case, where ''E['''εε'''^T^|'''X''']'' is still not practical. The assumptions made, when incorrect, lead to...
 * OLS estimators are not BLUE
   * they are unbiased, but no longer most efficient in terms of MSE
 * nonlinear GLMs, such as logit, can be biased
 * even if the model's estimates are unbiased, statistics derived from those estimates (e.g., conditioned probability distributions) can be biased

'''Eicker-Huber-White heterskedasticity consistent errors''' ('''HCE''') assume that errors are still independent but allowed to vary, i.e. '''''Σ''' = diag(ε,,1,,,...ε,,n,,)''. Importantly, this is not a function of '''''X''''', so the standard errors can be estimated as:

{{attachment:robust.svg}}

Robust errors are only appropriate with large sample sizes.

[[SamplingWeightsAndRegressionAnalysis|When fitting a model using data with survey weights, if those weights are a function of predictors including the dependent variable, then heteroskedastic consistent errors should be used.]]

[[HowRobustStandardErrorsExposeMethodologicalProblemsTheyDoNotFix|If a model significantly diverges after introducing robust errors, there is likely a specification error.]]

----



== Clustered ==

'''Liang-Zeger clustered robust standard errors''' assume that errors covary within clusters.

{{attachment:cluster1.svg}}

where '''''x''',,g,,'' is an ''n,,g,,'' by ''k'' matrix constructed by stacking '''''x''',,i,,'' for all ''i'' belonging to cluster ''g''; and '''''ε''',,g,,'' is an ''n,,g,,'' long vector holding the errors for each cluster ''g''.

The estimator becomes:

{{attachment:cluster2.svg}}

Clustered standard errors should only be used if the sample design or experimental design call for it.
 * A complex survey sample design leads to differential sampling errors across strata.
 * A two-stage sample design leads to differential sampling errors for the SSU within each PSU.
 * Assignment of an experimental treatment at a grouped level often leads to differential errors across those groups.
 * For time series evaluation of an experimental treatment that is assigned at the individual level, it is generally recommended to cluster at the individual level.

There are parallels between [[Statistics/FixedEffectsModel|fixed effects]] and clusters, but use of one does not mandate nor conflict with the other.



----
CategoryRicottone