Differences between revisions 8 and 17 (spanning 9 versions)
⇤ ← Revision 8 as of 2023-10-28 16:30:24 →
Size: 1412
Editor: DominicRicottone
Comment:
← Revision 17 as of 2024-06-05 14:58:24 → ⇥
Size: 1809
Editor: DominicRicottone
Comment: Simplify language
Deletions are marked like this. Additions are marked like this.
Line 21: Line 21:
These points, with the generic equation for a line, can [[Econometrics/OrdinaryLeastSquares/UnivariateProof|prove]] that the slope of the regression line is equal to: It can be [[Econometrics/OrdinaryLeastSquares/UnivariateProof|proven]] that the slope of the regression line is equal to:
Line 28: Line 28:

----



== Multivariate ==
Line 45: Line 51:
 5. [[Econometrics/Heteroskedasticity|Heteroskedasticity]]  5. [[Econometrics/Homoskedasticity|Homoskedasticity]]
Line 51: Line 57:
{{attachment:model3.svg}} {{attachment:model2.svg}}
Line 53: Line 59:
The variance for each coefficient is estimated as: The variances for each coefficient are:
Line 55: Line 61:
{{attachment:model4.svg}} {{attachment:homo1.svg}}
Line 57: Line 63:
Where R^2^ is calculated as: Note that the standard deviation of the population's parameter is unknown, so it's estimated like:
Line 59: Line 65:
{{attachment:model5.svg}} {{attachment:homo2.svg}}
Line 61: Line 67:
Note also that the standard deviation of the population's parameter is unknown, so it's estimated like: If the homoskedasticity assumption does not hold, then the estimators for each coefficient are actually:
Line 63: Line 69:
{{attachment:model6.svg}} {{attachment:hetero1.svg}}

Wherein, for example, ''r,,1j,,'' is the residual from regressing ''x,,1,,'' onto ''x,,2,,'', ... ''x,,k,,''.

The variances for each coefficient can be estimated with the Eicker-White formula:

{{attachment:hetero2.svg}}

See [[https://www.youtube.com/@kuminoff|Nicolai Kuminoff's]] video lectures for the derivation of the robust estimators.

Ordinary Least Squares

Ordinary Least Squares (OLS) is a linear regression method. It minimizes root mean square errors.

Contents

  1. Ordinary Least Squares
    1. Univariate
    2. Multivariate
    3. Linear Model

Univariate

The regression line passes through two points:

[ATTACH]

and

[ATTACH]

It can be proven that the slope of the regression line is equal to:

[ATTACH]

The generic formula for the regression line is:

[ATTACH]

Multivariate

Linear Model

The linear model can be expressed as:

model1.svg

If these assumptions can be made:

  1. Linearity

  2. Exogeneity

  3. Random sampling
  4. No perfect multicolinearity

  5. Homoskedasticity

Then OLS is the best linear unbiased estimator (BLUE) for these coefficients.

Using the computation above, the coefficients are estimated to produce:

model2.svg

The variances for each coefficient are:

[ATTACH]

Note that the standard deviation of the population's parameter is unknown, so it's estimated like:

[ATTACH]

If the homoskedasticity assumption does not hold, then the estimators for each coefficient are actually:

[ATTACH]

Wherein, for example, r1j is the residual from regressing x1 onto x2, ... xk.

The variances for each coefficient can be estimated with the Eicker-White formula:

[ATTACH]

See Nicolai Kuminoff's video lectures for the derivation of the robust estimators.

CategoryRicottone

Statistics/OrdinaryLeastSquares (last edited 2025-09-03 02:08:40 by DominicRicottone)