Differences between revisions 7 and 12 (spanning 5 versions)

Ordinary Least Squares

Ordinary Least Squares (OLS) is a linear regression method. It minimizes root mean square errors.

Contents

Univariate

The regression line passes through two points:

and

These points, with the generic equation for a line, can prove that the slope of the regression line is equal to:

The generic formula for the regression line is:

The linear model can be expressed as:

If these assumptions can be made:

Then OLS is the best linear unbiased estimator (BLUE) for these coefficients.

Using the computation above, the coefficients are estimated to produce:

The variances for each coefficient are:

Note also that the standard deviation of the population's parameter is unknown, so it's estimated like:

If the homoskedasticity assumption does not hold, then the estimators for each coefficient are actually:

It follows that the variances for each coefficient are:

These variances can be estimated with the Eicker-White formula:

Statistics/OrdinaryLeastSquares (last edited 2025-05-17 03:48:23 by DominicRicottone)

-  ⇤ ← Revision 7 as of 2023-10-28 16:21:41 → 
  Size: 1381
  Editor: DominicRicottone
  Comment:
+   ← Revision 12 as of 2023-10-28 17:49:29 → ⇥
  Size: 1660
  Editor: DominicRicottone
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 42:
-. Exogeneity

{{attachment:model2.svg}}

 3.#3 Random sampling
+. [[Econometrics/Exogeneity|Exogeneity]]
 3. Random sampling
-Line 48:
+Line 45:
-. Heteroskedasticity
+. [[Econometrics/Homoskedasticity|Homoskedasticity]]
-Line 56:
+Line 53:
-The variance for each coefficient is estimated as:
+The variances for each coefficient are:
-Line 59:
+Line 56:
-Where R^2^ is calculated as:

{{attachment:model5.svg}}
-Line 68:
+Line 61:
+If the homoskedasticity assumption does not hold, then the estimators for each coefficient are actually:

{{attachment:hetero1.svg}}

It follows that the variances for each coefficient are:

{{attachment:hetero2.svg}}

These variances can be estimated with the Eicker-White formula:

{{attachment:hetero3.svg}}