Differences between revisions 12 and 15 (spanning 3 versions)

Ordinary Least Squares

Ordinary Least Squares (OLS) is a linear regression method. It minimizes root mean square errors.

Contents

Univariate

The regression line passes through two points:

and

These points, with the generic equation for a line, can prove that the slope of the regression line is equal to:

The generic formula for the regression line is:

The linear model can be expressed as:

If these assumptions can be made:

Then OLS is the best linear unbiased estimator (BLUE) for these coefficients.

Using the computation above, the coefficients are estimated to produce:

The variances for each coefficient are:

Note that the standard deviation of the population's parameter is unknown, so it's estimated like:

If the homoskedasticity assumption does not hold, then the estimators for each coefficient are actually:

Wherein, for example, r_1j is the residual from regressing x₁ onto x₂, ... x_k.

The variances for each coefficient can be estimated with the Eicker-White formula:

Statistics/OrdinaryLeastSquares (last edited 2025-09-03 02:08:40 by DominicRicottone)

-  ⇤ ← Revision 12 as of 2023-10-28 17:49:29 → 
  Size: 1660
  Editor: DominicRicottone
  Comment:
+   ← Revision 15 as of 2023-10-28 18:25:46 → ⇥
  Size: 1730
  Editor: DominicRicottone
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 33:
+== Multivariate ==

----
-Line 51:
+Line 57:
-{{attachment:model3.svg}}
+{{attachment:model2.svg}}
-Line 55:
+Line 61:
-{{attachment:model4.svg}}
+{{attachment:homo1.svg}}
-Line 57:
+Line 63:
-Note also that the standard deviation of the population's parameter is unknown, so it's estimated like:
+Note that the standard deviation of the population's parameter is unknown, so it's estimated like:
-Line 59:
+Line 65:
-{{attachment:model6.svg}}
+{{attachment:homo2.svg}}
-Line 65:
+Line 71:
-It follows that the variances for each coefficient are:
+Wherein, for example, ''r,,1j,,'' is the residual from regressing ''x,,1,,'' onto ''x,,2,,'', ... ''x,,k,,''.

The variances for each coefficient can be estimated with the Eicker-White formula:
-Line 68:
+Line 76:
-These variances can be estimated with the Eicker-White formula:

{{attachment:hetero3.svg}}