Differences between revisions 6 and 7

Ordinary Least Squares

Ordinary Least Squares (OLS) is a linear regression method. It minimizes root mean square errors.

Contents

Univariate

The regression line passes through two points:

and

These points, with the generic equation for a line, can prove that the slope of the regression line is equal to:

The generic formula for the regression line is:

The linear model can be expressed as:

If these assumptions can be made:

Then OLS is the best linear unbiased estimator (BLUE) for these coefficients.

Using the computation above, the coefficients are estimated to produce:

The variance for each coefficient is estimated as:

Where R² is calculated as:

Note also that the standard deviation of the population's parameter is unknown, so it's estimated like:

Statistics/OrdinaryLeastSquares (last edited 2025-01-10 14:33:38 by DominicRicottone)

-  ⇤ ← Revision 6 as of 2023-10-28 07:04:18 → 
  Size: 2049
  Editor: DominicRicottone
  Comment:
+   ← Revision 7 as of 2023-10-28 16:21:41 → ⇥
  Size: 1381
  Editor: DominicRicottone
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 21:
-Take the generic equation form of a line:

{{attachment:b01.svg}}

Insert the first point into this form.

{{attachment:b02.svg}}

This can be trivially rewritten to solve for ''a'' in terms of ''b'':

{{attachment:b03.svg}}

Insert the second point into the original form.

{{attachment:b04.svg}}

Now additionally insert the solution for ''a'' in terms of ''b''.

{{attachment:b05.svg}}

Expand all terms to produce:

{{attachment:b06.svg}}

This can now be eliminated into:

{{attachment:b07.svg}}

Giving a solution for ''b'':

{{attachment:b08.svg}}

This solution is trivially rewritten as:

{{attachment:b09.svg}}

Expand the formula for correlation as:

{{attachment:b10.svg}}

This can now be eliminated into:

{{attachment:b11.svg}}

Finally, ''b'' can be eloquently written as:
+These points, with the generic equation for a line, can [[Econometrics/OrdinaryLeastSquares/UnivariateProof|prove]] that the slope of the regression line is equal to:
-Line 69:
+Line 25:
-Giving a generic formula for the regression line:
+The generic formula for the regression line is: