Differences between revisions 1 and 3 (spanning 2 versions)

Ordinary Least Squares Univariate Proof

The model is constructed like:

This is estimated as:

This line must pass through the mean and the slope of the line must be the marginal change in Y given a unit change in X. In other words, the line must pass through two points:

Insert the first point into the estimation. This is quickly solved for α.

Insert the second point into the original form.

Now additionally insert the solution for a in terms of b.

Expand all terms to produce:

This can now be eliminated into:

Giving a solution for b:

This solution is trivially rewritten as:

Expand the formula for correlation as:

This can now be eliminated into:

Finally, b can be eloquently written as:

Giving a generic formula for the regression line:

CategoryRicottone

Statistics/OrdinaryLeastSquares/Univariate (last edited 2025-01-10 14:33:54 by DominicRicottone)

-  ⇤ ← Revision 1 as of 2023-10-28 16:16:10 → 
  Size: 1172
  Editor: DominicRicottone
  Comment: Extracted this content from main page for OLS
+   ← Revision 3 as of 2024-06-05 20:57:05 → ⇥
  Size: 1253
  Editor: DominicRicottone
  Comment: Rewrite 2
-Deletions are marked like this.
+Additions are marked like this.
 Line 3:
-As a starting point, the regression line must pass through these two points:
+The model is constructed like:
 Line 5:
-{{attachment:regression1.svg}}
+{{attachment:model1.svg}}
 Line 7:
-and
+This is estimated as:
 Line 9:
-{{attachment:regression2.svg}}
+{{attachment:model2.svg}}
 Line 11:
-Take the generic equation form of a line:
+This line must pass through the mean and the slope of the line must be the marginal change in ''Y'' given a unit change in ''X''. In other words, the line must pass through two points:
 Line 13:
-{{attachment:b01.svg}}
+{{attachment:model3.svg}}
 Line 15:
-Insert the first point into this form.
+Insert the first point into the estimation. This is quickly solved for ''α''.
 Line 17:
-{{attachment:b02.svg}}
+{{attachment:alpha1.svg}}
 Line 19:
-This can be trivially rewritten to solve for ''a'' in terms of ''b'':

{{attachment:b03.svg}}
+{{attachment:alpha2.svg}}

Diff for "Statistics/OrdinaryLeastSquares/Univariate"

Ordinary Least Squares Univariate Proof