Differences between revisions 1 and 4 (spanning 3 versions)

Ordinary Least Squares Univariate Proof

The model is constructed like:

This is estimated as:

This line must pass through the mean and the slope of the line must be the marginal change in Y given a unit change in X. In other words, the line must pass through two points:

where:

X‾ is the sample mean of X (estimating μ_X)
Y‾ is the sample mean of Y (estimating μ_Y)
s_X is the sample standard deviation of X (estimating σ_X)
s_Y is the sample standard deviation of Y (estimating σ_Y)
and r_XY is the sample correlation coefficient between X and Y (estimating ρ_XY)

Insert the first point into the estimation. This is quickly solved for α.

Insert the second point and the solution for α into the estimation.

Expand the formula for correlation as:

This can now be eliminated into:

Finally, b can be eloquently written as:

Giving a generic formula for the regression line:

CategoryRicottone

Statistics/OrdinaryLeastSquares/Univariate (last edited 2025-01-10 14:33:54 by DominicRicottone)

-  ⇤ ← Revision 1 as of 2023-10-28 16:16:10 → 
  Size: 1172
  Editor: DominicRicottone
  Comment: Extracted this content from main page for OLS
+   ← Revision 4 as of 2024-06-05 21:09:40 → ⇥
  Size: 1437
  Editor: DominicRicottone
  Comment: Rewrite 3
-Deletions are marked like this.
+Additions are marked like this.
 Line 3:
-As a starting point, the regression line must pass through these two points:
+The model is constructed like:
 Line 5:
-{{attachment:regression1.svg}}
+{{attachment:model1.svg}}
 Line 7:
-and
+This is estimated as:
 Line 9:
-{{attachment:regression2.svg}}
+{{attachment:model2.svg}}
 Line 11:
-Take the generic equation form of a line:
+This line must pass through the mean and the slope of the line must be the marginal change in ''Y'' given a unit change in ''X''. In other words, the line must pass through two points:
 Line 13:
-{{attachment:b01.svg}}
+{{attachment:model3.svg}}
 Line 15:
-Insert the first point into this form.
+where:
 Line 17:
-{{attachment:b02.svg}}
+ * ''X‾'' is the sample mean of ''X'' (estimating ''μ,,X,,'')
 * ''Y‾'' is the sample mean of ''Y'' (estimating ''μ,,Y,,'')
 * ''s,,X,,'' is the sample standard deviation of ''X'' (estimating ''σ,,X,,'')
 * ''s,,Y,,'' is the sample standard deviation of ''Y'' (estimating ''σ,,Y,,'')
 * and ''r,,XY,,'' is the sample correlation coefficient between ''X'' and ''Y'' (estimating ''ρ,,XY,,'')
-Line 19:
+Line 23:
-This can be trivially rewritten to solve for ''a'' in terms of ''b'':
+Insert the first point into the estimation. This is quickly solved for ''α''.
-Line 21:
+Line 25:
-{{attachment:b03.svg}}
+{{attachment:alpha1.svg}}
-Line 23:
+Line 27:
-Insert the second point into the original form.
+{{attachment:alpha2.svg}}
-Line 25:
+Line 29:
-{{attachment:b04.svg}}
+Insert the second point and the solution for ''α'' into the estimation.
-Line 27:
+Line 31:
-Now additionally insert the solution for ''a'' in terms of ''b''.
+{{attachment:beta1.svg}}
-Line 29:
+Line 33:
-{{attachment:b05.svg}}
+{{attachment:beta2.svg}}
-Line 31:
+Line 35:
-Expand all terms to produce:
+{{attachment:beta3.svg}}
-Line 33:
+Line 37:
-{{attachment:b06.svg}}

This can now be eliminated into:

{{attachment:b07.svg}}

Giving a solution for ''b'':

{{attachment:b08.svg}}

This solution is trivially rewritten as:

{{attachment:b09.svg}}
+{{attachment:beta4.svg}}

Diff for "Statistics/OrdinaryLeastSquares/Univariate"

Ordinary Least Squares Univariate Proof