Differences between revisions 3 and 4

Ordinary Least Squares Univariate Proof

The model is constructed like:

This is estimated as:

This line must pass through the mean and the slope of the line must be the marginal change in Y given a unit change in X. In other words, the line must pass through two points:

where:

X‾ is the sample mean of X (estimating μ_X)
Y‾ is the sample mean of Y (estimating μ_Y)
s_X is the sample standard deviation of X (estimating σ_X)
s_Y is the sample standard deviation of Y (estimating σ_Y)
and r_XY is the sample correlation coefficient between X and Y (estimating ρ_XY)

Insert the first point into the estimation. This is quickly solved for α.

Insert the second point and the solution for α into the estimation.

Expand the formula for correlation as:

This can now be eliminated into:

Finally, b can be eloquently written as:

Giving a generic formula for the regression line:

CategoryRicottone

Statistics/OrdinaryLeastSquares/Univariate (last edited 2025-01-10 14:33:54 by DominicRicottone)

-  ⇤ ← Revision 3 as of 2024-06-05 20:57:05 → 
  Size: 1253
  Editor: DominicRicottone
  Comment: Rewrite 2
+   ← Revision 4 as of 2024-06-05 21:09:40 → ⇥
  Size: 1437
  Editor: DominicRicottone
  Comment: Rewrite 3
-Deletions are marked like this.
+Additions are marked like this.
 Line 15:
+where:

 * ''X‾'' is the sample mean of ''X'' (estimating ''μ,,X,,'')
 * ''Y‾'' is the sample mean of ''Y'' (estimating ''μ,,Y,,'')
 * ''s,,X,,'' is the sample standard deviation of ''X'' (estimating ''σ,,X,,'')
 * ''s,,Y,,'' is the sample standard deviation of ''Y'' (estimating ''σ,,Y,,'')
 * and ''r,,XY,,'' is the sample correlation coefficient between ''X'' and ''Y'' (estimating ''ρ,,XY,,'')
-Line 21:
+Line 29:
-Insert the second point into the original form.
+Insert the second point and the solution for ''α'' into the estimation.
-Line 23:
+Line 31:
-{{attachment:b04.svg}}
+{{attachment:beta1.svg}}
-Line 25:
+Line 33:
-Now additionally insert the solution for ''a'' in terms of ''b''.
+{{attachment:beta2.svg}}
-Line 27:
+Line 35:
-{{attachment:b05.svg}}
+{{attachment:beta3.svg}}
-Line 29:
+Line 37:
-Expand all terms to produce:

{{attachment:b06.svg}}

This can now be eliminated into:

{{attachment:b07.svg}}

Giving a solution for ''b'':

{{attachment:b08.svg}}

This solution is trivially rewritten as:

{{attachment:b09.svg}}
+{{attachment:beta4.svg}}

Diff for "Statistics/OrdinaryLeastSquares/Univariate"

Ordinary Least Squares Univariate Proof