= OLS Univariate Derivation =

The model is constructed like:

{{attachment:model1.svg}}

The model is fit by a minimization problem:

{{attachment:min.svg}}

This is estimated as:

{{attachment:model2.svg}}

This line must pass through the mean and the slope of the line must be the marginal change in ''Y'' given a unit change in ''X''. In other words, the line must pass through two points:

{{attachment:model3.svg}}

where:

 * ''X‾'' is the sample mean of ''X'' (estimating ''μ,,X,,'')
 * ''Y‾'' is the sample mean of ''Y'' (estimating ''μ,,Y,,'')
 * ''s,,X,,'' is the sample standard deviation of ''X'' (estimating ''σ,,X,,'')
 * ''s,,Y,,'' is the sample standard deviation of ''Y'' (estimating ''σ,,Y,,'')
 * and ''r,,XY,,'' is the sample correlation coefficient between ''X'' and ''Y'' (estimating ''ρ,,XY,,'')

Insert the first point into the estimation. This is quickly solved for ''α''.

{{attachment:alpha1.svg}}

{{attachment:alpha2.svg}}

Insert the second point and the solution for ''α'' into the estimation.

{{attachment:beta1.svg}}

{{attachment:beta2.svg}}

{{attachment:beta3.svg}}

This reduced form can be quickly solved for ''β''.

{{attachment:beta4.svg}}

Because the correlation coefficient can be expressed in terms of covariance and standard deviations...

{{attachment:correlation.svg}}

...the solution for ''β'' can be further reduced.

{{attachment:beta5.svg}}

Therefore, the regression line is estimated to be:

{{attachment:regression.svg}}



----
CategoryRicottone