OLS Univariate Derivation

The model is constructed like:

model1.svg

The model is fit by a minimization problem:

min.svg

This is estimated as:

model2.svg

This line must pass through the mean and the slope of the line must be the marginal change in Y given a unit change in X. In other words, the line must pass through two points:

model3.svg

where:

Insert the first point into the estimation. This is quickly solved for α.

alpha1.svg

alpha2.svg

Insert the second point and the solution for α into the estimation.

beta1.svg

beta2.svg

beta3.svg

This reduced form can be quickly solved for β.

beta4.svg

Because the correlation coefficient can be expressed in terms of covariance and standard deviations...

correlation.svg

...the solution for β can be further reduced.

beta5.svg

Therefore, the regression line is estimated to be:

regression.svg


CategoryRicottone

Statistics/OrdinaryLeastSquares/Univariate (last edited 2025-01-10 14:33:54 by DominicRicottone)