Differences between revisions 2 and 12 (spanning 10 versions)
Revision 2 as of 2023-10-28 05:37:22
Size: 1293
Comment:
Revision 12 as of 2023-10-28 17:49:29
Size: 1660
Comment:
Deletions are marked like this. Additions are marked like this.
Line 21: Line 21:
Take the generic equation form of a line:

{{attachment:b01.svg}}

Insert the first point into this form.

{{attachment:b02.svg}}

This can be trivially rewritten to solve for ''a'' in terms of ''b'':

{{attachment:b03.svg}}

Insert the second point into the original form.

{{attachment:b04.svg}}

Now additionally insert the solution for ''a'' in terms of ''b''.

{{attachment:b05.svg}}

Expand all terms to produce:

{{attachment:b06.svg}}

This can now be eliminated into:

{{attachment:b07.svg}}

Giving a solution for ''b'':

{{attachment:b08.svg}}

This solution is trivially rewritten as:

{{attachment:b09.svg}}

Expand the formula for correlation as:

{{attachment:b10.svg}}

This can now be eliminated into:

{{attachment:b11.svg}}

Finally, ''b'' can be eloquently written as:
These points, with the generic equation for a line, can [[Econometrics/OrdinaryLeastSquares/UnivariateProof|prove]] that the slope of the regression line is equal to:
Line 69: Line 25:
Giving a generic formula for the regression line: The generic formula for the regression line is:
Line 72: Line 28:

----



== Linear Model ==

The linear model can be expressed as:

{{attachment:model1.svg}}

If these assumptions can be made:

 1. Linearity
 2. [[Econometrics/Exogeneity|Exogeneity]]
 3. Random sampling
 4. No perfect multicolinearity
 5. [[Econometrics/Homoskedasticity|Homoskedasticity]]

Then OLS is the best linear unbiased estimator ('''BLUE''') for these coefficients.

Using the computation above, the coefficients are estimated to produce:

{{attachment:model3.svg}}

The variances for each coefficient are:

{{attachment:model4.svg}}

Note also that the standard deviation of the population's parameter is unknown, so it's estimated like:

{{attachment:model6.svg}}

If the homoskedasticity assumption does not hold, then the estimators for each coefficient are actually:

{{attachment:hetero1.svg}}

It follows that the variances for each coefficient are:

{{attachment:hetero2.svg}}

These variances can be estimated with the Eicker-White formula:

{{attachment:hetero3.svg}}

Ordinary Least Squares

Ordinary Least Squares (OLS) is a linear regression method. It minimizes root mean square errors.


Univariate

The regression line passes through two points:

[ATTACH]

and

[ATTACH]

These points, with the generic equation for a line, can prove that the slope of the regression line is equal to:

[ATTACH]

The generic formula for the regression line is:

[ATTACH]


Linear Model

The linear model can be expressed as:

model1.svg

If these assumptions can be made:

  1. Linearity
  2. Exogeneity

  3. Random sampling
  4. No perfect multicolinearity
  5. Homoskedasticity

Then OLS is the best linear unbiased estimator (BLUE) for these coefficients.

Using the computation above, the coefficients are estimated to produce:

[ATTACH]

The variances for each coefficient are:

[ATTACH]

Note also that the standard deviation of the population's parameter is unknown, so it's estimated like:

[ATTACH]

If the homoskedasticity assumption does not hold, then the estimators for each coefficient are actually:

[ATTACH]

It follows that the variances for each coefficient are:

[ATTACH]

These variances can be estimated with the Eicker-White formula:

[ATTACH]


CategoryRicottone

Statistics/OrdinaryLeastSquares (last edited 2025-08-06 00:56:27 by DominicRicottone)