Differences between revisions 2 and 9 (spanning 7 versions)

Ordinary Least Squares

Ordinary Least Squares (OLS) is a linear regression method. It minimizes root mean square errors.

Contents

Ordinary Least Squares
1. Univariate
2. Linear Model

Univariate

The regression line passes through two points:

and

These points, with the generic equation for a line, can prove that the slope of the regression line is equal to:

The generic formula for the regression line is:

Linear Model

The linear model can be expressed as:

If these assumptions can be made:

Linearity
Exogeneity
Random sampling
No perfect multicolinearity
Homoskedasticity

Then OLS is the best linear unbiased estimator (BLUE) for these coefficients.

Using the computation above, the coefficients are estimated to produce:

The variance for each coefficient is estimated as:

Where R² is calculated as:

Note also that the standard deviation of the population's parameter is unknown, so it's estimated like:

CategoryRicottone

Statistics/OrdinaryLeastSquares (last edited 2025-01-10 14:33:38 by DominicRicottone)

-  ⇤ ← Revision 2 as of 2023-10-28 05:37:22 → 
  Size: 1293
  Editor: DominicRicottone
  Comment:
+   ← Revision 9 as of 2023-10-28 16:49:33 → ⇥
  Size: 1408
  Editor: DominicRicottone
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 21:
-Take the generic equation form of a line:

{{attachment:b01.svg}}

Insert the first point into this form.

{{attachment:b02.svg}}

This can be trivially rewritten to solve for ''a'' in terms of ''b'':

{{attachment:b03.svg}}

Insert the second point into the original form.

{{attachment:b04.svg}}

Now additionally insert the solution for ''a'' in terms of ''b''.

{{attachment:b05.svg}}

Expand all terms to produce:

{{attachment:b06.svg}}

This can now be eliminated into:

{{attachment:b07.svg}}

Giving a solution for ''b'':

{{attachment:b08.svg}}

This solution is trivially rewritten as:

{{attachment:b09.svg}}

Expand the formula for correlation as:

{{attachment:b10.svg}}

This can now be eliminated into:

{{attachment:b11.svg}}

Finally, ''b'' can be eloquently written as:
+These points, with the generic equation for a line, can [[Econometrics/OrdinaryLeastSquares/UnivariateProof|prove]] that the slope of the regression line is equal to:
-Line 69:
+Line 25:
-Giving a generic formula for the regression line:
+The generic formula for the regression line is:
-Line 72:
+Line 28:
+----



== Linear Model ==

The linear model can be expressed as:

{{attachment:model1.svg}}

If these assumptions can be made:

 1. Linearity
 2. [[Econometrics/Exogeneity|Exogeneity]]
 3. Random sampling
 4. No perfect multicolinearity
 5. [[Econometrics/Homoskedasticity|Homoskedasticity]]

Then OLS is the best linear unbiased estimator ('''BLUE''') for these coefficients.

Using the computation above, the coefficients are estimated to produce:

{{attachment:model3.svg}}

The variance for each coefficient is estimated as:

{{attachment:model4.svg}}

Where R^2^ is calculated as:

{{attachment:model5.svg}}

Note also that the standard deviation of the population's parameter is unknown, so it's estimated like:

{{attachment:model6.svg}}

Diff for "Statistics/OrdinaryLeastSquares"

Ordinary Least Squares

Univariate

Linear Model