|
Size: 1172
Comment: Extracted this content from main page for OLS
|
Size: 1518
Comment: Rewrite 5 (final)
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 3: | Line 3: |
| As a starting point, the regression line must pass through these two points: | The model is constructed like: |
| Line 5: | Line 5: |
| {{attachment:regression1.svg}} | {{attachment:model1.svg}} |
| Line 7: | Line 7: |
| and | This is estimated as: |
| Line 9: | Line 9: |
| {{attachment:regression2.svg}} | {{attachment:model2.svg}} |
| Line 11: | Line 11: |
| Take the generic equation form of a line: | This line must pass through the mean and the slope of the line must be the marginal change in ''Y'' given a unit change in ''X''. In other words, the line must pass through two points: |
| Line 13: | Line 13: |
| {{attachment:b01.svg}} | {{attachment:model3.svg}} |
| Line 15: | Line 15: |
| Insert the first point into this form. | where: |
| Line 17: | Line 17: |
| {{attachment:b02.svg}} | * ''X‾'' is the sample mean of ''X'' (estimating ''μ,,X,,'') * ''Y‾'' is the sample mean of ''Y'' (estimating ''μ,,Y,,'') * ''s,,X,,'' is the sample standard deviation of ''X'' (estimating ''σ,,X,,'') * ''s,,Y,,'' is the sample standard deviation of ''Y'' (estimating ''σ,,Y,,'') * and ''r,,XY,,'' is the sample correlation coefficient between ''X'' and ''Y'' (estimating ''ρ,,XY,,'') |
| Line 19: | Line 23: |
| This can be trivially rewritten to solve for ''a'' in terms of ''b'': | Insert the first point into the estimation. This is quickly solved for ''α''. |
| Line 21: | Line 25: |
| {{attachment:b03.svg}} | {{attachment:alpha1.svg}} |
| Line 23: | Line 27: |
| Insert the second point into the original form. | {{attachment:alpha2.svg}} |
| Line 25: | Line 29: |
| {{attachment:b04.svg}} | Insert the second point and the solution for ''α'' into the estimation. |
| Line 27: | Line 31: |
| Now additionally insert the solution for ''a'' in terms of ''b''. | {{attachment:beta1.svg}} |
| Line 29: | Line 33: |
| {{attachment:b05.svg}} | {{attachment:beta2.svg}} |
| Line 31: | Line 35: |
| Expand all terms to produce: | {{attachment:beta3.svg}} |
| Line 33: | Line 37: |
| {{attachment:b06.svg}} | This reduced form can be quickly solved for ''β''. |
| Line 35: | Line 39: |
| This can now be eliminated into: | {{attachment:beta4.svg}} |
| Line 37: | Line 41: |
| {{attachment:b07.svg}} | Because the correlation coefficient can be expressed in terms of covariance and standard deviations... |
| Line 39: | Line 43: |
| Giving a solution for ''b'': | {{attachment:correlation.svg}} |
| Line 41: | Line 45: |
| {{attachment:b08.svg}} | ...the solution for ''β'' can be further reduced. |
| Line 43: | Line 47: |
| This solution is trivially rewritten as: | {{attachment:beta5.svg}} |
| Line 45: | Line 49: |
| {{attachment:b09.svg}} | Therefore, the regression line is estimated to be: |
| Line 47: | Line 51: |
| Expand the formula for correlation as: {{attachment:b10.svg}} This can now be eliminated into: {{attachment:b11.svg}} Finally, ''b'' can be eloquently written as: {{attachment:b12.svg}} Giving a generic formula for the regression line: {{attachment:b13.svg}} |
{{attachment:regression.svg}} |
Ordinary Least Squares Univariate Proof
The model is constructed like:
This is estimated as:
This line must pass through the mean and the slope of the line must be the marginal change in Y given a unit change in X. In other words, the line must pass through two points:
where:
X‾ is the sample mean of X (estimating μX)
Y‾ is the sample mean of Y (estimating μY)
sX is the sample standard deviation of X (estimating σX)
sY is the sample standard deviation of Y (estimating σY)
and rXY is the sample correlation coefficient between X and Y (estimating ρXY)
Insert the first point into the estimation. This is quickly solved for α.
Insert the second point and the solution for α into the estimation.
This reduced form can be quickly solved for β.
Because the correlation coefficient can be expressed in terms of covariance and standard deviations...
...the solution for β can be further reduced.
Therefore, the regression line is estimated to be:
