|
Size: 1576
Comment: Heteroskedasticity
|
Size: 1860
Comment: Rewrite 2
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 13: | Line 13: |
| The regression line passes through two points: | Given one independent variable and one dependent (outcome) variable, the OLS model is specified as: |
| Line 15: | Line 15: |
| {{attachment:regression1.svg}} | {{attachment:model.svg}} |
| Line 17: | Line 17: |
| and | It is estimated as: |
| Line 19: | Line 19: |
| {{attachment:regression2.svg}} | {{attachment:estimate.svg}} |
| Line 21: | Line 21: |
| These points, with the generic equation for a line, can [[Econometrics/OrdinaryLeastSquares/UnivariateProof|prove]] that the slope of the regression line is equal to: | This model describes (1) the mean observation and (2) the marginal changes to the outcome per unit changes in the independent variable. |
| Line 23: | Line 23: |
| {{attachment:b12.svg}} The generic formula for the regression line is: {{attachment:b13.svg}} |
The proof can be seen [[Econometrics/OrdinaryLeastSquares/UnivariateProof|here]]. |
| Line 33: | Line 29: |
| == Linear Model == | == Multivariate == |
| Line 35: | Line 31: |
| The linear model can be expressed as: | Given ''k'' independent variables, the OLS model is specified as: |
| Line 37: | Line 33: |
| {{attachment:model1.svg}} | {{attachment:mmodel.svg}} It is estimated as: {{attachment:mestimate.svg}} ---- == Estimated Coefficients == |
| Line 47: | Line 53: |
| Then OLS is the best linear unbiased estimator ('''BLUE''') for these coefficients. | Then OLS is the best linear unbiased estimator ('''BLUE''') for regression coefficients. |
| Line 49: | Line 55: |
| Using the computation above, the coefficients are estimated to produce: | The variances for each coefficient are: |
| Line 51: | Line 57: |
| {{attachment:model3.svg}} | {{attachment:homo1.svg}} |
| Line 53: | Line 59: |
| The variances for each coefficient is estimated as: | Note that the standard deviation of the population's parameter is unknown, so it's estimated like: |
| Line 55: | Line 61: |
| {{attachment:model4.svg}} Note also that the standard deviation of the population's parameter is unknown, so it's estimated like: {{attachment:model6.svg}} |
{{attachment:homo2.svg}} |
| Line 65: | Line 67: |
| And the variances for each coefficient are estimated as: | Wherein, for example, ''r,,1j,,'' is the residual from regressing ''x,,1,,'' onto ''x,,2,,'', ... ''x,,k,,''. The variances for each coefficient can be estimated with the Eicker-White formula: |
| Line 68: | Line 72: |
See [[https://www.youtube.com/@kuminoff|Nicolai Kuminoff's]] video lectures for the derivation of the robust estimators. |
Ordinary Least Squares
Ordinary Least Squares (OLS) is a linear regression method. It minimizes root mean square errors.
Univariate
Given one independent variable and one dependent (outcome) variable, the OLS model is specified as:
It is estimated as:
This model describes (1) the mean observation and (2) the marginal changes to the outcome per unit changes in the independent variable.
The proof can be seen here.
Multivariate
Given k independent variables, the OLS model is specified as:
It is estimated as:
Estimated Coefficients
If these assumptions can be made:
- Linearity
- Random sampling
- No perfect multicolinearity
Then OLS is the best linear unbiased estimator (BLUE) for regression coefficients.
The variances for each coefficient are:
![[ATTACH]](/Statistics/OrdinaryLeastSquares?action=AttachFile&do=upload_form&target=homo1.svg&ticket=0069f67c45.4d8cd4a7da27bd9667d848684f8f85d7bc44d9fa "[ATTACH]"){kind=small}
Note that the standard deviation of the population's parameter is unknown, so it's estimated like:
![[ATTACH]](/Statistics/OrdinaryLeastSquares?action=AttachFile&do=upload_form&target=homo2.svg&ticket=0069f67c45.4d8cd4a7da27bd9667d848684f8f85d7bc44d9fa "[ATTACH]"){kind=small}
If the homoskedasticity assumption does not hold, then the estimators for each coefficient are actually:
![[ATTACH]](/Statistics/OrdinaryLeastSquares?action=AttachFile&do=upload_form&target=hetero1.svg&ticket=0069f67c45.4d8cd4a7da27bd9667d848684f8f85d7bc44d9fa "[ATTACH]"){kind=small}
Wherein, for example, r1j is the residual from regressing x1 onto x2, ... xk.
The variances for each coefficient can be estimated with the Eicker-White formula:
![[ATTACH]](/Statistics/OrdinaryLeastSquares?action=AttachFile&do=upload_form&target=hetero2.svg&ticket=0069f67c45.4d8cd4a7da27bd9667d848684f8f85d7bc44d9fa "[ATTACH]"){kind=small}
See Nicolai Kuminoff's video lectures for the derivation of the robust estimators.