|
Size: 2037
Comment: Simplifications
|
← Revision 30 as of 2026-07-21 01:33:10 ⇥
Size: 2068
Comment: Relink
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 13: | Line 13: |
| A linear model is expressed as either {{attachment:model.svg}} (univariate) or {{attachment:mmodel.svg}} (multivariate with ''k'' terms). Either way, a crucial assumption is that the expected value of the error term is 0, such that the [[Statistics/Moments|first moment]] is ''E[y,,i,,|x,,i,,] = α + βx,,i,,''. | A linear model is expressed as either {{attachment:model.svg}} (univariate) or {{attachment:mmodel.svg}} (multivariate with ''k'' terms). Either way, a crucial assumption is that the [[Analysis/ExpectedValue|expected value]] of the [[Analysis/Error|error term]] is 0, such that ''E[y,,i,,|x,,i,,] = α + βx,,i,,''. |
| Line 17: | Line 17: |
| === Univariate === | === Single Regression === |
| Line 19: | Line 19: |
| In the univariate case, the OLS regression is: | In the case of a single predictor, the OLS regression is: |
| Line 25: | Line 25: |
| The derivation can be seen [[Statistics/OrdinaryLeastSquares/Univariate|here]]. | The derivation can be seen [[Statistics/OrdinaryLeastSquares/Single|here]]. |
| Line 30: | Line 30: |
| == Multivariate == | === Multiple Regression === |
| Line 32: | Line 32: |
| In the multivariate case, the regression is fit like: | In the case of multiple predictors, the regression is fit like: |
| Line 36: | Line 36: |
| But conventionally, multivariate OLS is solved using [[LinearAlgebra|linear algebra]] as: | But conventionally, this OLS system is solved using [[LinearAlgebra|linear algebra]] as: |
| Line 42: | Line 42: |
| The derivation can be seen [[Statistics/OrdinaryLeastSquares/Multivariate|here]]. | The derivation can be seen [[Statistics/OrdinaryLeastSquares/Multiple|here]]. |
Ordinary Least Squares
Ordinary Least Squares (OLS) is a linear regression method, and is effectively synonymous with the linear regression model.
Contents
Description
A linear model is expressed as either 
(univariate) or 
(multivariate with k terms). Either way, a crucial assumption is that the expected value of the error term is 0, such that E[yi|xi] = α + βxi.
Single Regression
In the case of a single predictor, the OLS regression is:
This formulation leaves the components explicit: the y-intercept term is the mean outcome at x=0, and the slope term is marginal change to the outcome per a unit change in x.
The derivation can be seen here.
Multiple Regression
In the case of multiple predictors, the regression is fit like:
But conventionally, this OLS system is solved using linear algebra as:
Note that using a b here is intentional.
The derivation can be seen here.
Estimated Coefficients
The Gauss-Markov theorem demonstrates that (with some assumptions) the OLS estimations are the best linear unbiased estimators (BLUE) for the regression coefficients. The assumptions are:
- Linearity
- Exogeneity, i.e. predictors are independent of the outcome and the error term
- Random sampling
No perfect multicolinearity
- Homoskedasticity, i.e. error terms are constant across observations
#5 mostly comes into the estimation of standard errors, and there are alternative estimators that are robust to heteroskedasticity.