|
Size: 1749
Comment: Partial
|
Size: 797
Comment: Killing Econometrics page
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| = Ordinary Least Squares Univariate Proof = | ## page was renamed from Econometrics/OrdinaryLeastSquares/Multivariate = OLS Multivariate Derivation = |
| Line 13: | Line 14: |
| The model is fit by a minimization problem: {{attachment:min.svg}} |
|
| Line 21: | Line 18: |
| ---- This line must pass through the mean and the slope of the line must be the marginal change in ''Y'' given a unit change in ''X''. In other words, the line must pass through two points: |
with the constraint: |
| Line 28: | Line 22: |
| where: | The [[Calculus/PartialDerivatives|partial derivative]] of this constraint with respect to ''b'' is calculated like: |
| Line 30: | Line 24: |
| * ''X‾'' is the sample mean of ''X'' (estimating ''μ,,X,,'') * ''Y‾'' is the sample mean of ''Y'' (estimating ''μ,,Y,,'') * ''s,,X,,'' is the sample standard deviation of ''X'' (estimating ''σ,,X,,'') * ''s,,Y,,'' is the sample standard deviation of ''Y'' (estimating ''σ,,Y,,'') * and ''r,,XY,,'' is the sample correlation coefficient between ''X'' and ''Y'' (estimating ''ρ,,XY,,'') |
{{attachment:b1.svg}} |
| Line 36: | Line 26: |
| Insert the first point into the estimation. This is quickly solved for ''α''. | {{attachment:b2.svg}} |
| Line 38: | Line 28: |
| {{attachment:alpha1.svg}} | Set this derivative to 0 to find the minimum with respect to ''b''. |
| Line 40: | Line 30: |
| {{attachment:alpha2.svg}} | {{attachment:b3.svg}} |
| Line 42: | Line 32: |
| Insert the second point and the solution for ''α'' into the estimation. | {{attachment:b4.svg}} |
| Line 44: | Line 34: |
| {{attachment:beta1.svg}} | {{attachment:b5.svg}} |
| Line 46: | Line 36: |
| {{attachment:beta2.svg}} | {{attachment:b6.svg}} |
| Line 48: | Line 38: |
| {{attachment:beta3.svg}} This reduced form can be quickly solved for ''β''. {{attachment:beta4.svg}} Because the correlation coefficient can be expressed in terms of covariance and standard deviations... {{attachment:correlation.svg}} ...the solution for ''β'' can be further reduced. {{attachment:beta5.svg}} Therefore, the regression line is estimated to be: {{attachment:regression.svg}} |
OLS Multivariate Derivation
The model is constructed like:
where:
y and ε are vectors of size n
β is a vector of size p
X is a matrix of shape n by p
This is estimated as:
with the constraint:
The partial derivative of this constraint with respect to b is calculated like:
Set this derivative to 0 to find the minimum with respect to b.
