|
Size: 1749
Comment: Partial
|
Size: 486
Comment: Partial 3?
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| = Ordinary Least Squares Univariate Proof = | = Ordinary Least Squares Multivariate Proof = |
| Line 13: | Line 13: |
| The model is fit by a minimization problem: {{attachment:min.svg}} |
|
| Line 21: | Line 17: |
| ---- This line must pass through the mean and the slope of the line must be the marginal change in ''Y'' given a unit change in ''X''. In other words, the line must pass through two points: |
with the constraint: |
| Line 28: | Line 21: |
| where: | The partial derivative of this constraint with regard to ''b'' is calculated like: |
| Line 30: | Line 23: |
| * ''X‾'' is the sample mean of ''X'' (estimating ''μ,,X,,'') * ''Y‾'' is the sample mean of ''Y'' (estimating ''μ,,Y,,'') * ''s,,X,,'' is the sample standard deviation of ''X'' (estimating ''σ,,X,,'') * ''s,,Y,,'' is the sample standard deviation of ''Y'' (estimating ''σ,,Y,,'') * and ''r,,XY,,'' is the sample correlation coefficient between ''X'' and ''Y'' (estimating ''ρ,,XY,,'') |
|
| Line 36: | Line 24: |
| Insert the first point into the estimation. This is quickly solved for ''α''. {{attachment:alpha1.svg}} {{attachment:alpha2.svg}} Insert the second point and the solution for ''α'' into the estimation. {{attachment:beta1.svg}} {{attachment:beta2.svg}} {{attachment:beta3.svg}} This reduced form can be quickly solved for ''β''. {{attachment:beta4.svg}} Because the correlation coefficient can be expressed in terms of covariance and standard deviations... {{attachment:correlation.svg}} ...the solution for ''β'' can be further reduced. {{attachment:beta5.svg}} Therefore, the regression line is estimated to be: {{attachment:regression.svg}} |
Ordinary Least Squares Multivariate Proof
The model is constructed like:
where:
y and ε are vectors of size n
β is a vector of size p
X is a matrix of shape n by p
This is estimated as:
with the constraint:
The partial derivative of this constraint with regard to b is calculated like: