|
Size: 1978
Comment: Regression
|
Size: 2105
Comment: Emphasis on independent variables
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 39: | Line 39: |
| * For the ''independent variables'' to be able to ''independently vary'', there must be more degrees of freedom. | |
| Line 40: | Line 41: |
| The sample mean deducts 1 degree of freedom, and every variable's standard error also deducts 1 degree of freedom. The degrees of freedom on a regression are ''n - k - 1''. | The sample mean deducts 1 degree of freedom, and every independent variable's standard error also deducts 1 degree of freedom. The degrees of freedom on a regression are ''n - k - 1''. |
Degrees of Freedom
Degrees of freedom is a measure of how much an estimate is able to vary.
Contents
Mean
Given a random sample of n observations (xi) from a larger unknown population (X), the true population's mean (μ) can be estimated using the sample mean.
With the mean estimated from the sample, the sample has lost a degree of freedom. As long as the mean is fixed at this estimate, the first n - 1 observations are allowed to vary, but the nth observation is fixed at whatever value enables the mean equation to remain true.
As a result, subsequent equations making use of the estimated mean must deduct 1 from the sample size. For example, estimation of the true population's standard deviation (σ) with the sample standard deviation while making use of the sample mean.
Regression
A regression is a (frequently but not necessarily linear) model in terms of variables that minimizes an error term. Consider OLS:
This model describes (1) the mean observation and (2) the marginal changes to a dependent variable per unit changes in independent variables, given a standard error term on each variable.
Intuitively consider:
Given a model of 2 variables (an independent x and a dependent y) and 2 observations, a line can be drawn in terms of those 2 variables but there can be no error; the line will directly connect those 2 observations.
- Given a model of 3 variables and 3 observations, a plane can be drawn but there can be no error; the plane will directly connect those 3 observations.
For the independent variables to be able to independently vary, there must be more degrees of freedom.
The sample mean deducts 1 degree of freedom, and every independent variable's standard error also deducts 1 degree of freedom. The degrees of freedom on a regression are n - k - 1.