|
⇤ ← Revision 1 as of 2024-06-05 14:25:01
Size: 964
Comment: Initial commit
|
Size: 1978
Comment: Regression
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 23: | Line 23: |
| ---- == Regression == A regression is a (''frequently'' but not ''necessarily'' linear) model in terms of variables that minimizes an error term. Consider [[Econometrics/OrdinaryLeastSquares|OLS]]: {{attachment:ols.svg}} This model describes (1) the mean observation and (2) the marginal changes to a dependent variable per unit changes in independent variables, given a standard error term on each variable. Intuitively consider: * Given a model of 2 variables (an independent ''x'' and a dependent ''y'') and 2 observations, a line can be drawn in terms of those 2 variables but there can be no error; the line will directly connect those 2 observations. * Given a model of 3 variables and 3 observations, a plane can be drawn but there can be no error; the plane will directly connect those 3 observations. The sample mean deducts 1 degree of freedom, and every variable's standard error also deducts 1 degree of freedom. The degrees of freedom on a regression are ''n - k - 1''. |
Degrees of Freedom
Degrees of freedom is a measure of how much an estimate is able to vary.
Contents
Mean
Given a random sample of n observations (xi) from a larger unknown population (X), the true population's mean (μ) can be estimated using the sample mean.
With the mean estimated from the sample, the sample has lost a degree of freedom. As long as the mean is fixed at this estimate, the first n - 1 observations are allowed to vary, but the nth observation is fixed at whatever value enables the mean equation to remain true.
As a result, subsequent equations making use of the estimated mean must deduct 1 from the sample size. For example, estimation of the true population's standard deviation (σ) with the sample standard deviation while making use of the sample mean.
Regression
A regression is a (frequently but not necessarily linear) model in terms of variables that minimizes an error term. Consider OLS:
This model describes (1) the mean observation and (2) the marginal changes to a dependent variable per unit changes in independent variables, given a standard error term on each variable.
Intuitively consider:
Given a model of 2 variables (an independent x and a dependent y) and 2 observations, a line can be drawn in terms of those 2 variables but there can be no error; the line will directly connect those 2 observations.
- Given a model of 3 variables and 3 observations, a plane can be drawn but there can be no error; the plane will directly connect those 3 observations.
The sample mean deducts 1 degree of freedom, and every variable's standard error also deducts 1 degree of freedom. The degrees of freedom on a regression are n - k - 1.