= Degrees of Freedom = '''Degrees of freedom''' is a measure of how much an estimate is able to vary. <> ---- == Mean == Given a random sample of ''n'' observations (''x,,i,,'') from a larger unknown population (''X''), the true population's mean (''μ'') can be estimated using the sample mean. {{attachment:mean.svg}} With the mean estimated from the sample, the sample has lost a '''degree of freedom'''. As long as the mean is fixed at this estimate, the first ''n - 1'' observations are allowed to vary, but the ''n''th observation is fixed at whatever value enables the mean equation to remain true. As a result, subsequent equations making use of the estimated mean must deduct 1 from the sample size. For example, estimation of the true population's standard deviation (''σ'') with the sample standard deviation while making use of the sample mean. {{attachment:stddev.svg}} ---- == Regression == A regression is a (''frequently'' but not ''necessarily'' linear) model in terms of variables that minimizes an error term. Consider [[Statistics/OrdinaryLeastSquares|OLS]]: {{attachment:ols.svg}} This model describes (1) the mean observation and (2) the marginal changes to a dependent variable per unit changes in independent variables, given a standard error term on each variable. Intuitively consider: * Given a model of 2 variables (an independent ''x'' and a dependent ''y'') and 2 observations, a line can be drawn in terms of those 2 variables but there can be no error; the line will directly connect those 2 observations. * Given a model of 3 variables and 3 observations, a plane can be drawn but there can be no error; the plane will directly connect those 3 observations. * For the ''independent variables'' to be able to ''independently vary'', there must be more degrees of freedom. The sample mean deducts 1 degree of freedom, and every independent variable's standard error also deducts 1 degree of freedom. The degrees of freedom on a regression are ''n - k - 1''. ---- CategoryRicottone