= Normal Distribution = The '''normal distribution''' is a bell-shaped continuous probability distribution function that is parameterized to a mean and standard deviation. <> ---- == Description == The distribution is bell-shaped and parameterized to the [[Analysis/Moment|first and second central moments]]. This has useful consequences for estimating the probability that a given value is in the distribution. For example: * 68.27% of the cumulative distribution is within 1 standard deviation of the mean * 95.45% within 2 * 99.73% within 3 A variable distributed this way is notated like ''X ~ N(μ, σ^2^)''. The probability density function is given (and commonly notated) as: {{attachment:norm.svg}} Correspondingly, the cumulative probability function is commonly notated as ''Φ(.)'' rather than ''F(.)''. Furthermore the quantile function is commonly notated as ''Φ^-1^(.)''. === Standard Normal Distribution === The '''standard normal distribution''' is specified as ''Z ~ N(0, 1)''. Compare to the calculation for '''Z scores''' that will be compared to the standard normal distribution: ''Z = (x - μ)/σ''. The probability density function simplifies to: {{attachment:stdnorm.svg}}. This is graphed as: {{attachment:stdnormgraph.png||width=300px}} ---- == Moments == The [[Analysis/ExpectedValue|expected value]] is given as ''E[X] = μ''. [[Analysis/Variance|Variance]] is given as ''Var[X] = σ^2^''. ---- == Estimation == The [[Statistics/MaximumLikelihood|MLE]] for ''μ'' is ''X̅''. This is derived as: {{attachment:mu1.svg}} {{attachment:mu2.svg}} Now applying log properties: {{attachment:mu3.svg}} And now pulling all constants out of summations: {{attachment:mu4.svg}} The critical points of this equation are found by deriving with respect to ''μ''. The first part of the RHS is a constant with respect to ''μ'' to becomes 0. {{attachment:mu5.svg}} {{attachment:mu6.svg}} Multiply both sides by ''σ^2^'' and add ''nμ'' to both sides to get: {{attachment:mu7.svg}} Finally, recognize that ''X̅ = (1/n) Σ,,i,,(x,,i,,)''. Therefore ''μ,,MLE,, = X̅''. ---- == Usage == === Transformations and Combinations === Let ''X ~ N(μ, σ^2^)''. Adding a constant to ''X,,1,,'' shifts the mean by that constant, without affecting the variance. Multiplying ''X,,1,,'' by a constant factor multiplies the mean by that factor and multiplies the variance by the square of that factor. Succinctly: ''aX + b ~ N(aμ + b, a^2^σ^2^)''. Let ''X,,1,, ~ N(μ,,1,,``, σ^2^,,1,,)'' and ''X,,2,, ~ N(μ,,2,,``, σ^2^,,2,,)'', requiring that they are independent. The sum or difference of the two is also a random variable. In the case of a difference: ''X,,1,, - X,,2,, ~ N(μ,,1,, - μ,,2,,``, σ^2^,,1,, + σ^2^,,2,,)''. The sum is similarly distributed but centered on ''μ,,1,, + μ,,2,,''. The important detail to remember is that variances are added regardless of what is done to the means. Also take care to remember that a sample statistic ''X̅'' varies according to ''σ^2^,,X̅,, = σ^2^,,X,,/n'', so the difference of two sample statistics is expressed as ''X̅,,1,, - X̅,,2,, ~ N(μ,,1,, - μ,,2,,``, σ^2^,,1,,/n,,1,, + σ^2^,,2,,/n,,2,,)'' where ''n,,i,,'' is the sample size for sample ''i''. === Independence === Ordinarily, for given random variables ''X'' and ''Y'', a covariance of zero does not equate to independence. If ''X'' and ''Y'' follow a bivariate normal distribution however, the only way that covariance can be zero is if they are independent. === Normality of Sampling Distributions === For a population parameter whose population distribution is normal with mean ''μ'' and variance ''σ^2^'', the sampling distribution of the sample statistic which estimates that parameter is also normal. In particular, ''X̅'' is the sample statistic that estimates ''μ'', and its sampling distribution is normal with mean ''X̅ = μ'' and variance ''σ^2^,,X̅,, = σ^2^,,X,,/n'', where ''n'' is the sample size. By the central limit theorem, even when the population distribution in question is '''not''' normal, with a sufficiently large sample size the sampling distribution of the sample statistic which estimates that parameter '''is'''. A note on variance of a sampling distribution (i.e., ''σ^2^,,X̅,,'' not ''σ^2^,,X,,''). This is a function of both the sample variance ''σ^2^,,X,,'', which is usually itself estimated using sample variance (i.e., ''σ^2^,,X,, = s^2^,,X̅,,''), and the sample size ''n''. Consider a census of the population; there is only one possible sample so the sampling distribution has only a single level with 100% certainty. As the sample size decreases, the number of possible samples grows. The number of levels in the sampling distribution is also expected to grow accordingly. === Confidence Intervals === Given a known population distribution (i.e., ''X ~ N(μ, σ^2^)'' with known ''μ'' and ''σ''), a confidence interval can be constructed; 95% of samples will lead to a sample statistic that is within approximately 1.96 [[Statistics/SamplingDistribution#Standard_Error|standard errors]] of ''μ''. For similar reasons, a confidence interval can be constructed around a sample statistic. For example, if a 95% confidence interval is constructed around ''X̅'', it can be said with 95% confidence that the true population mean is within that interval. In the case that the distribution is known to be normal but the variance is unknown, use the [[Analysis/StudentsTDistribution|Student's t distribution]] instead. === Probability Tests === As noted above, '''Z scores''' (alt. ''Z statistics'') are test statistics for which likelihood can be calculated using the standard normal distribution. This is combined with the intuition of confidence intervals above. A null hypothesis is formulated in terms of a sample statistic and a confidence interval is constructed around it. The level of confidence should be based on tolerance for error. As an example, for a two-tailed test and a [[Statistics/TestStatistic|significance level]] of 5%, the critical Z score value is 1.96. This score is generally notated as ''Z,,α/2,,''. ---- CategoryRicottone