Differences between revisions 3 and 12 (spanning 9 versions)

Normal Distribution

The normal distribution is a bell-shaped continuous probability distribution function that is parameterized to a mean and standard deviation.

Contents

Normal Distribution

Description

The distribution is bell-shaped and parameterized to the first and second central moments. This has useful consequences for estimating the probability that a given value is in the distribution. For example:

68.27% of the cumulative distribution is within 1 standard deviation of the mean
95.45% within 2
99.73% within 3

A variable distributed this way is notated like X ~ N(μ, σ²).

The probability density function is given (and commonly notated) as:

Correspondingly, the cumulative probability function is commonly notated as Φ(.) rather than F(.). Furthermore the quantile function is commonly notated as Φ^-1(.).

Standard Normal Distribution

The standard normal distribution is specified as Z ~ N(0, 1).

Compare to the calculation for Z scores that will be compared to the standard normal distribution: Z = (x - μ)/σ.

The probability density function simplifies to:

This is graphed as:

Moments

The expected value is given as E[X] = μ.

Variance is given as Var[X] = σ².

Estimation

The MLE for μ is X̅. This is derived as:

Now applying log properties:

And now pulling all constants out of summations:

The critical points of this equation are found by deriving with respect to μ. The first part of the RHS is a constant with respect to μ to becomes 0.

Multiply both sides by σ² and add nμ to both sides to get:

Finally, recognize that X̅ = (1/n) Σ_i(x_i). Therefore μ_MLE = X̅.

Usage

Transformations and Combinations

Let X ~ N(μ, σ²). Adding a constant to X₁ shifts the mean by that constant, without affecting the variance. Multiplying X₁ by a constant factor multiplies the mean by that factor and multiplies the variance by the square of that factor. Succinctly: aX + b ~ N(aμ + b, a²σ²).

Let X₁ ~ N(μ₁, σ²₁) and X₂ ~ N(μ₂, σ²₂), requiring that they are independent. The sum or difference of the two is also a random variable. In the case of a difference: X₁ - X₂ ~ N(μ₁ - μ₂, σ²₁ + σ²₂). The sum is similarly distributed but centered on μ₁ + μ₂. The important detail to remember is that variances are added regardless of what is done to the means. Also take care to remember that a sample statistic X̅ varies according to σ²_X̅ = σ²_X/n, so the difference of two sample statistics is expressed as X̅₁ - X̅₂ ~ N(μ₁ - μ₂, σ²₁/n₁ + σ²₂/n₂) where n_i is the sample size for sample i.

Normality of Sampling Distributions

For a population parameter whose population distribution is normal with mean μ and variance σ², the sampling distribution of the sample statistic which estimates that parameter is also normal. In particular, X̅ is the sample statistic that estimates μ, and its sampling distribution is normal with mean X̅ = μ and variance σ²_X̅ = σ²_X/n, where n is the sample size.

By the central limit theorem, even when the population distribution in question is not normal, with a sufficiently large sample size the sampling distribution of the sample statistic which estimates that parameter is.

A note on variance of a sampling distribution (i.e., σ²_X̅ not σ²_X). This is a function of both the sample variance σ²_X, which is usually itself estimated using sample variance (i.e., σ²_X = s²_X̅), and the sample size n. Consider a census of the population; there is only one possible sample so the sampling distribution has only a single level with 100% certainty. As the sample size decreases, the number of possible samples grows. The number of levels in the sampling distribution is also expected to grow accordingly.

Confidence Intervals

Given a known population distribution (i.e., X ~ N(μ, σ²) with known μ and σ), a confidence interval can be constructed; 95% of samples will lead to a sample statistic that is within approximately 1.96 standard errors of μ.

For similar reasons, a confidence interval can be constructed around a sample statistic. For example, if a 95% confidence interval is constructed around X̅, it can be said with 95% confidence that the true population mean is within that interval.

In the case that the distribution is known to be normal but the variance is unknown, use the Student's t distribution instead.

Probability Tests

As noted above, Z scores (alt. Z statistics) are test statistics for which likelihood can be calculated using the standard normal distribution.

This is combined with the intuition of confidence intervals above. A null hypothesis is formulated in terms of a sample statistic and a confidence interval is constructed around it. The level of confidence should be based on tolerance for error.

As an example, for a two-tailed test and a significance level of 5%, the critical Z score value is 1.96. This score is generally notated as Z_α/2.

CategoryRicottone

Analysis/NormalDistribution (last edited 2026-08-04 17:57:56 by DominicRicottone)

-  ⇤ ← Revision 3 as of 2025-11-03 01:52:44 → 
  Size: 1669
  Editor: DominicRicottone
  Comment: Link
+   ← Revision 12 as of 2026-08-04 17:57:56 → ⇥
  Size: 6085
  Editor: DominicRicottone
  Comment: Notation
-Deletions are marked like this.
+Additions are marked like this.
 Line 13:
-The distribution is bell-shaped and parameterized to the [[Statistics/Moments|first and second moments]]. This has useful consequences for estimating the probability that a given value is in the distribution. For example:
+The distribution is bell-shaped and parameterized to the [[Analysis/Moment|first and second central moments]]. This has useful consequences for estimating the probability that a given value is in the distribution. For example:
 Line 19:
-A variable distributed this way is notated (especially in [[Statistics/EconometricsNotation|econometrics]]) like X ~ N(μ, σ^2^).
+A variable distributed this way is notated like ''X ~ N(μ, σ^2^)''.
 Line 21:
-When the mean is 0 and the [[Statistics/Variance|variance]] is 1, the p.d.f. is specifically referred to as the '''standard normal distribution'''. This is defined as {{attachment:stdnorm.svg}}.
+The probability density function is given (and commonly notated) as:

{{attachment:norm.svg}}

Correspondingly, the cumulative probability function is commonly notated as ''Φ(.)'' rather than ''F(.)''. Furthermore the quantile function is commonly notated as ''Φ^-1^(.)''.



=== Standard Normal Distribution ===

The '''standard normal distribution''' is specified as ''Z ~ N(0, 1)''.

Compare to the calculation for '''Z scores''' that will be compared to the standard normal distribution: ''Z = (x - μ)/σ''.

The probability density function simplifies to:

{{attachment:stdnorm.svg}}.

This is graphed as:
-Line 24:
+Line 42:
-More generally, the p.d.f. is given by ''f(x) = (1/σ) * φ(x-μ/σ)''.

The c.d.f. for the standard normal distribution is notated as ''Φ(.)'', while the c.d.f. for the generic normal distribution is sometimes notated as ''F(.)''.
-Line 35:
+Line 49:
-The [[Statistics/Moments|first and second moments]] are intrinsic to the distribution definition.
+The [[Analysis/ExpectedValue|expected value]] is given as ''E[X] = μ''.

[[Analysis/Variance|Variance]] is given as ''Var[X] = σ^2^''.

----



== Estimation ==

The [[Statistics/MaximumLikelihood|MLE]] for ''μ'' is ''X̅''. This is derived as:

{{attachment:mu1.svg}}

{{attachment:mu2.svg}}

Now applying log properties:

{{attachment:mu3.svg}}

And now pulling all constants out of summations:

{{attachment:mu4.svg}}

The critical points of this equation are found by deriving with respect to ''μ''. The first part of the RHS is a constant with respect to ''μ'' to becomes 0.

{{attachment:mu5.svg}}

{{attachment:mu6.svg}}

Multiply both sides by ''σ^2^'' and add ''nμ'' to both sides to get:

{{attachment:mu7.svg}}

Finally, recognize that ''X̅ = (1/n) Σ,,i,,(x,,i,,)''. Therefore ''μ,,MLE,, = X̅''.
-Line 45:
+Line 93:
+=== Transformations and Combinations ===

Let ''X ~ N(μ, σ^2^)''. Adding a constant to ''X,,1,,'' shifts the mean by that constant, without affecting the variance. Multiplying ''X,,1,,'' by a constant factor multiplies the mean by that factor and multiplies the variance by the square of that factor. Succinctly: ''aX + b ~ N(aμ + b, a^2^σ^2^)''. 

Let ''X,,1,, ~ N(μ,,1,,``, σ^2^,,1,,)'' and ''X,,2,, ~ N(μ,,2,,``, σ^2^,,2,,)'', requiring that they are independent. The sum or difference of the two is also a random variable. In the case of a difference: ''X,,1,, - X,,2,, ~ N(μ,,1,, - μ,,2,,``, σ^2^,,1,, + σ^2^,,2,,)''. The sum is similarly distributed but centered on ''μ,,1,, + μ,,2,,''. The important detail to remember is that variances are added regardless of what is done to the means. Also take care to remember that a sample statistic ''X̅'' varies according to ''σ^2^,,X̅,, = σ^2^,,X,,/n'', so the difference of two sample statistics is expressed as ''X̅,,1,, - X̅,,2,, ~ N(μ,,1,, - μ,,2,,``, σ^2^,,1,,/n,,1,, + σ^2^,,2,,/n,,2,,)'' where ''n,,i,,'' is the sample size for sample ''i''.



=== Normality of Sampling Distributions ===

For a population parameter whose population distribution is normal with mean ''μ'' and variance ''σ^2^'', the sampling distribution of the sample statistic which estimates that parameter is also normal. In particular, ''X̅'' is the sample statistic that estimates ''μ'', and its sampling distribution is normal with mean ''X̅ = μ'' and variance ''σ^2^,,X̅,, = σ^2^,,X,,/n'', where ''n'' is the sample size.

By the central limit theorem, even when the population distribution in question is '''not''' normal, with a sufficiently large sample size the sampling distribution of the sample statistic which estimates that parameter '''is'''.

A note on variance of a sampling distribution (i.e., ''σ^2^,,X̅,,'' not ''σ^2^,,X,,''). This is a function of both the sample variance ''σ^2^,,X,,'', which is usually itself estimated using sample variance (i.e., ''σ^2^,,X,, = s^2^,,X̅,,''), and the sample size ''n''. Consider a census of the population; there is only one possible sample so the sampling distribution has only a single level with 100% certainty. As the sample size decreases, the number of possible samples grows. The number of levels in the sampling distribution is also expected to grow accordingly.



=== Confidence Intervals ===

Given a known population distribution (i.e., ''X ~ N(μ, σ^2^)'' with known ''μ'' and ''σ''), a confidence interval can be constructed; 95% of samples will lead to a sample statistic that is within approximately 1.96 [[Statistics/SamplingDistribution#Standard_Error|standard errors]] of ''μ''.

For similar reasons, a confidence interval can be constructed around a sample statistic. For example, if a 95% confidence interval is constructed around ''X̅'', it can be said with 95% confidence that the true population mean is within that interval.

In the case that the distribution is known to be normal but the variance is unknown, use the [[Analysis/StudentsTDistribution|Student's t distribution]] instead.
-Line 47:
+Line 123:
-The standard normal distribution is referenced for '''Z scores''' (alternatively called '''Z statistics'''). As an example, for a two-tailed test and a [[Statistics/TestStatistic|significance level]] of 5%, the critical Z score value is 1.96.
+As noted above, '''Z scores''' (alt. ''Z statistics'') are test statistics for which likelihood can be calculated using the standard normal distribution.

This is combined with the intuition of confidence intervals above. A null hypothesis is formulated in terms of a sample statistic and a confidence interval is constructed around it. The level of confidence should be based on tolerance for error.

As an example, for a two-tailed test and a [[Statistics/TestStatistic|significance level]] of 5%, the critical Z score value is 1.96. This score is generally notated as ''Z,,α/2,,''.

Diff for "Analysis/NormalDistribution"