Bayesian Notation

Because the Bayesian approach to probability differs in meaningful ways from classical statistics, slightly different notation is typically used to express the even more precise intention of certain words.

Priors

The prior probability function of a random variable X (PMF or PDF depending on what X represents) is notated p(X|θ) to indicate that it reflects priors about X captured in an uncertainty (θ) term. This is sometimes instead notated as p_θ(X).

The prior uncertainty term (θ) itself is a random variable with a PDF notated as π(θ).

The expected value for an X with an uncertainty term θ is expressed as p(X) = E_π[p_Θ(X)]. Note the capitalized Θ here, which reflects the expected value of the uncertainty term θ. This embedded expectation creates subtle limitations on computation. For example, p(Y|X) is equivalent to E_π[p_Θ(Y|X)], but the latter term cannot be rewritten as E_π[ p_Θ(X,Y) / p_Θ(Y) ]. Instead it should be expanded like:

Posteriors

Uncertainty is updated given X; this is notated with the probability function p(θ|X). Here X is the observed data, which typically is a vector, rather than a random variable. To differentiate the meaning, sometimes the function is notated p(θ|D).

The posterior uncertainty probability function is now notated as π|X(θ) (or π|D(θ)).

CategoryRicottone