= Probability Generating Functions =

When a discrete random variable's probability mass function is used as a [[Analysis/GeneratingFunctions|generating function]], it is known as a '''probability generating function'''.

<<TableOfContents>>

----



== Introduction ==

A [[Analysis/GeneratingFunctions|generating function]] encodes information into a sequence that is paired with a [[Analysis/Series#Power_Series|power series]]. The series is then used to decompose and simplify expressions making use of the encoded information.

Generically this function is notated ''G(z)'' where ''z'' is a dummy variable.

{{attachment:prob0.svg}}

In the general form of the [[Analysis/ZTransform|z transform]] it is treated as a complex valued constant, but in this application it will always be ''1''.

Note that this is effectively a calculation for the [[Analysis/ExpectedValue|expected value]] of a transformed random variable ''z^X^''.

----



== Description ==

For a given random variable ''X'', the probability generating function may be notated ''G,,X,,''. Consider a ''X ~ Bernoulli(p)'':

{{attachment:prob1.svg}}

{{attachment:prob2.svg}}

This is not likely an interesting application. But now consider the sum of ''n'' independent trials as described by ''X''. Convolutions become products in the transformed domain, such that ''Y ~ Binomial(p, n)'' is:

{{attachment:prob3.svg}}



=== Properties ===

For any discrete random variable, the series of this function evaluated at ''z=1'' sums to 1. This should be obvious in the Bernoulli and binomial distribution examples above. More generally:

{{attachment:prob4.svg}}

The most important application of a probability generating function is the extraction of [[Analysis/Moment|moments]]. Consider the following:

''G,,X,,(z) = Prob(X=0) + Prob(X=1)z + Prob(X=2)z^2^ + Prob(X=3)z^3^ + ...''

''G',,X,,(z) = (0) + Prob(X=1) + Prob(X=2)z + Prob(X=3)z^2^ + ...''

''G'``',,X,,(z) = (0) + (0) + Prob(X=2) + Prob(X=3) + ...''

...and so on. While ''G(z) = E[z^X^]'' as stated above, a much more meaningful note is that when ''z=1'', ''G'(z) = E[X]''. And less obviously, ''G'``'(z) = E[X(X-1)]''. Generally, the ''n''th derivative of ''G,,X,,(z)'' evaluated at ''z=1'' gives ''E[X(X-1)(X-2)...(X-n+1)]''.

Note that:

{{attachment:fact.svg}}

Therefore we say that the ''n''th derivative of ''G,,X,,(z)'' evaluated at ''z=1'' gives the ''n''th '''falling factorial moment''' of ''X''. A series of [[Analysis/Factorials#Falling_Factorials|falling factorial]] moments can be constructed that will sum to the ordinary moment. For example, the second raw moment is ''E(X^2^] = G'``',,X,,(1) + G',,X,,(1)''. It follows that variance is ''G'``',,X,,(1) + G',,X,,(1) - (G',,X,,(1))^2^''.

Let ''Z'' be the sum of two independent random variables. Then ''E[z^Z^] = E[z^XY^] = E[z^X^] E[z^Y^]'', so ''E[Z] = G,,Z,,'(1) = G,,X,,'(1) G,,Y,,'(1)''.

Let ''Z'' take the value of ''X'' with probability ''p'' and the value of ''Y'' with probability ''1-p''. Then ''E[z^Z^] = p E[z^X^] + (1-p) E[z^Y^]'', so ''E[Z] = p G,,X,,'(1) + (1-p) G,,Y,,'(1)''.

Let ''Z'' be the sum of ''N'' independent trials distributed identically as ''X'', where ''N'' is itself a random variable. If ''N = n'' and then

{{attachment:prob5.svg}}

Allowing ''N'' to vary is equivalent to the series ''Prob(N=0) E[z^X^]^0^ + Prob(N=1) E[z^X^]^1^ + Prob(N=2) E[z^X^]^2^ + ...''. By effectively a u substitution: ''E[z^X^] = G,,N,,(G,,X,,(z))'', so ''E[Z] = G,,N,,'( G,,X,,'(1))''.

The joint distribution of two independent random variables is ordinarily a convolution, but in the transformed domain it is a simple product: ''E[Z] = E[z^XY^] = E[z^X^] E[z^Y^]''.

Any z transform of a function has a unique inverse that transforms back into the original function. This is to say:
 * The probability mass function can be recovered from the probability generating function.
 * A distribution is uniquely identified by its probability generating function.



----
CategoryRicottone