Probability Generating Functions

When a discrete random variable's probability mass function is used as a generating function, it is known as a probability generating function.

Introduction

A generating function encodes information into a sequence that is paired with a power series. The series is then used to decompose and simplify expressions making use of the encoded information.

Generically this function is notated G(z) where z is a dummy variable.

In the general form of the z transform it is treated as a complex valued constant, but in this application it will always be 1.

Note that this is effectively a calculation for the expected value of a transformed random variable z^X.

Description

For a given random variable X, the probability generating function may be notated G_X. Consider a X ~ Bernoulli(p):

This is not likely an interesting application. But now consider the sum of n independent trials as described by X. Convolutions become products in the transformed domain, such that Y ~ Binomial(p, n) is:

Properties

For any discrete random variable, the series of this function evaluated at z=1 sums to 1. This should be obvious in the Bernoulli and binomial distribution examples above. More generally:

The most important application of a probability generating function is the extraction of moments. Consider the following:

G_X(z) = Prob(X=0) + Prob(X=1)z + Prob(X=2)z² + Prob(X=3)z³ + ...

G'_X(z) = (0) + Prob(X=1) + Prob(X=2)z + Prob(X=3)z² + ...

G''_X(z) = (0) + (0) + Prob(X=2) + Prob(X=3) + ...

...and so on. While G(z) = E[z^X] as stated above, a much more meaningful note is that when z=1, G'(z) = E[X]. And less obviously, G''(z) = E[X(X-1)]. Generally, the nth derivative of G_X(z) evaluated at z=1 gives E[X(X-1)(X-2)...(X-n+1)].

Note that:

Therefore we say that the nth derivative of G_X(z) evaluated at z=1 gives the nth falling factorial moment of X. A series of falling factorial moments can be constructed that will sum to the ordinary moment. For example, the second raw moment is E(X²] = G''_X(1) + G'_X(1). It follows that variance is G''_X(1) + G'_X(1) - (G'_X(1))².

Let Z be the sum of two independent random variables. Then E[z^Z] = E[z^XY] = E[z^X] E[z^Y], so E[Z] = G_Z'(1) = G_X'(1) G_Y'(1).

Let Z take the value of X with probability p and the value of Y with probability 1-p. Then E[z^Z] = p E[z^X] + (1-p) E[z^Y], so E[Z] = p G_X'(1) + (1-p) G_Y'(1).

Let Z be the sum of N independent trials distributed identically as X, where N is itself a random variable. If N = n and then

Allowing N to vary is equivalent to the series Prob(N=0) E[z^X]⁰ + Prob(N=1) E[z^X]¹ + Prob(N=2) E[z^X]² + .... By effectively a u substitution: E[z^X] = G_N(G_X(z)), so E[Z] = G_N'( G_X'(1)).

The joint distribution of two independent random variables is ordinarily a convolution, but in the transformed domain it is a simple product: E[Z] = E[z^XY] = E[z^X] E[z^Y].

Any z transform of a function has a unique inverse that transforms back into the original function. This is to say:

• The probability mass function can be recovered from the probability generating function.
• A distribution is uniquely identified by its probability generating function.

CategoryRicottone