Moment generating functions, and what they are for

The definition and the one property that matters

The moment generating function of a random variable is the expected value of the exponential of t times that variable, treated as a function of t. That definition looks arbitrary until you notice what it does: it packs every moment of the distribution into a single function, from which they can be extracted by differentiation.

The property that makes it worth learning is uniqueness. If two random variables have the same moment generating function on an interval around zero, they have the same distribution. That is what licenses the standard exam move of computing an MGF, recognising it, and reading off the answer without any further work.

Every MGF equals one at t equal to zero, because the exponential of zero is one and the expectation of a constant is that constant. A candidate who derives an MGF that does not satisfy that check has made an algebra error and can stop looking for it anywhere else.

Moments by differentiation

Differentiate the MGF k times and evaluate at zero, and you get the kth raw moment. The first derivative at zero is the mean; the second is the expected value of the square, from which the variance follows by subtracting the square of the mean.

Take a variable whose MGF is one minus 2500t, all raised to the power minus four. Differentiating once and setting t to zero gives 10,000, so that is the mean. Differentiating twice gives 125,000,000, so the variance is 125,000,000 minus 10,000 squared, which is 25,000,000 and a standard deviation of 5,000.

In practice you will rarely differentiate on the exam. Recognising the function is faster, and differentiating is the fallback for the case where you do not recognise it - which is exactly the case the technique exists for.

One warning about the domain. An MGF only has to exist in some interval around zero, and for several distributions it does not exist beyond a threshold - the gamma MGF above is undefined once t reaches the rate parameter, and the lognormal has no MGF for any positive t at all. That is why the lognormal moments in the last section came from the normal's MGF rather than the lognormal's own, and it is worth stating rather than discovering mid-question.

Sums of independent variables multiply

If two variables are independent, the MGF of their sum is the product of their MGFs. This follows in one line from the definition, because the exponential of a sum factorises and independence lets the expectation of the product become the product of the expectations.

That single fact is why MGFs appear on the syllabus at all. The sum of independent Poisson variables with rates 3 and 5 has an MGF that is the product of two Poisson MGFs, which is the Poisson MGF with rate 8 - so the sum is Poisson with mean and variance both 8, established without touching a convolution.

The same argument gives the other closure results worth knowing: independent normals sum to a normal, independent gammas with a common rate sum to a gamma, independent binomials with a common success probability sum to a binomial, and a sum of independent exponentials with a common rate is a gamma.

Recognising a distribution from its MGF

Four shapes cover almost every MGF an Exam P question will hand you, and being able to match a function to a family in a few seconds is worth more than any amount of differentiation practice.

The recognition step is the whole technique. A question that gives you an MGF is not asking you to do calculus; it is asking whether you know which distribution you are looking at, and the calculus is only there for candidates who do not.

• An exponential of a linear-plus-quadratic function of t: normal, with the mean and variance read off the two coefficients.
• One minus a constant times t, raised to a negative power: gamma, with the power giving the shape and the constant giving the scale.
• An exponential of a constant times the quantity e to the t minus one: Poisson, with that constant as the rate.
• A constant plus another constant times e to the t, raised to a positive integer power: binomial, with the power as the number of trials.

The normal, the lognormal and the trick that follows

The normal MGF is the exponential of the mean times t plus half the variance times t squared. For a normal with mean 3 and variance 16 that is the exponential of 3t plus 8t squared, and its value at t equal to one is the exponential of 11, or about 59,874.

That evaluation is not idle. If a variable is lognormal, its logarithm is normal, so the mean of the lognormal is the normal MGF evaluated at t equal to one, and the expected value of its square is the normal MGF at t equal to two. For an underlying normal with mean 2 and standard deviation 0.5, the lognormal mean is 8.3729 and the expected square is 90.0171, giving a variance of 19.9117.

This is the cleanest example of what MGFs are actually for. Nothing was integrated, no density was written down, and two moments of an awkward distribution fell out of one function evaluated at two points.

What to memorise

Four MGFs and one rule. The four are the ones in the recognition list above; the rule is that independent variables multiply. Everything else on the topic can be rebuilt from the definition in under a minute, which is exactly the kind of knowledge that survives an exam.

Do not memorise the MGFs of the hypergeometric or the beta. Neither is asked for, and the effort is better spent on the recognition drill, where the return is immediate.