Exam PApplicationsFree to read

Mixtures and the law of total probability in practice

A population made of sub-populations is a mixture: probabilities average with the mixing weights, but variances do not.

The formulas

Mixed density

$$f(x) = \sum_k w_k f_k(x), \qquad \sum_k w_k = 1$$

Mixed mean

$$E[X] = \sum_k w_k \mu_k$$

Mixed second moment

$$E[X^{2}] = \sum_k w_k\left(\sigma_k^{2} + \mu_k^{2}\right)$$

Variance

$$\operatorname{Var}(X) = \sum_k w_k\left(\sigma_k^{2}+\mu_k^{2}\right) - \left(\sum_k w_k\mu_k\right)^{2}$$

Where it comes from

1. A mixture is a two-stage experiment: pick the sub-population, then draw from it - which is exactly the setting of the law of total probability.
2. Expectation is linear, so means mix linearly; the SQUARE of the mean does not, which is why the variance picks up a between-group term.
3. This is the conditional variance decomposition written for a discrete mixing variable.

Worked example

This answer is recomputed from the site’s own interest-theory and probability functions every time the test suite runs, so the page and the mathematics cannot drift apart.

Memory hooks

• Means mix; variances do not. The extra term is the spread of the group means themselves.
• A mixture of exponentials is NOT exponential - its tail is heavier than any of its components.

Traps

• Averaging the component variances and calling that the mixture's variance.
• Mixing densities but forgetting the weights must sum to 1.

Related

• Bayes' theorem and the law of total probability
• Conditional expectation and the variance decomposition
• Expectation, variance and moments

Drill this: the Exam P question bank has original questions on this topic, and today’s free round is open to everyone.