Exam PProbability foundationsFree to read

Bayes' theorem and the law of total probability

Reverse a conditional probability: weight each cause by its prior and its likelihood, then normalise. The denominator is always the law of total probability.

The formulas

Law of total probability

$$P(A) = \sum_{k} P(A\mid B_k)P(B_k)$$

Bayes

$$P(B_j\mid A) = \dfrac{P(A\mid B_j)P(B_j)}{\sum_k P(A\mid B_k)P(B_k)}$$

Two-event form

$$P(B\mid A) = \dfrac{P(A\mid B)P(B)}{P(A\mid B)P(B) + P(A\mid B^{c})P(B^{c})}$$

Odds form

$$\dfrac{P(B_1\mid A)}{P(B_2\mid A)} = \dfrac{P(A\mid B_1)}{P(A\mid B_2)}\cdot\dfrac{P(B_1)}{P(B_2)}$$

Where it comes from

1. The partition $\{B_k\}$ cuts $A$ into disjoint pieces $A \cap B_k$, so $P(A) = \sum_k P(A\cap B_k) = \sum_k P(A\mid B_k)P(B_k)$.
2. Bayes is then just $P(B_j\mid A) = P(A\cap B_j)/P(A)$ with both parts rewritten using the multiplication rule.
3. The odds form drops the shared denominator, which is why it is faster when only a ratio is asked for.

Worked example

This answer is recomputed from the site’s own interest-theory and probability functions every time the test suite runs, so the page and the mathematics cannot drift apart.

Memory hooks

• Numerator is one prior times one likelihood; denominator is the sum of all of them. The posteriors must sum to 1 - check it.
• The denominator IS the law of total probability. Compute it once and reuse it for every class.

Traps

• Forgetting a class in the denominator, so the posteriors do not sum to 1.
• Swapping prior and likelihood - the prior is the class probability, the likelihood is the event probability GIVEN the class.

Related

• Conditional probability and independence
• Probability axioms and set identities

Drill this: the Exam P question bank has original questions on this topic, and today’s free round is open to everyone.