You believe something is unlikely. New evidence arrives. How much should you change your mind? Bayes' theorem is the exact answer — and it's why a 99%-accurate test can still be wrong most of the time.
Bayes' theorem updates a probability when new evidence arrives. It flips P(evidence | hypothesis) — which you usually know — into P(hypothesis | evidence) — which you usually want.
Spam filters, medical diagnosis, machine learning, search algorithms, A/B testing, scientific inference — Bayesian updating is the mathematics of learning from data.
Disease affects 1%. Test: 95% sensitive, 90% specific (10% false positive). You test positive. Chance you have it?
P(H) = prior belief. P(E|H) = likelihood of the evidence if H is true. P(E) = total probability of the evidence. P(H|E) = updated (posterior) belief.
Two coins: one fair, one double-headed. You pick one at random, flip it, get heads. P(it's the double-headed coin)?
Why does a 99% accurate rare-disease test mislead?
Because false positives from the huge healthy population outnumber true positives from the tiny sick one. Bayes makes this precise: posterior ∝ likelihood × prior, and a tiny prior keeps the posterior small even with a strong likelihood.
Ignoring the base rate (the prior) is the cardinal Bayesian sin — the 'base rate fallacy'. Test accuracy alone doesn't tell you P(disease | positive); you must factor in how common the disease is.
Reverend Thomas Bayes' note was published posthumously in 1763. Two and a half centuries later it powers spam filters, search engines, and the 'Bayesian' half of modern statistics.
- Bayes flips P(E|H) into P(H|E) — evidence into updated belief.
- Posterior ∝ likelihood × prior — never ignore the prior (base rate).
- A strong test + a rare condition can still yield a low posterior.