Chapter 6: The Reverend’s Ledger
Ground floor
Two jars on a shelf. The left jar holds mostly red marbles with a few blue; the right jar, mostly blue with a few red. Someone picks a jar behind your back, draws one marble, and shows you: red. Which jar?
The child says “the red jar” and is mostly right. Now change one thing: on the shelf stood a hundred of the mostly-blue jars and only one mostly-red. Now a red marble barely helps; it was probably an unlucky draw from one of the ninety-nine blue jars, because there were so many more chances for that to happen. The evidence did not change. What was on the shelf changed, and the answer moved.
That is the whole of it: an honest belief is evidence combined with what was on the shelf beforehand. Everything else in this chapter is bookkeeping, and the bookkeeping was worked out by an eighteenth-century English reverend named Thomas Bayes, published only after his death, as if he suspected what he was starting.
The stairs
For once, the famous formula deserves to be derived rather than displayed, because the derivation is two lines and it settles the question of where the rule gets its authority. The probability that two things are both true can be split in either order: the probability of A-and-B equals the probability of B times the probability of A given B, and it equally equals the probability of A times the probability of B given A. Those are the same quantity. Set them equal, divide, and you have Bayes’ theorem: P(A given B) = P(B given A) × P(A) / P(B). That is the entire proof. The rule is not a school of thought or a philosophical option; it is what the word “given” means, unpacked. Anyone reasoning about evidence either obeys it or is inconsistent with their own premises somewhere.
The working form, though, the one to carry in your pocket, is odds. Take two rival hypotheses and divide everything through, and the theorem becomes: posterior odds = prior odds × likelihood ratio. The prior odds are the shelf: how the two hypotheses stood before this piece of evidence. The likelihood ratio is the strength of the evidence itself: how much more probable this observation is if hypothesis A is true than if hypothesis B is true. Multiply. Done. New evidence tomorrow: multiply again. Belief is a running product, and the ledger metaphor is exact.
Feel the machine run on the classic ambush, which you should be able to do on a napkin for the rest of your life. A screening test for a rare condition is 99 percent sensitive (it catches 99 of 100 true cases) and 99 percent specific (it falsely alarms on only 1 of 100 healthy people). The condition afflicts one person in a thousand. A random person tests positive. What is the chance they have it?
Nearly everyone, including, when this has been put to them, a large fraction of doctors, says something near 99 percent. Now do it in counts, in whole people, and watch the fog lift. Take a thousand people. About one has the condition, and the test almost certainly catches him: one true alarm. Of the 999 healthy, the test falsely alarms on one percent: about ten false alarms. Eleven alarms ring; one is a wolf. The answer is one in eleven, roughly nine percent. Not ninety-nine. The intuition failed because it stared at the test’s accuracy and never looked at the shelf, and the failure has a name, base-rate neglect, and a repair with a name too: natural frequencies, counts of people instead of percentages, which is what we just did. That one translation trick, a career’s worth of research by Gigerenzer showed, cures most professional innumeracy on contact.
Notice, before moving on, that you have seen this machinery before, wearing feathers. The bell tower’s hit rate and false alarm rate are likelihoods. The boy’s criterion is a prior wearing a costume, with the village’s costs stitched into the lining. And eleven-alarms-one-wolf is precisely why a detector’s positives mean almost nothing when wolves are rare, no matter how good its eyes are. Strand One and Strand Two are the same strand viewed from the world’s side and from the believer’s side.
Now the discipline the formula enforces, which matters more than the formula. The likelihood ratio asks: how differently probable is this evidence under the rival hypotheses? If the answer is “about the same,” the evidence teaches nothing, however dramatic it feels. Run it on a wound every maker knows. A builder releases a project into a busy forum where hundreds of releases per day sink unread, survival hanging on which random handful of eyes passes in the first hour. The release sinks: zero response. Hypothesis A, the work is not good enough. Hypothesis B, the work is fine and the channel never sampled it. Ask the only question the ledger permits: how probable is a zero under A, and how probable under B? On such a channel the two numbers are close, precisely how close is an empirical question about the channel rather than a fact this page can hand you, but the ratio sits far nearer to one than the feeling of verdict suggests. The posterior barely moves. The zero that felt like a verdict was, arithmetically, almost no evidence at all, and both despair and defiant pride are overreactions to a likelihood ratio of one. What the silence actually measured is a question for the channel chapter, three doors down.
The constructive corollary is the most useful sentence in this strand: before you look at an outcome, write down what it would look like under each hypothesis. If the answers match, the test was never a test; design one where the hypotheses disagree loudly about what you will see. That habit, priors and likelihoods stated before the reveal, is the entire operational content of the word “rigor,” and it costs one paragraph in a notebook.
The locked door
Behind the door: where priors come from when nobody hands you a shelf, the objective-versus-subjective wars, reference priors, maximum entropy; conjugate families, where the running product has a closed form; Bayesian model comparison and its own pathologies; and E. T. Jaynes’ Probability Theory: The Logic of Science, the imperious posthumous book arguing that probability theory is not about randomness at all but is the unique consistent extension of deductive logic to uncertain propositions, Cox’s theorem being the proof. For a reader who arrived at statistics through logic rather than through the course catalog, that book is not an option. It is the homeland.