Chapter 2: The Shape of Scatter
Ground floor
You have ten numbers now, from the stopwatch. Suppose someone demands a single number instead. Which one, and why that one?
The reflex answer is “the average,” and the reflex is so fast that almost nobody asks where it came from. So ask. Why add them all up and divide? Who decided?
Here is the honest derivation, and it is the kind of thing that should have been said on the first day of every statistics class ever taught. Pick any single number, call it your guess g. It will be wrong about each of the ten, by some distance. Now decide how you want to be punished for being wrong, because until you decide that, the question has no answer. If you declare that being wrong hurts in proportion to the square of the distance, then, of all possible guesses, exactly one minimizes your total pain, and it is the arithmetic mean. If you declare that being wrong hurts in proportion to the plain distance, a different number wins: the median. If you declare that only exact hits count and everything else hurts equally, the winner is the most common value, the mode.
So the mean is not a convention and it is not “the” average. It is the answer to a question, and the question is how do you wish to be punished? Three punishments, three “averages,” all of them correct, none of them interchangeable. Every argument you have ever seen about whether to report the mean or the median is actually an argument about the loss function, conducted by people who do not know that is what they are arguing about.
That is the shape of this chapter: everything in statistics that looks like an arbitrary recipe is the answer to a question somebody forgot to write down. We are going to write the questions down.
The stairs
A distribution is a tally. Nothing more mystical than that. Line up all the values a thing can take, and record how often each one happens, or, for continuous things like milliseconds, how densely they crowd near each place. That tally, normalized so the whole thing sums to one, is a probability distribution. It is not a curve that reality obeys. It is a bookkeeping of a pile.
The center and the spread. From the ground floor, the mean, written with an overbar or the Greek μ, is the balance point under squared loss. Now we need the second number, and it should be built from the same material. The deviations, each value minus the mean, are the raw stuff of spread, but they cannot simply be added: by the mean’s own construction, they sum to exactly zero, every time, which is a beautiful fact and a useless statistic. Squaring them fixes the cancellation. The average of the squared deviations is the variance, σ². Its square root is the standard deviation, σ, which has the virtue of being back in the original units: milliseconds, not milliseconds-squared.
And now the question a logician immediately asks, and which is almost never answered: why squares, rather than just taking absolute values, which would also fix the cancellation? The absolute deviation is a perfectly sensible measure of spread, more robust than the variance, and genuinely used wherever robustness is the point. Yet variance is the currency the whole field settled on. Why?
Because of one structural fact that makes the whole edifice stand up: variances add, and absolute deviations do not. If you have two independent sources of wobble and you stack them, the variance of the total is the sum of the two variances. Exactly. No correction terms, no conditions beyond independence. Nothing else in the neighborhood behaves that way. That single additive property is why variance is the currency of the field: it means error can be decomposed, this much from the instrument, this much from the day, this much from the person, and the parts sum to the whole. Every analysis-of-variance table, every reliability coefficient, every mixed model you will ever meet is that one property being spent.
Notice what standard deviations do not do: they do not add. Stack two independent wobbles of 3 and 4, and the total wobble is not 7 but 5, because 9 plus 16 is 25. Errors combine in right triangles. That is not a metaphor; it is the same theorem.
Now the square root of n, derived rather than asserted. In chapter one I told you the uncertainty of a mean shrinks like one over the square root of n. Here is why, and it costs two lines. Take n independent measurements, each with variance σ². Add them up: variances add, so the sum has variance nσ². Divide by n to get the mean: dividing by a constant divides the variance by that constant squared, so the mean has variance nσ²/n² = σ²/n. Take the square root, and the standard deviation of the mean is σ/√n. That is the whole derivation. The famous law that repetition buys precision, and buys it at a punishing exchange rate, falls directly out of “variances add,” and out of nothing else.
Sit with the exchange rate, because it prices most research and most products. To halve your uncertainty you need four times the data. To cut it by ten, a hundred times. This is why underpowered studies are not slightly wrong but hopeless, why a “quick pilot” answers nothing, and why the phrase “we’ll just gather more data” is a budget statement, not a plan.
Why the bell keeps showing up. Here is one of the deepest facts in mathematics, and it is usually taught as a shrug. Take almost any source of randomness you like, dice, coin flips, the lumpy uneven distribution of how long a synapse dawdles, it does not matter, as long as the pieces are independent and none dominates. Add many of them together. The sum’s distribution converges to the bell, the normal distribution, regardless of the shape of the pieces. That is the central limit theorem, and its content is startling once you hear it correctly: the bell is not a law of nature. It is a law of addition. It shows up wherever a quantity is a sum of many small independent contributions, which is to say, in heights and in measurement errors and in almost anything built by accumulation.
And now the corollary, which is where the theorem earns its keep and where most practitioners fall in: for a quantity that is not a sum of many roughly independent contributions, the theorem gives you no reason to expect a bell, and often there is none. Reaction times are the standing example. The best models of them treat the deliberative part not as a sum but as a first-passage time, the wait until accumulating evidence crosses a threshold, and first-passage times are skewed: a hard floor on the left (nerves take what they take), a long tail on the right (attention wandered). Summarizing them with a mean lets three lapses own the session. This is not pedantry; it is the reason a later chapter of this book exists, and it is why any instrument that reports a mean reaction time is reporting mostly the user’s worst three moments.
Standardizing, and the z that unlocks the next chapter. If a distribution is bell-shaped, it is fully described by two numbers, its mean and its standard deviation, and every such bell is the same bell in different clothes. So strip the clothes: subtract the mean, divide by the standard deviation, and every value becomes a z-score, its distance from the center measured in standard deviations. A z of 0 is dead center; 1 is one deviation above; −2 is two below. About 68 percent of a bell lies within one deviation of the mean, 95 percent within two. Memorize those two numbers; they are the field’s rulers.
The move runs backwards too, and this is the one that matters shortly. Hand me a proportion, say “84 percent of values are below this point,” and I can hand you back the z-score that sits at that boundary: about 1.0. That inverse function, from proportion to z, is what turns a hit rate into a distance. Hold it for the bell tower.
Correlation, and the two lies it tells. Take two variables measured on the same things. The covariance is the average product of their deviations: positive when they tend to be high together, negative when one’s high is the other’s low. Covariance has ugly units (milliseconds times points), so divide it by both standard deviations and it becomes correlation, r, penned into the range −1 to 1, unitless, comparable across everything.
Two things r does not mean. First, it does not mean cause; a whole chapter later in this book is about the difference, and it is a bigger difference than the slogan suggests. Second, and this one bites daily, r is not a percentage of anything. An r of 0.5 is not “half related.” The quantity with an interpretation is r squared: the share of one variable’s variance that is accounted for by the other, which for r = 0.5 is 0.25, a quarter. Correlations that sound impressive are usually explaining less than you think. An r of 0.3, common and celebrated in psychology, accounts for nine percent of the variance and leaves ninety-one on the table.
The difference that travels. Finally, the practical sentence. When comparing two groups, do not say “A scored higher than B.” Say how many standard deviations higher: take the difference of the means and divide by the standard deviation. That ratio is Cohen’s d, and it is the version of the finding that survives translation between scales, labs, and instruments. Rough calibration for minds: d = 0.2 is barely detectable in a single life, 0.5 is visible, 0.8 is obvious to a stranger.
Which sets up the distinction that separates people who understand statistics from people who have merely passed a course in it. “Statistically significant” does not mean the effect is probably real, and it is not the probability that the effect is nonzero; no such number is on offer from this machinery at all. It means exactly this: if the true effect were precisely zero, data this extreme would be rare. A statement about the data under an assumption, never a statement about the assumption, and quietly inverting the two is perhaps the oldest and best-defended error in the applied literature; the reverend, four chapters from here, will show you why the inversion cannot be done without paying his toll. And with enough data, every effect on earth clears the bar, because nothing is ever exactly zero. Significance is, in practice, largely a claim about your n. Effect size is a claim about the world. When you read a result, the effect size is the finding and the p-value is the receipt.
The locked door
Behind the door: the central limit theorem’s actual proof, and its fine print, the conditions under which it fails, which is where the wild distributions live: heavy tails, infinite variance, Lévy flights, and Taleb’s whole argument. Also the exponential family and why a handful of distributions keep recurring for reasons of pure information geometry rather than coincidence; robust statistics, where the median’s virtues get formalized as a breakdown point; and the deep version of the loss-function insight from the ground floor, decision theory, in which every estimator is the answer to a stated question about pain, and the arguments finally become explicit.