Chapter 9: The Channel
Ground floor
A child on one hill signals a friend on another with a lantern. One flash, wolf; two flashes, dinner. Between the hills: fog, distance, a wobbling hand. Some evenings the friend reads dinner as wolf. The child learns, without any theory, the three facts every engineer eventually learns. Surprise is the substance: if the lantern always flashed once regardless of events, the flashes would carry nothing, because a message that cannot be otherwise tells you nothing. The fog has a budget: past some rate of flashing, errors eat everything, and no cleverness of wrist beats the fog’s arithmetic. Repetition buys truth: flash each signal three times and take the majority, and dinner arrives reliably, at the price of patience.
In 1948, a young engineer named Claude Shannon turned those three childhood facts into exact mathematics in a single paper, and it is fair to say the century downstream of that paper is the one you are living in. It is also, quietly, one of the most readable founding documents in all of science, which almost nobody who quotes it has checked.
The stairs
Information is measured in bits, and a bit is one halving of ignorance. A fair coin’s outcome carries one bit; a roll of a die about 2.6; the answer to a question you already knew carries zero, however long it takes to say. The average surprise of a source, weighted by how often it says each thing, is its entropy, and entropy is a hard floor: no scheme, however ingenious, can compress the source’s messages below it on average. Compression is the removal of everything the receiver could have predicted, which is why compressed data looks like noise: whatever still has pattern in it was not finished being compressed.
A channel, in turn, has a capacity: the ceiling, in bits per second at a given noise level, on what can pass through it. And Shannon’s noisy-channel theorem, the genuinely shocking result, says the ceiling is sharp on both sides: below capacity, communication can be made as error-free as you like, purchasable with clever redundancy, error-correcting codes, majority votes, structure; above capacity, no scheme survives, purchasable with nothing. Hard floor, hard ceiling, and the whole of communications engineering lives in the gap, spending redundancy to buy reliability at Shannon’s posted exchange rate. Notice this is the third time this book has said the same sentence in different clothes: repetition is the only substance reliability has ever been made of. It was true of measurements, it was true of majority-flashing lanterns, and it will be true again, two chapters from now, of something much stranger.
One more Shannon fact, easy to state and heavy to hold: the theory is deliberately, proudly silent about meaning. It prices the transmission of distinguishable symbols and says nothing about what they signify; meaning is imposed at the endpoints, by agreement between sender and receiver. A channel can be flawless and understand nothing. Set that sentence somewhere safe. The third strand will come back for it.
Now the turn this chapter exists for, and it begins at a party. You are in a room of thirty conversations, a wall of acoustic noise, and someone across the room says your name, not loudly, and it cuts through everything. Nothing else in that wall of sound reached you, but that one waveform did, because somewhere in your auditory machinery a template of your own name is running continuously against the incoming stream, and whatever correlates with the template gets amplified into awareness while everything else, however loud, is treated as fog. Engineers build the same device on purpose and call it a matched filter: a receiver that correlates the incoming signal against a known template and rejects all else. It is provably the best possible detector of a known shape in Gaussian noise, and a very good one in most noise that is not. Every mind runs banks of them. So does every audience.
Attention is a channel, with everything the word implies: a capacity, brutally small, seconds wide at first contact; a noise floor, set by everything else competing at that moment; and a matched filter at the receiving end, tuned by the audience’s history to a repertoire of shapes it knows how to want. A message posted into a busy forum, a paper submitted to a flooded field, an introduction made at that same loud party, all of these are transmissions into a channel whose filter was tuned long before you arrived. And this reframes the fate that every maker knows and most misread: the release that sinks with zero response. A zero cannot, by itself, tell you which of two things happened: the audience sampled the payload and passed, or the transmission never rose above the correlation threshold and was never demodulated at all, wolf lost in wind. On a channel this noisy the second is at least as plausible as the first, and the two failures call for opposite repairs, which is exactly why the distinction is worth money: judging the payload from silence fixes the wrong stage. So the working rule is the radio operator’s rather than the critic’s: instrument the channel before treating silence as a verdict. Learn what silence usually means on this channel, at this hour, for transmissions of this shape, and only then decide what your zero was evidence of. The reverend’s chapter already priced how little an uninstrumented zero proves; this chapter adds where the missing information lives: in the channel’s own statistics, which can be gathered, unlike the audience’s unspoken verdict, which cannot.
The consequences are engineering, not psychology, and they are the same moves a radio operator makes from a bad propagation report. Learn the filter empirically: study a hundred transmissions that got through this channel, and extract the waveform, the shape and rhythm of the opening seconds, not the topics. Encode for the aperture: the first line is the preamble, and if the preamble fails, the payload was never read, so it deserves engineering effort comparable to the payload itself. Respect capacity: a complex thing cannot pass through a seconds-wide aperture, so what goes through first is a hook whose only job is to widen the channel for the second stage. And add redundancy across time and channels, because one transmission is one draw from a noisy channel, and Strand One already told you, at length, what one draw is worth. None of this is pandering, any more than impedance matching is pandering. The message that dies in the fog serves no one, however true it was.
The locked door
Behind the door: the actual proof of the noisy-channel theorem, which conjures the existence of perfect codes out of purely random ones without ever exhibiting a single example, one of the great nonconstructive audacities in mathematics; rate-distortion theory, the price of lossy honesty; Kolmogorov complexity, where the information in one object, rather than a source, is defined as the length of its shortest description, and randomness finally gets a definition; and Landauer’s principle, the receipt proving information is physical: erasing one bit costs a minimum quantum of heat, no matter the technology, forever. Reading: Shannon’s 1948 paper itself, then Pierce’s An Introduction to Information Theory: Symbols, Signals and Noise for the guided tour, in that order or the reverse, both work.