Download the Book (PDF): Introduction In the summer of 2024, Meta published a long technical report on the Llama 3 family of language models. Most readers went straight to the benchmark tables. The more revealing pages were further in, in the sections on infrastructure. The largest model, with 405 billion parameters, had been trained on up to 16,384 Nvidia H100 GPUs for several months. During one 54-day window the job was interrupted 466 times, 419 of them unexpectedly, which works out to roughly one unplanned stoppage every three hours. More than half of those were traced to the GPUs themselves or their high-bandwidth memory. The engineers also noticed that throughput drifted by one to two percent over the course of a day as the data centre warmed and cooled, and that when sixteen thousand GPUs paused together to synchronise and then resumed together, the facility's power draw swung by tens of megawatts in an instant. None of this is about neural networks in the textbook sense. There is no new activation function in those paragraphs, no clever loss. It is about plumbing: how to keep tens of thousands of processors working on one mathematical object, in lockstep, for months, when the processors fail, the cables are slower than the chips, and the whole system is only as fast as its slowest member. That plumbing is the subject of this book. The argument The controlling idea is simple to state and takes the rest of the book to earn. Training a massive model is a communication problem wearing the costume of a computation problem. Every technique in distributed deep learning, from the humblest data-parallel job to the elaborate multi-dimensional layouts used for frontier models, is a way of trading three scarce resources against each other: memory on each device, arithmetic throughput, and the bandwidth of the links between devices. The strategies differ in which of those they spend and which they save. The art lies in matching each strategy to the part of the physical network that can afford it. Put that way, the alphabet soup of the field becomes legible. Data parallelism copies the whole model to every device and pays for it with one large exchange of gradients per step. Sharded data parallelism, known by the names ZeRO and FSDP, stops copying and starts lending, at the price of more frequent exchanges. Tensor parallelism splits individual matrix multiplications across devices and so needs the fastest links available, which is why it almost never leaves a single server or rack. Pipeline parallelism splits the model by layers, passes only thin slices of activations between neighbours, and pays instead in idle time. Expert parallelism and context parallelism are newer answers to newer shapes of model and data. Each one is a bargain, and every bargain has a price paid in bytes moved, bytes stored or cycles wasted. A reader who holds on to that framing can look at any training configuration, however baroque, and ask the right questions. What is being split? What has to cross the wire, how often, and over which link? What is sitting idle while it does? Why this matters now A decade ago these were specialist concerns. A strong research result could be produced on a single workstation, and the largest training runs used a few dozen accelerators. That world has gone. The compute used for frontier training has grown by several orders of magnitude, and no single device has grown remotely as fast. A modern GPU holds 80 to 192 gigabytes of memory; the raw training state of a 400-billion-parameter model, before counting a single activation, runs to several terabytes. Arithmetic is the same story: training Llama 3 405B took about 3.8 × 10^25 floating-point operations, a figure that would occupy a single H100 for many centuries. So the work must be divided, and the moment it is divided, the devices must talk. Talking is slow. Within a GPU, data moves between memory and arithmetic units at several terabytes per second. Between GPUs in the same server it moves at hundreds of gigabytes per second. Between servers it typically moves at tens of gigabytes per second per GPU. Each step outward costs roughly an order of magnitude, and every distributed training technique is an attempt to keep the heavy traffic on the fast inner links and send only light traffic across the slow outer ones. The consequences are economic as much as technical. A cluster of sixteen thousand GPUs costs on the order of a billion dollars to build and a great deal to run. The standard measure of how well it is used, model FLOPs utilisation, compares the useful arithmetic actually achieved with the theoretical peak of the hardware. Published figures for well-engineered large runs sit around 40 to 55 percent. Meta reported 38 to 43 percent for Llama 3 405B depending on configuration; ByteDance reported 55.2 percent for a 175-billion-parameter model on 12,288 GPUs with its MegaScale system. Every percentage point is millions of dollars of hardware either doing useful work or waiting on a cable. What the book covers, and what it leaves out The book proceeds in the order a practitioner meets the problems. Chapter 1 does the arithmetic: how much memory and computation a large model actually needs, and why the answer forces distribution. Chapter 2 covers data parallelism and the all-reduce collective at its heart, including why the ring algorithm made large-scale training practical. Chapter 3 is about sharding the training state across data-parallel workers, the idea behind ZeRO and PyTorch's Fully Sharded Data Parallel. Chapter 4 explains tensor parallelism, the technique popularised by Nvidia's Megatron-LM, along with sequence parallelism and the reason this approach is bound to the fastest interconnect. Chapter 5 treats pipeline parallelism, from GPipe to the schedules that have nearly eliminated idle time. Chapter 6 shows how these strategies are composed into the multi-dimensional layouts used for real frontier runs, and adds the two newer dimensions, context and expert parallelism. Chapter 7 turns to communication itself: the physical network, the collective libraries, and the techniques for hiding, shrinking and avoiding traffic. Chapter 8 deals with the problem that dominates the operational life of a large training job, which is that at this scale something is always broken. Several things are deliberately left out. The book is about training, not inference; serving large models raises its own distributed-systems questions, and they deserve separate treatment. It does not survey optimisation theory, model architecture or data curation except where they bear directly on how work is divided. It does not attempt to document any particular framework's configuration flags, which change faster than any book. The aim is to give the reader the underlying mechanics clearly enough that the documentation of Megatron-LM, DeepSpeed, PyTorch or JAX reads as a set of familiar choices rather than a list of incantations. A note on the reader The book assumes the reader knows what a neural network is, roughly how training by gradient descent works, and that modern language models are built from stacked Transformer layers containing attention and feed-forward blocks. It does not assume prior knowledge of parallel computing. Terms are defined where they first appear, and a short glossary collects the ones that recur. Where numbers are given, they come from published sources, which are listed in the notes. Hardware specifications quoted are the manufacturers' headline figures; achievable bandwidths in real systems are lower, and where the gap matters the text says so. The field moves quickly, and some specific figures will date. The trade-offs they illustrate will not. Memory, arithmetic and bandwidth have been the three sides of this problem since the first multi-GPU training runs, and they will remain so on whatever hardware comes next. Chapter 1: The Arithmetic of Scale Before deciding how to divide a training job, it helps to know exactly what is being divided. Engineers who work on large models carry a handful of numbers in their heads: how many bytes each parameter costs during training, how many floating-point operations each token costs, how much memory the activations of a single layer occupy. These numbers are not exotic. They follow from a few lines of reasoning about how gradient descent works and how a Transformer is built. Once they are in hand, the need for distribution stops being an abstract claim about "big models" and becomes a concrete shortfall, measured in gigabytes and in years. What a parameter really costs A parameter in a trained model is a single number. Stored in the 16-bit bfloat16 format common today, it takes two bytes. A 70-billion-parameter model therefore occupies about 140 gigabytes once training is finished, and a 405-billion-parameter model about 810 gigabytes. Those figures already exceed the memory of a single high-end GPU, but they understate the training problem badly, because training does not store one number per parameter. It stores several. Consider the standard recipe of mixed-precision training with the Adam optimiser, introduced in this form by Micikevicius and colleagues at Nvidia and Baidu in 2018 and still the default starting point. The forward and backward passes run in 16-bit precision for speed, so each parameter has a 16-bit working copy. The backward pass produces a gradient for every parameter, also 16-bit. But updating weights in 16-bit arithmetic loses small updates to rounding, so the optimiser keeps a 32-bit master copy of every weight, and it is this master copy that actually gets updated before being cast down again. Adam, in addition, keeps two running statistics for every parameter: an exponential average of the gradient, called the first moment or momentum, and an exponential average of its square, the second moment or variance. Both are kept in 32-bit precision. Adding these up gives the figure that anchors the whole field. Table 1 sets out the accounting, following the analysis in the 2020 ZeRO paper by Rajbhandari and colleagues at Microsoft. Table 1. Training-state memory per parameter under mixed-precision Adam (after Rajbhandari et al., 2020). Component Precision Bytes per parameter Total for a 70B model Working weights bf16 2 140 GB Gradients bf16 2 140 GB Master weights fp32 4 280 GB Adam first moment fp32 4 280 GB Adam second moment fp32 4 280 GB Total 16 1,120 GB Sixteen bytes per parameter is the number to remember. For a 70-billion-parameter model it means more than a terabyte of state; for a 405-billion-parameter model, about 6.5 terabytes. The largest single GPUs on the market in 2026 carry between 80 and roughly 288 gigabytes of high-bandwidth memory. Even the most generous of them falls short by a factor of four for the smaller model and more than twenty for the larger, and that is before a single activation has been stored or a single working buffer allocated. Notice where the bulk sits. The 16-bit weights and gradients account for only four of the sixteen bytes. Three quarters of the memory belongs to the optimiser: the master weights and the two moments. This observation, obvious once the table is written down, was the seed of the most influential memory-saving technique of the past several years. The optimiser state is only needed once per step, when the update is applied, and nothing requires every device to hold all of it. Chapter 3 is about what follows from that. Variations on the recipe change the details but not the order of magnitude. Some systems keep gradients in 32-bit precision for accuracy, which raises the total to eighteen bytes. Some use optimisers with less state, such as Adafactor, which factorises the second moment into row and column statistics, or keep the moments in 8-bit or 16-bit formats. Training in FP8, discussed in Chapter 7, shrinks the working copies further. But even aggressive schemes leave a large model needing several times its own inference footprint during training, and the conclusion stands: the training state of any model worth discussing here does not fit on one device. Why bigger chips do not solve it A natural objection is that hardware will catch up. Each generation of GPU carries more memory than the last: the Nvidia V100 of 2017 offered 16 or 32 gigabytes, the A100 of 2020 40 or 80, the H100 of 2022 80, the H200 141, and the Blackwell parts of 2025 between 180 and 288. That is roughly a tenfold increase in eight years, which is impressive by most standards of engineering. Over the same period, the largest openly documented language models went from a few hundred million parameters to several hundred billion, an increase of roughly a thousandfold. The gap between what a model needs and what a chip holds has therefore widened, not narrowed, and there is no sign of the trend reversing. Memory capacity on a chip is constrained by the physical area available for stacks of high-bandwidth memory beside the processor and by the cost and yield of those stacks. Model size is constrained mainly by the training budget, which has been growing much faster. A simple calculation makes the point concrete. Holding the training state of Llama 3 405B, 6.5 terabytes at sixteen bytes per parameter, on 80-gigabyte GPUs requires at least 81 of them if every byte is perfectly divided and nothing else is stored. In reality, each device also needs room for activations, communication buffers, temporary workspace for matrix multiplications and memory lost to fragmentation, so the practical minimum is well over a hundred. Even on the largest Blackwell parts the count would be in the dozens. A single training replica of a frontier model is, irreducibly, a multi-server object. The question is never whether to distribute it, only how. The activation problem Parameters and optimiser state are fixed costs: they depend on the size of the model and nothing else. Activations are variable costs, and at long sequence lengths they can be larger than everything in Table 1 combined. During the forward pass, each layer computes outputs from its inputs. To run the backward pass, the network needs many of those intermediate values again, because the gradient of a layer's weights depends on the inputs that layer saw. So the forward pass leaves a trail of saved tensors behind it, layer by layer, and the backward pass consumes them in reverse order. The peak memory use comes just as the forward pass finishes and the backward pass begins, when the whole trail is still in memory. How big is the trail? For a Transformer, the activation memory per layer scales with the number of sequences in the micro-batch, the sequence length and the hidden dimension, plus a term that scales with the square of the sequence length times the number of attention heads, which comes from the attention score matrix. Korthikanti and colleagues at Nvidia worked through this accounting carefully in a 2022 paper on reducing activation memory. For a model with a hidden dimension in the tens of thousands, a sequence of a few thousand tokens and a modest micro-batch, each layer stores on the order of gigabytes, and a model has dozens to over a hundred layers. The quadratic attention term, left untreated, makes long sequences ruinous: doubling the context length quadruples that part of the bill. Two developments have tamed the worst of this. The first is FlashAttention, published by Tri Dao and colleagues in 2022, which computes attention in tiles that fit in the GPU's fast on-chip memory and never materialises the full score matrix. It removes the quadratic memory term without changing the mathematics, and it is now standard. The second is activation recomputation, also called gradient checkpointing, an idea with a long history in automatic differentiation that Chen and colleagues popularised for deep learning in 2016. Rather than saving every intermediate tensor, the forward pass saves only a few checkpoints, typically the input to each Transformer layer, and discards the rest. During the backward pass, each layer's forward computation is rerun from its checkpoint to regenerate the discarded values just before they are needed. Recomputation is the purest example of the trade-offs this book is about. It spends arithmetic to save memory. Full recomputation of every layer costs roughly one extra forward pass, which adds about a third to the total arithmetic of a training step. Selective recomputation, the refinement Korthikanti and colleagues proposed, recomputes only the operations whose outputs are large but cheap to regenerate, chiefly those inside attention, and recovers most of the memory savings for a small fraction of the extra arithmetic. Large training runs almost always use some form of it. Even with both techniques, activation memory for long sequences remains substantial, which is why Chapter 6 introduces context parallelism, a way of splitting a single sequence across devices. For now the point is that memory pressure comes from two directions. The fixed cost of the model's state grows with parameter count. The variable cost of activations grows with batch size and sequence length. Different parallelism strategies relieve different parts of this pressure, and choosing among them begins with knowing which part is binding. Counting the arithmetic Memory is one wall. Time is the other. Suppose, implausibly, that a device existed with enough memory to hold the whole training state. How long would training take? The answer comes from one of the most useful approximations in the field. For a dense Transformer with N parameters, the forward pass costs about 2N floating-point operations per token: each parameter participates in one multiplication and one addition. The backward pass costs about twice the forward, because it must compute gradients with respect to both the activations and the weights. So a training step costs about 6N operations per token, and training on D tokens costs about 6ND operations. Kaplan and colleagues at OpenAI used this approximation in their 2020 work on scaling laws, and it has become the standard way to estimate training compute. It ignores the attention computation, which adds a term proportional to sequence length, and it ignores recomputation, but for large models at moderate sequence lengths it is accurate to within a modest margin. Apply it to Llama 3 405B. With N of 405 billion and D of 15.6 trillion tokens, 6ND comes to about 3.8 × 10^25 operations, which matches the figure Meta reported. An H100 GPU has a peak dense throughput in bfloat16 of roughly a petaflop per second, 10^15 operations. At peak, with perfect efficiency, a single H100 would need about 3.8 × 10^10 seconds, a little over twelve hundred years. At the 40 percent or so efficiency that real large runs achieve, it would need about three thousand years. Spread across 16,384 GPUs, the same work at the same efficiency takes about two months. That is roughly what happened. The arithmetic is unforgiving in both directions: the number of devices needed to finish in a reasonable time follows directly from the compute budget, and the compute budget follows from the model size and the number of training tokens. Since the scaling-law work of Kaplan and colleagues, and especially since the 2022 Chinchilla paper by Hoffmann and colleagues at DeepMind showed that models should be trained on far more tokens than had been common, the token counts have grown as fast as the parameter counts. Llama 3 405B saw about 38 tokens per parameter, well above the roughly 20 that Chinchilla suggested as compute-optimal, because a smaller model trained longer is cheaper to serve. Utilisation: the honest metric The 6ND formula gives the work that must be done. The hardware's peak gives the rate at which it could be done. The ratio between what is achieved and what is possible is the field's central efficiency measure. Model FLOPs utilisation, or MFU, was defined in Google's 2022 PaLM paper by Chowdhery and colleagues. It counts only the arithmetic the model mathematically requires, using a formula like 6N per token plus the attention term, and divides the achieved rate by the hardware's theoretical peak. Crucially, it does not count recomputation as useful work. A related measure, hardware FLOPs utilisation, does count recomputation, and so always reads higher. PaLM, trained on 6,144 TPU v4 chips across two pods, reported an MFU of 46.2 percent. The Llama 3 paper reported 38 to 43 percent for its 405B configurations. ByteDance's MegaScale paper reported 55.2 percent for a 175-billion-parameter model on 12,288 GPUs, a 1.34-fold improvement over its Megatron-LM baseline. Why is the number never close to 100 percent? Some of the loss happens inside the GPU: not every operation is a large matrix multiplication that keeps the tensor cores saturated; normalisation layers, activation functions and the attention softmax are limited by memory bandwidth rather than arithmetic. But at scale, the dominant losses are distributed-systems losses, and they fall into three families. The first is communication that cannot be hidden. Whenever a device must wait for data from another device before it can continue, its arithmetic units sit idle. The techniques in Chapters 2 through 7 are, to a large extent, techniques for overlapping communication with computation, so that data moves while the arithmetic continues. The second is idle time built into the schedule. Pipeline parallelism, the subject of Chapter 5, inherently leaves some devices waiting at the start and end of each step. This waste is called the pipeline bubble. The third is imbalance and variance. A synchronous training step finishes only when the slowest device finishes. If one GPU in sixteen thousand runs slightly hot and throttles its clock, or one network link is congested, every other device waits. At large scale, the tail dominates the average, and Chapter 8 returns to this. The 6ND rule also gives a sense of what "massive" means in a given year. A useful way to read any announcement of a new model is to multiply parameters by tokens by six, divide by the cluster size and the hardware peak, and see whether the implied training time and utilisation are plausible. When the numbers do not add up, either the announcement omits something, or the model is not dense: mixture-of-experts models, discussed in Chapter 6, activate only a fraction of their parameters per token, and for them N in the formula means active parameters rather than total. DeepSeek-V3, for example, has 671 billion parameters in total but activates only 37 billion per token, which is why its reported training cost of about 2.8 million H800 GPU-hours is so much lower than its headline size would suggest. The shape of the problem Put the two walls together and the design space comes into focus. The memory wall says the training state must be split across many devices; for frontier models, across dozens at minimum just to hold the weights and optimiser state, before any thought of speed. The compute wall says the work must be spread across thousands or tens of thousands of devices to finish in months rather than millennia. These are different requirements. Splitting for memory and splitting for speed need not use the same technique, and in practice they rarely do. The simplest approach to the compute wall is to give every device its own share of the data and let them all work on the same model. That is data parallelism, and it scales arithmetic almost perfectly, but on its own it does nothing for memory: every device still holds the full sixteen bytes per parameter. The approaches to the memory wall all involve splitting the model or its state, and each of them introduces communication that data parallelism alone does not need. The rest of this book is a tour of those splits: what each one divides, what it forces across the network, and where in the network it can afford to do so. A final number frames everything that follows. The amount of arithmetic per byte of data that a modern GPU can perform without stalling, its so-called arithmetic intensity threshold, has been rising for a decade, because compute has grown faster than memory bandwidth and much faster than network bandwidth. An H100 can perform several hundred floating-point operations in the time it takes to read one byte from its own memory, and thousands in the time it takes to receive one byte from another server. Every distributed training strategy is ultimately judged by whether it gives each device enough arithmetic to do while the bytes are in flight. Chapter 2: Data Parallelism and the All-Reduce The oldest and still the most widely used way to train on many devices is also the easiest to describe. Give every device a complete copy of the model. Split each batch of training data into equal shares, one per device. Let every device run the forward and backward pass on its own share. Then, before anyone updates their weights, average the gradients across all devices, so that every copy applies exactly the same update and the copies stay identical. That is data parallelism. Every large training run uses it in some form, usually as the outermost layer of a more complicated arrangement, and understanding its single communication step, the all-reduce, is the key to understanding communication in the field generally. Why averaging is enough The mathematics that makes data parallelism correct is pleasingly simple. The loss over a batch is an average of the losses over individual examples, and the gradient of an average is the average of the gradients. If a batch of 1,024 sequences is split across 64 devices, each handling 16, then averaging the 64 local gradients produces exactly the gradient that a single device would have computed over all 1,024 sequences. Up to floating-point rounding and the order of summation, the result is mathematically identical to training on one enormous machine. This is synchronous data parallelism, and it is what nearly every large language model is trained with. In the early 2010s there was considerable interest in asynchronous alternatives, in which workers pushed gradients to central parameter servers and pulled back fresh weights whenever convenient, without waiting for each other. Google's DistBelief system of 2012, described by Dean and colleagues, was the landmark example. Asynchrony avoids waiting for stragglers but introduces stale gradients: a worker computes a gradient against weights that have already moved on, and the resulting noise harms convergence in ways that are hard to predict. As networks became faster and models larger, the field settled firmly on synchronous training, where every step sees the same weights everywhere and the results are reproducible. The cost of synchrony is a hard barrier in every step. No device may apply its update until it has the averaged gradient, and the averaged gradient does not exist until every device has contributed. The time spent producing that average is the price of data parallelism, and the whole engineering history of the technique is the history of making that price smaller and hiding it better. The all-reduce and the ring The operation that sums a set of arrays held on different devices and leaves the result on all of them is called an all-reduce. It belongs to a family of operations known as collectives, inherited from decades of high-performance computing and standardised in the Message Passing Interface, or MPI, in the 1990s. Its siblings will appear throughout the book: broadcast copies one device's array to all; reduce-scatter sums arrays across devices but leaves each device with only one slice of the result; all-gather collects a slice from every device and gives everyone the full concatenation; and all-to-all has every device send a distinct piece of data to every other device. The naive way to perform an all-reduce is to send every gradient to one device, sum them there, and send the result back. This is the parameter-server pattern, and it scales badly. If the model has G bytes of gradients and there are P devices, the central device must receive P × G bytes and send P × G bytes. Its network link becomes the bottleneck, and adding devices makes the bottleneck worse. The ring all-reduce, which entered deep learning around 2017 through an influential write-up by Andrew Gibiansky at Baidu and was then built into Uber's Horovod library, solves this elegantly. Arrange the P devices in a logical ring, each with a left and right neighbour. Divide each device's gradient array into P equal chunks. The algorithm then runs in two phases. In the first phase, a reduce-scatter, each device sends one chunk to its right neighbour and receives a different chunk from its left neighbour, adding the received chunk to its own corresponding chunk. After P − 1 such steps, each chunk has travelled around the ring accumulating contributions, and each device holds one chunk that is the complete sum across all devices. In the second phase, an all-gather, the completed chunks are passed around the ring once more, without further addition, so that after another P − 1 steps every device holds every completed chunk. The beauty lies in the traffic. In each of the 2(P − 1) steps, every device sends and receives exactly one chunk of size G / P. So each device sends a total of 2(P − 1) × G / P bytes, which for large P approaches 2G. That figure does not grow with the number of devices. A ring of eight devices and a ring of eight thousand devices each require every participant to send about twice the size of the gradients. Every link in the ring is busy at once, so the total bandwidth of the system is used rather than funnelled through one bottleneck. In the language of algorithm analysis, the ring all-reduce is bandwidth-optimal. It is not latency-optimal. The number of sequential steps grows linearly with P, and each step carries a fixed overhead: the time to initiate a transfer, traverse switches and synchronise. For large gradient arrays the bandwidth term dominates and the ring is excellent. For small arrays, or very large rings, the latency term takes over. Nvidia's Collective Communications Library, NCCL, which handles collectives for almost every GPU training framework, therefore also implements tree-based algorithms, which complete in a number of steps that grows only logarithmically with P, and chooses between ring and tree depending on message size and topology. Chapter 7 returns to these choices. Overlapping communication with computation Even a bandwidth-optimal all-reduce takes time. For a model with a billion parameters and 16-bit gradients, each device must send and receive about four gigabytes per step. At a typical inter-server bandwidth of 50 gigabytes per second per GPU, that is close to a tenth of a second of pure communication, during which, in a naive implementation, the GPU does nothing. The first major optimisation is to stop waiting until the backward pass is finished. The backward pass computes gradients layer by layer, from the last layer to the first. The gradient of the final layer is ready long before the gradient of the first. So the all-reduce for later layers can begin while the backward pass is still computing gradients for earlier layers. By the time the backward pass reaches the input, most of the communication is already done. PyTorch's DistributedDataParallel, described in a 2020 paper by Li and colleagues, implements this with gradient bucketing. Parameters are grouped into buckets of a configurable size, 25 megabytes by default. As soon as every gradient in a bucket has been computed, the bucket's all-reduce is launched asynchronously on a separate communication stream, and the backward pass continues on the compute stream. The bucket size is itself a trade-off. Buckets that are too small launch many tiny all-reduces, each paying the fixed latency cost. Buckets that are too large delay the start of communication and leave less room for overlap. With good bucketing, the visible communication time shrinks to whatever remains after the backward pass finishes: roughly the all-reduce of the first layers' gradients, plus any portion of the rest that took longer than the computation it was hidden behind. When the ratio of computation to communication is favourable, data parallelism approaches perfect scaling. When it is not, no amount of bucketing helps, and that ratio is governed by something easily overlooked: the batch size each device processes. A worked example: what one step costs It is worth putting numbers on the computation-to-communication ratio, because doing so shows exactly when data parallelism is cheap and when it is not. Take a model with 7 billion parameters, small by frontier standards but large enough to be useful, and suppose it fits on each GPU with room to spare. Its 16-bit gradients occupy 14 gigabytes. A ring all-reduce across many GPUs requires each GPU to send and receive about twice that, 28 gigabytes. Over an inter-server link delivering 50 gigabytes per second in each direction, the communication takes a little over half a second if nothing is hidden. Now the computation. By the rule from Chapter 1, each token costs about 6 × 7 billion, or 42 billion, floating-point operations for a forward and backward pass. Suppose each GPU processes eight sequences of 4,096 tokens per step, about 33,000 tokens. That is about 1.4 × 10^15 operations. An H100 running at a realistic 40 percent of its dense bfloat16 peak delivers about 400 trillion operations per second, so the computation takes about three and a half seconds. Half a second of communication against three and a half seconds of computation is comfortable. Even without overlap, communication would add only about 15 percent to the step time; with bucketing and overlap, almost all of it disappears behind the backward pass, which accounts for roughly two-thirds of the computation. This is why data parallelism on models of this size scales very nearly linearly across hundreds of GPUs. Change one number at a time and the picture shifts. Halve the tokens per GPU and the computation halves while communication stays the same; the ratio doubles. Move from 50 gigabytes per second between servers to the hundreds available on NVLink within a server and communication becomes trivial. Increase the model to 70 billion parameters while keeping the tokens per GPU fixed and both the communication and the computation grow tenfold, so the ratio is unchanged, but the model no longer fits on one GPU, and data parallelism alone ceases to be an option. The ratio depends on the tokens processed per device per step and on the bandwidth of the link, not on the size of the model. That observation is the hinge on which the rest of the chapter turns. The batch-size ceiling Data parallelism scales by adding devices, each with its own share of the batch. There are two ways to do that. Keep the per-device batch fixed and let the global batch grow with the number of devices, or keep the global batch fixed and shrink each device's share. Shrinking each device's share is called strong scaling, and it runs into the computation-to-communication ratio directly. The all-reduce costs the same regardless of how much data each device processed, because the gradient is always the size of the model. If each device processes less data, it computes for less time, and the fixed communication cost looms larger. Eventually the GPU spends more time waiting for the network than doing arithmetic, and it also falls below the batch size needed to keep its own tensor cores busy. Growing the global batch, weak scaling, keeps every device efficient but changes the optimisation problem. A larger batch gives a less noisy gradient estimate, which means each step is more reliable but the number of steps falls; past a certain point, doubling the batch no longer halves the number of steps needed to reach a given loss, and the extra computation is wasted. The 2017 paper by Goyal and colleagues at Facebook showed that ResNet-50 could be trained on ImageNet with a batch of 8,192 images across 256 GPUs in one hour, without loss of accuracy, using two now-standard tricks: scaling the learning rate linearly with the batch size, and warming up the learning rate gradually over the first few epochs. Later work pushed image-classification batches much larger with layer-wise adaptive optimisers such as LARS and LAMB. For language models, McCandlish and colleagues at OpenAI introduced in 2018 the idea of a critical batch size: a quantity, measurable from the gradient noise during training, beyond which larger batches give rapidly diminishing returns. The critical batch size grows as training progresses and the loss falls, which is one reason large runs often ramp the batch size up over the first part of training. Llama 3 405B, for instance, began with a batch of 4 million tokens and increased it in stages to 16 million. Sixteen million tokens sounds enormous, but divide it among sixteen thousand GPUs and each gets about a thousand tokens per step, roughly one-eighth of a single 8,192-token sequence. A thousand tokens per device is far too little work to keep a GPU busy, and even if it were enough, the device could not hold the model in the first place. This is the batch-size ceiling, and it is why pure data parallelism cannot train a frontier model on a frontier cluster. The global batch the optimiser can usefully absorb is finite. Divided by the number of devices needed for speed, it leaves each device with too little to do. Other forms of parallelism are needed not just to save memory but to put several devices to work on each piece of data, so that the number of data-parallel replicas, and the corresponding division of the batch, stays within limits the optimiser tolerates. Gradient accumulation and the opposite problem The batch-size question also has a mirror image. Sometimes a device cannot fit even its share of the global batch in memory. A run might want 512 sequences per step across 64 replicas, which is eight sequences per device, but memory permits only two at a time. The answer is gradient accumulation. The device processes a micro-batch of two sequences, computes gradients, and adds them to an accumulation buffer without communicating or updating. It repeats this four times. Only after the fourth micro-batch does it join the all-reduce and apply the update. Mathematically the result is identical to processing eight sequences at once. The cost is that the all-reduce can only overlap with the final micro-batch's backward pass, which is a smaller window, and frameworks provide hooks to suppress communication during the earlier micro-batches so that no bandwidth is wasted. Gradient accumulation decouples the batch the optimiser sees from the batch the device holds, and the idea of splitting each device's work into micro-batches reappears in pipeline parallelism, where it becomes essential rather than optional. What data parallelism leaves unsolved Data parallelism is a near-perfect solution to the compute wall, within the limits of the batch size. It does nothing for the memory wall. Every replica holds the full working weights, the full gradients and the full optimiser state, sixteen bytes for every parameter. A cluster of a thousand GPUs running pure data parallelism holds a thousand identical copies of the master weights and a thousand identical copies of each Adam moment. For a model small enough to fit on one device, that redundancy is a fair price for simplicity. For a model that does not fit, it is not a price at all; it is impossible. The redundancy is also striking once noticed. Of the sixteen bytes per parameter on each device, twelve belong to optimiser state that is used only at the very end of each step to compute the update, and every replica computes exactly the same update. If each of P replicas kept only one P-th of the optimiser state, updated only its own slice of the parameters, and shared the results, the same mathematics would be carried out with a fraction of the memory. The next chapter shows how that observation, with some careful accounting of communication, turned data parallelism from a technique for small models into one that can train models with hundreds of billions of parameters. Chapter 3: Sharding the State: ZeRO and FSDP In 2019 a team at Microsoft led by Samyam Rajbhandari posted a paper with a deliberately provocative title: "ZeRO: Memory Optimizations Toward Training Trillion Parameter Models." The acronym stood for Zero Redundancy Optimizer, and the central idea was almost embarrassingly simple. Data parallelism keeps a full copy of everything on every device. Most of that everything is redundant. Stop storing the redundant parts, fetch them when needed, and the memory available for the model grows in proportion to the number of devices. The paper, published at the SC20 supercomputing conference, became one of the most consequential systems papers of the decade, and its ideas, implemented in Microsoft's DeepSpeed library and later in PyTorch's Fully Sharded Data Parallel, are now part of the default toolkit for anyone training a model of any size. Three stages of giving things up ZeRO is usually described in three stages, each sharding one more component of the training state across the data-parallel devices. Sharding here means partitioning: dividing an array into P pieces and having each of P devices keep only its own piece. Stage 1 shards the optimiser state. Each of the P data-parallel devices keeps the 32-bit master weights and the two Adam moments for only one P-th of the parameters. The forward and backward passes are unchanged; every device still holds all the 16-bit working weights and computes a full gradient. But at the end of the step, instead of an all-reduce that gives every device the full averaged gradient, the system performs a reduce-scatter: each device receives the averaged gradient only for its own slice of parameters. It applies the Adam update to that slice using the optimiser state it holds. Then an all-gather distributes the updated 16-bit weights from every slice to every device, so all devices again hold the full working model for the next step. The communication volume is the striking part. Recall from Chapter 2 that a ring all-reduce is itself a reduce-scatter followed by an all-gather. ZeRO stage 1 also performs a reduce-scatter followed by an all-gather, simply with the optimiser update inserted in between. The bytes on the wire are the same as plain data parallelism. Memory falls dramatically: the twelve bytes per parameter of optimiser state become twelve divided by P. With 64 data-parallel devices, a 70-billion-parameter model's 840 gigabytes of optimiser state shrinks to about 13 gigabytes per device. This is free memory in the most literal sense, which is why ZeRO stage 1 is used almost universally in large-scale training. DeepSeek-V3, for example, was trained with ZeRO-1 as its data-parallel layer. Stage 2 also shards the gradients. Once the reduce-scatter has given each device the averaged gradient for its own slice, the device has no further use for the gradients of the other slices. Stage 2 discards them as soon as they have been reduced, rather than holding a full gradient buffer. The gradients are reduced in buckets during the backward pass, as in ordinary data parallelism, but each bucket is reduce-scattered and the non-owned portions freed immediately. Communication is still the same as plain data parallelism, and the two bytes per parameter of gradients drop to roughly two divided by P, plus a working buffer. Stage 3 shards the parameters themselves. Now each device holds only one P-th of the 16-bit working weights as well. This is the stage that changes the character of the computation, because a device cannot run a layer's forward pass without that layer's weights. So before each layer runs, the devices perform an all-gather to assemble the layer's full weights; they compute; and then they discard the non-owned portions. The backward pass does the same in reverse: all-gather the layer's weights again, compute the gradients, reduce-scatter them so each device keeps only its slice, and discard. The memory saving is now total. All sixteen bytes per parameter are divided by P. In principle, a model of any size can be trained if enough devices are added, and each device's memory need for model state falls without limit. The cost is communication. Stage 3 performs one all-gather of the parameters in the forward pass, another in the backward pass, and one reduce-scatter of the gradients. Each of these moves about the size of the model across each device's link. Plain data parallelism's all-reduce moves about twice the model size; stage 3 moves about three times. The original paper characterised this as a 1.5-fold increase in communication volume in exchange for memory that scales linearly with the number of devices. The stages in numbers The progression is easiest to appreciate with a concrete case. Take a 70-billion-parameter model trained across 64 GPUs with 80 gigabytes each, using the sixteen-byte accounting of Chapter 1. Under plain data parallelism, each GPU must hold all 1,120 gigabytes of training state. The job is impossible by a factor of fourteen before activations are even considered. Under ZeRO stage 1, each GPU holds the full 140 gigabytes of working weights and the full 140 gigabytes of gradients, but only one sixty-fourth of the 840 gigabytes of optimiser state, about 13 gigabytes. The total is around 293 gigabytes per GPU. Still impossible, but the problem has shrunk by a factor of nearly four. Under stage 2, the gradients are sharded too, falling to about 2 gigabytes per GPU plus a working buffer. The total is roughly 155 gigabytes, dominated now by the working weights. Still too large for an 80-gigabyte device. Under stage 3, every component is divided by 64. Each GPU holds about 17.5 gigabytes of persistent state, plus the temporarily gathered weights of the layer or two being computed at any moment. For a model with 80 layers, one layer's 16-bit weights come to under 2 gigabytes. That leaves the large majority of each GPU's memory free for activations. The job that was impossible under plain data parallelism now fits comfortably, with no change to the model or the mathematics. The example also shows why practitioners treat the stages as a menu rather than a ladder to be climbed to the top. If the model is small enough that stage 1 or 2 fits, there is no reason to pay stage 3's extra communication. If other forms of parallelism, introduced in the next chapters, already divide the weights across devices, then stage 1 on the data-parallel dimension may be all that is needed. Llama 3 and DeepSeek-V3 both combined sharding on the data-parallel axis with tensor, pipeline or expert parallelism on the others, rather than relying on full sharding alone. Fully Sharded Data Parallel ZeRO stage 3 proved so useful that it was reimplemented natively in PyTorch. Facebook's FairScale library introduced Fully Sharded Data Parallel, or FSDP, in 2021, and it moved into PyTorch itself shortly after. A 2023 paper by Zhao and colleagues at Meta, published at VLDB, describes its design and reports training models of well over a hundred billion parameters with it. FSDP's core design decision is the unit of sharding. Rather than shard each parameter tensor separately, it groups the parameters of a module, typically one Transformer layer, into a single flat buffer and shards that buffer evenly across devices. The all-gather before a layer's forward pass then becomes one large, efficient collective rather than many small ones. Users choose how to wrap their model into such units, and the choice matters: wrapping too finely produces many small collectives dominated by latency, while wrapping too coarsely forces a large chunk of the model to be fully materialised on every device at once, eroding the memory savings. To hide the communication, FSDP prefetches. While layer k is computing its forward pass, the all-gather for layer k + 1 is already running on a separate communication stream. In the backward pass, the all-gather for the preceding layer overlaps with the current layer's gradient computation, and the reduce-scatter of the current layer's gradients overlaps with the next. When each layer's computation takes longer than its communication, the communication disappears from the critical path entirely, and the device pays only in memory for the one or two layers that are fully materialised at any moment. A rewrite known as FSDP2, introduced in PyTorch during 2024, represents each sharded parameter as a distributed tensor that knows its own partitioning, rather than as a slice of a flat buffer. This makes FSDP compose more cleanly with tensor and pipeline parallelism, and Meta's TorchTitan project, described in a 2024 paper by Liang and colleagues, used it as the data-parallel layer of a multi-dimensional training stack built entirely in native PyTorch. Hybrid sharding and the network ZeRO stage 3 and FSDP assume that the all-gathers and reduce-scatters can be overlapped with computation. Whether they can depends on the ratio between the computation per layer and the communication per layer, and that depends heavily on which network link the collectives travel across. Within a single server of eight GPUs connected by Nvidia's NVLink, bandwidth per GPU runs to hundreds of gigabytes per second, and sharding across those eight is nearly free. Across servers, bandwidth per GPU drops by roughly an order of magnitude, and a collective spanning a thousand GPUs on a multi-tier network also pays more in latency. Sharding parameters across the entire cluster means that every layer's all-gather must cross the slowest links in the system, for every layer, twice per step. At large scale this can dominate. The response is hybrid sharding, which PyTorch's FSDP and DeepSpeed's variant called ZeRO++ both support in different forms. Devices are divided into groups, often a single server or a small set of servers. Within each group, the model state is fully sharded, so that the frequent parameter all-gathers travel over fast links. Across groups, the model is replicated, and the groups behave like ordinary data-parallel replicas, exchanging gradients once per step with an all-reduce over the slow links. The memory saving is now limited by the group size rather than the full cluster, but the expensive per-layer traffic stays on the fast part of the network. This is the first appearance of a pattern that will recur throughout the book: put the chatty communication on the short, fast links, and reserve the long, slow links for traffic that happens rarely. ZeRO++, described by Wang and colleagues at Microsoft in 2023, adds two further tricks: quantising the weights to 8-bit integers before the all-gather and the gradients to 4-bit before the reduce-scatter, and keeping a secondary copy of the parameters within each server so that the backward-pass all-gather never crosses the slow network. The authors reported reductions of up to four times in cross-node communication volume. Quantised communication of this kind trades a small amount of numerical fidelity for bandwidth, and Chapter 7 discusses when that trade is safe. When overlap breaks down Whether full sharding hides its communication depends on a ratio that can be estimated directly. For each layer, the all-gather moves the layer's weights: two bytes per parameter. The computation in that layer's forward pass is about two operations per parameter per token. So the ratio of computation to communication for one layer is about one operation per byte for every token the device processes. A device processing 16,000 tokens per step performs roughly 16,000 operations for every byte it gathers. That sounds generous, but the bandwidth matters as much as the ratio. An H100 at realistic utilisation performs about 400 trillion operations per second; across servers at 50 gigabytes per second it can receive about 50 billion bytes per second. Dividing the two gives a threshold of about 8,000 operations per byte. Below that, the all-gathers take longer than the computation they are meant to hide behind, and the GPU waits. A device processing 16,000 tokens is above the threshold, though not by a wide margin once latency and contention are counted. A device processing 4,000 tokens is well below it. This makes clear why full sharding and the batch-size ceiling of Chapter 2 interact so badly at very large scale. As more devices are added, each processes fewer tokens, and the ratio falls. Full sharding across a cluster of sixteen thousand GPUs, each processing a thousand tokens, would be hopelessly communication-bound over the inter-server network. The same calculation over NVLink, at many times the bandwidth, gives a threshold an order of magnitude lower, which is why sharding within a fast domain works when sharding across the whole cluster does not. Offloading: the memory beyond the GPU If GPU memory is the scarce resource, an obvious thought is to use other memory. A server typically has far more CPU memory than GPU memory, often a terabyte or more, and solid-state storage beyond that. ZeRO-Offload, published in 2021 by Ren and colleagues, moves the optimiser state and the optimiser computation itself to the CPU. The GPU runs the forward and backward passes, streams gradients to CPU memory over the PCIe bus, the CPU applies the Adam update with a carefully vectorised implementation, and the updated weights are streamed back. ZeRO-Infinity, published the same year, extends this to NVMe solid-state drives, treating the whole hierarchy of GPU memory, CPU memory and flash storage as one pool. These systems made it possible to fine-tune very large models on a handful of GPUs. Offloading spends the slowest bandwidth in the system, the PCIe connection between CPU and GPU, at around 64 gigabytes per second per direction for a PCIe Gen5 x16 link, to save the most expensive memory. For fine-tuning on modest hardware, that is often a good trade. For frontier pre-training, where the goal is maximum throughput on a cluster that has been sized to hold the model, it rarely is, and large pre-training runs generally avoid offloading for the main training state. They may still offload activations or use CPU memory for checkpoint staging. What sharding cannot do ZeRO and FSDP are among the most elegant ideas in the field because they change nothing about the mathematics of training and nothing about how the model is written. A researcher can take a model that runs on one GPU, wrap it in FSDP and train a version many times larger on many GPUs. That ease of use explains their popularity, and for models up to some tens of billions of parameters, FSDP or ZeRO on its own is often all that is needed. At the largest scales, two limits appear. The first is the batch-size ceiling from Chapter 2. FSDP is still data parallelism: every device processes its own share of the batch. Adding devices for memory means adding data-parallel replicas, and the global batch, divided among sixteen thousand devices, still leaves each with too little to do. Sharding solves the memory wall without loosening the batch constraint. The second is that sharding only addresses model state. Activations are not sharded by ZeRO or FSDP, because each device's activations belong to its own data and are not redundant. For a device to process even a single long sequence through a large model, it must at some moment hold the full activations of at least one layer for that sequence, and it must hold one layer's full weights while computing it. For the largest layers at long sequence lengths, even that can exceed what a single device can comfortably manage. Both limits point in the same direction. To go further, a single layer's computation, on a single piece of data, must itself be divided among several devices. That is tensor parallelism, and it requires a quite different relationship with the network. Chapter 4: Tensor Parallelism: Splitting the Layer In 2019 Mohammad Shoeybi and colleagues at Nvidia released a paper describing how they had trained a language model with 8.3 billion parameters, at the time the largest Transformer ever trained, on 512 GPUs. The model did not fit on a single GPU of the day, and the team did not have the sharding machinery that ZeRO would soon provide. Their solution was to split every layer of the network across several GPUs, so that each GPU held and computed only a slice of every weight matrix. The system was called Megatron-LM. The technique, now known as tensor parallelism or tensor model parallelism, remains a pillar of large-scale training, and the particular way Megatron arranged the split has become the standard pattern for Transformers. Tensor parallelism is the most intimate form of distribution. Data parallelism has devices exchange gradients once per step. Sharded data parallelism has them exchange weights once per layer. Tensor parallelism has them exchange activations in the middle of every layer, in both the forward and the backward pass, with each device unable to proceed until the exchange completes. It is the most powerful way to divide the work on a single piece of data, and it is the most demanding of the network. Understanding why it is both, and where in the network it can live, explains a great deal about how large training clusters are designed. How to split a matrix multiplication Almost all the arithmetic in a Transformer is matrix multiplication. The feed-forward block, often called the MLP, takes an input X, multiplies it by a weight matrix A to expand it to a larger hidden size, applies a nonlinear activation function, then multiplies by a second matrix B to bring it back to the original size. The attention block similarly multiplies its input by weight matrices to produce queries, keys and values, computes attention, and multiplies the result by an output matrix. There are two natural ways to split a matrix multiplication Y = XA across devices. One is to split A by columns. If A is divided into two halves side by side, A1 and A2, then XA1 and XA2 are the two halves of the output Y, also side by side. Each device holds half of A, receives the full input X, and produces half of Y. No communication is needed to compute the output; each device simply ends up holding a different slice of it. The other is to split A by rows. If A is divided into a top half and a bottom half, then the input X must also be split, into left and right halves X1 and X2, and the full output is X1A1 + X2A2. Each device computes one of these partial products, and the full result requires summing them, which is an all-reduce. Megatron's insight was to pair these two splits so that each block of the Transformer needs only one all-reduce in the forward pass. In the MLP, the first matrix A is split by columns. Each device computes its slice of the expanded hidden representation, and because the activation function is applied element by element, each device can apply it to its own slice without needing the others. The second matrix B is split by rows, matching the slices the first step produced. Each device multiplies its slice of the hidden representation by its rows of B, producing a partial sum of the full output. A single all-reduce sums these partial results, and every device ends up with the full output of the MLP, ready for the next block. The attention block splits even more naturally, because multi-head attention is already a set of independent computations, one per head. The matrices that produce queries, keys and values are split by columns so that each device gets a subset of the heads. Each device computes attention for its own heads with no communication at all. The output projection is split by rows, and once again a single all-reduce assembles the result. So each Transformer layer, consisting of one attention block and one MLP, requires two all-reduces in the forward pass and, by symmetry, two in the backward pass. Each all-reduce moves a tensor the size of the layer's activations: the number of tokens in the micro-batch times the hidden dimension. Shoeybi and colleagues reported 76 percent scaling efficiency for their 8.3-billion-parameter model across 512 GPUs, combining 8-way tensor parallelism within each server with data parallelism across servers. Why tensor parallelism lives inside the box Consider what these all-reduces mean in practice. They sit directly on the critical path. The MLP's second matrix multiplication cannot be completed into a usable output until the partial sums are combined, and the next block cannot start until it has that output. Unlike the gradient all-reduce of data parallelism, which can overlap with the backward computation of other layers, the activation all-reduces of tensor parallelism are tightly bound to the computation around them. Some overlap is possible with careful engineering, as discussed below, but much less. They are also large and frequent. Four all-reduces per layer, across a model with a hundred layers, means four hundred synchronising exchanges per micro-batch. Each moves an activation tensor that, for a hidden dimension of 16,384 and a micro-batch of 8,192 tokens in 16-bit precision, comes to about 268 megabytes. And as the number of tensor-parallel devices grows, the computation per device shrinks while the communication per device does not. With eight-way tensor parallelism each device does one-eighth of each matrix multiplication, but the all-reduce still moves about twice the full activation tensor across each device's link. Double the tensor-parallel degree to sixteen and the computation halves again while the communication stays roughly the same. The ratio of arithmetic to communication falls linearly, and eventually communication dominates. The only way tensor parallelism works at all is on links that are dramatically faster than those between servers. Table 2 sets out the bandwidth hierarchy of a modern GPU cluster, using manufacturers' headline figures, which explains where tensor parallelism can and cannot go. Table 2. Approximate headline bandwidths in a modern GPU cluster (manufacturer specifications). Link Connects Bandwidth per GPU Generation cited HBM3 / HBM3e memory GPU to its own memory 3.35 TB/s; 8 TB/s H100 SXM; B200 NVLink 4 GPUs within an 8-GPU server 900 GB/s (both directions) H100 NVLink 5 GPUs within a 72-GPU rack domain 1.8 TB/s (both directions) GB200 NVL72 PCIe Gen5 x16 GPU to CPU or network card about 64 GB/s each way Current servers InfiniBand NDR or 400G Ethernet Servers across the cluster 400 Gb/s, about 50 GB/s each way Common in 2024 to 2026 The step from NVLink to the inter-server network is roughly a factor of nine to eighteen in raw bandwidth, and in practice larger, because inter-server collectives also pay higher latency through multiple switch hops. DeepSeek's V3 report gives a useful real-world calibration: on its H800 cluster, achievable NVLink bandwidth was about 160 gigabytes per second per GPU and InfiniBand about 50, a ratio of about 3.2. The H800 was an export-restricted variant with deliberately reduced NVLink bandwidth, which is one reason DeepSeek chose not to use tensor parallelism at all. The consequence is a rule followed in almost every large training configuration: tensor parallelism stays within the NVLink domain. For most of the past five years, that domain was a single server of eight GPUs, and eight-way tensor parallelism became the near-universal default. Llama 3 405B used exactly that. Nvidia's rack-scale NVL72 systems, in which 72 Blackwell GPUs share a single NVLink fabric with a quoted aggregate bandwidth of 130 terabytes per second, enlarge the domain within which tensor parallelism is affordable. Whether the extra room is best spent on wider tensor parallelism or on other kinds of traffic, such as the all-to-all communication of mixture-of-experts models, is one of the live design questions of current systems. A worked example: the shrinking ratio The argument about ratios can be made exact with a little arithmetic. Consider the MLP block of a model with hidden dimension h, expanded to 4h in the middle, processing a micro-batch of t tokens. The two matrix multiplications together cost about 16 × t × h² floating-point operations in the forward pass: each of the two matrices has 4h² parameters, and each parameter costs two operations per token. Split across T tensor-parallel devices, each device performs 16th² / T operations. The communication is one all-reduce of the output, a tensor of t × h values, or 2th bytes in 16-bit precision. A ring all-reduce among T devices requires each to send about 2(T − 1)/T times the tensor size, close to 4th bytes for large T. Divide the two and the computation per byte of communication comes to about 4h / T. With a hidden dimension of 16,384 and eight-way tensor parallelism, each device performs roughly 8,000 operations for every byte it sends. Over NVLink 4, which moves around 450 gigabytes per second in each direction, an H100 at realistic utilisation can afford roughly a thousand operations per byte before communication becomes the bottleneck, so eight-way tensor parallelism has a comfortable margin, enough to absorb latency, synchronisation and imperfect overlap. Move the same tensor-parallel group across servers at 50 gigabytes per second and the affordable threshold rises to about 8,000 operations per byte, which the computation only just meets on paper and fails to meet in practice once latency is included. The formula also shows the two levers. A wider model, with larger h, can afford more tensor parallelism; this is one reason the largest dense models have been both wide and split eight ways. A higher tensor-parallel degree, with larger T, erodes the margin linearly. Doubling T to sixteen halves the operations per byte, which is why sixteen-way tensor parallelism was rarely attractive on eight-GPU servers, where half the group would have to communicate over the slow network. A constraint from the architecture Tensor parallelism also interacts with the shape of the model in ways that limit how far it can be pushed, quite apart from bandwidth. Splitting attention by heads requires the number of heads to be divisible by the tensor-parallel degree. That is rarely a problem for query heads, of which large models have dozens to over a hundred. It is more of a problem for the key and value heads in models that use grouped-query attention, a technique introduced by Ainslie and colleagues at Google in 2023 and now nearly universal, in which many query heads share a smaller number of key and value heads to reduce memory during inference. Llama 3 405B, for example, uses 128 query heads but only 8 key-value heads. With eight-way tensor parallelism, each device gets exactly one key-value head and its sixteen associated query heads, a clean division. With sixteen-way tensor parallelism, the eight key-value heads would have to be duplicated across pairs of devices, wasting memory and computation. The same pattern appears elsewhere. Vocabulary embedding tables, normalisation layers and other components each need a strategy for division, and each strategy has its own communication. Model designers who expect their models to be trained with tensor parallelism therefore choose dimensions that divide cleanly by the expected degree, and in practice most large models have hidden dimensions, head counts and expansion factors that are multiples of eight or more. The hardware has quietly shaped the architecture. Sequence parallelism and the leftovers Megatron's original tensor parallelism split the matrix multiplications but left some operations unsplit. Layer normalisation, dropout and the residual additions between blocks were computed redundantly: every tensor-parallel device performed the same operation on the same full activation tensor. These operations are cheap in arithmetic, but they are not cheap in memory, because their inputs and outputs are full-size activations that must be stored for the backward pass. With eight-way tensor parallelism, the matrix-multiplication activations were divided by eight but these remained whole, and they came to dominate activation memory. Korthikanti and colleagues addressed this in their 2022 paper, introducing what Megatron calls sequence parallelism. The observation is that layer normalisation and dropout operate independently on each token. So in the regions between the tensor-parallel blocks, the activation tensor can be split along the sequence dimension: each of the eight devices holds and processes one-eighth of the tokens. Entering a tensor-parallel block, the devices need the full activation tensor, so they perform an all-gather along the sequence dimension. Leaving the block, instead of an all-reduce that gives every device the full summed output, they perform a reduce-scatter that gives each device the summed output for its own share of tokens. The communication accounting is elegant. As Chapter 2 noted, an all-reduce is exactly a reduce-scatter followed by an all-gather. Sequence parallelism simply separates the two halves and places the layer normalisation and dropout between them, operating on the scattered state. The bytes moved are the same as before, but now every activation in the layer is divided across the tensor-parallel devices. The authors reported that sequence parallelism combined with selective recomputation reduced activation memory by about five times and cut the overhead of recomputation by over 90 percent, allowing them to train models up to a trillion parameters with much less wasted arithmetic. The name causes some confusion, because "sequence parallelism" has been used for several different techniques. Megatron's version operates only within a tensor-parallel group and only on the non-matrix-multiplication parts of the layer. Splitting the attention computation itself across devices along the sequence dimension, which is needed for very long contexts, is a different and harder problem, now usually called context parallelism and discussed in Chapter 6. Hiding the activation traffic Although tensor-parallel communication is harder to overlap than data-parallel communication, it is not impossible. The general strategy is to break each large matrix multiplication and its associated collective into smaller pieces, so that while one piece is being communicated, the next is being computed. Consider the all-gather that precedes a column-split matrix multiplication under sequence parallelism. Instead of waiting for the full gathered tensor, a device can begin multiplying the chunk of tokens it already holds, while the next chunk arrives from a neighbour. With the right scheduling, the matrix multiplication proceeds chunk by chunk, each one overlapping with the transfer of the next. Researchers at Google described such a decomposition for TPUs in a 2023 paper by Wang and colleagues, and similar ideas appear in Nvidia's Transformer Engine and in PyTorch's asynchronous tensor-parallel implementation. ByteDance's MegaScale paper credited this kind of fused overlap of tensor-parallel communication with computation as one of the main contributors to its 55 percent utilisation. These techniques work best when the matrices are large enough that each chunk still keeps the tensor cores busy, and they consume some of the GPU's own resources. Communication kernels run on the GPU's streaming multiprocessors, the same processors that do the arithmetic. Every multiprocessor devoted to moving data is one not devoted to computing. DeepSeek's report was unusually candid about this, describing custom communication kernels engineered to saturate both NVLink and InfiniBand while occupying only 20 of the H800's 132 multiprocessors. The price and the payoff Tensor parallelism earns its place for three reasons. It divides both the memory and the computation of each layer on a single piece of data, so it relieves the activation pressure that ZeRO and FSDP cannot touch. It does so with no change to the optimisation: the mathematics is exactly that of the unsplit model. And by putting several devices to work on each micro-batch, it reduces the number of data-parallel replicas needed for a given cluster size, which lets the global batch stay within the bounds the optimiser can use. Its price is that it consumes the most precious bandwidth in the system, and that it scales poorly beyond the boundary of the fast interconnect. Eight-way tensor parallelism on a model with 16,384-wide hidden layers keeps each device busy with large matrix multiplications. Sixty-four-way tensor parallelism would leave each device multiplying thin slivers of matrices, with communication swamping the arithmetic, even on the fastest links available. So tensor parallelism alone can extend a single replica of the model across a server, or perhaps a rack. A 405-billion-parameter model, with its multiple terabytes of training state, needs more than eight GPUs to hold even one replica, even with sharded optimiser state. The model must be divided further, across servers, using a technique whose communication is light enough to cross the slower links. That technique is pipeline parallelism. Chapter 5: Pipeline Parallelism and the Bubble Tensor parallelism splits each layer across devices. Pipeline parallelism does the opposite: it keeps each layer whole and splits the model by depth, giving the first group of layers to one device, the next group to another, and so on, like stations on an assembly line. Each device, called a stage, receives activations from the stage before it, runs its layers, and passes its output to the stage after it. In the backward pass the flow reverses, with gradients passing from later stages to earlier ones. The appeal is in the communication. Between stages, the only thing that crosses the wire is the activation tensor at the boundary between two layers, sent once forward and once backward per micro-batch. There are no all-reduces, no all-gathers, just point-to-point transfers between neighbours. For a model with a hundred layers split into sixteen stages, each stage sends one activation tensor per micro-batch in each direction, where a tensor-parallel layout would perform hundreds of collectives. This light, local traffic can cross the slower links between servers without dominating the step. Pipeline parallelism is therefore the natural way to spread a model across many servers when tensor parallelism has been confined within each one. The difficulty is time rather than bandwidth. An assembly line is efficient only when every station is busy, and a training step, which ends with a synchronised weight update, is an assembly line that must be emptied and refilled on every step. The idle time that results is called the pipeline bubble, and the history of pipeline parallelism is the history of shrinking it. The naive pipeline and its waste Start with the simplest version. Split a model into four stages on four devices and feed it one batch. Stage 1 runs its forward pass while stages 2, 3 and 4 wait. Then stage 2 runs while 1, 3 and 4 wait, and so on to stage 4. The backward pass then runs in reverse, stage 4 first, stage 1 last. At any moment, exactly one of the four devices is working. Utilisation is 25 percent. With sixteen stages it is about 6 percent. This arrangement saves memory, since each device holds only a quarter of the model, but it does nothing for speed. GPipe: micro-batches fill the line The first widely influential fix came from Google in 2019, in a paper by Yanping Huang and colleagues describing a system called GPipe. The idea borrows directly from the gradient accumulation of Chapter 2. Split each batch into many micro-batches. As soon as stage 1 finishes the forward pass of micro-batch 1 and sends it on, it starts micro-batch 2, while stage 2 works on micro-batch 1. After a short ramp-up, all stages are busy simultaneously, each working on a different micro-batch. When all micro-batches have completed their forward passes, the backward passes run through in the same staggered fashion. Gradients accumulate at each stage, and the weight update happens once at the end, as in ordinary synchronous training. The mathematics is unchanged. The bubble does not vanish; it moves to the edges. At the start of the forward phase, later stages wait for the first micro-batch to reach them. At the end, earlier stages finish and wait. The same happens around the backward phase. If there are p stages and m micro-batches, the fraction of time wasted in the bubble works out to (p − 1) / (m + p − 1). With four stages and four micro-batches, that is three-sevenths, about 43 percent. With four stages and thirty-two micro-batches, it falls to under 9 percent. The lesson is that the number of micro-batches must be large compared with the number of stages. That lesson collides with memory. In GPipe, every stage must store the activations for every micro-batch it has processed in the forward phase until the backward pass for that micro-batch arrives. With thirty-two micro-batches in flight, each stage holds thirty-two micro-batches' worth of activations. GPipe used activation recomputation to make this tolerable, storing only the inputs at stage boundaries, but the memory pressure still grows with the number of micro-batches, which is exactly the quantity that must be large to shrink the bubble. 1F1B: the steady state The next step came from work at Microsoft Research and Stanford on a system called PipeDream, described by Narayanan and colleagues in 2019. The original PipeDream was asynchronous, allowing different stages to use slightly different versions of the weights, which raised the same convergence concerns as asynchronous data parallelism. But it introduced a schedule that, adapted to synchronous training, became the standard: one forward, one backward, usually written 1F1B. In the 1F1B schedule, each stage begins by running enough forward passes to fill the pipeline, a number equal to the number of stages after it. Then it settles into a steady rhythm, alternating one forward pass of a new micro-batch with one backward pass of an older one. Because each backward pass frees the activations of its micro-batch, the number of micro-batches whose activations are held in memory at once is capped by the number of stages, not by the total number of micro-batches. At the end, the stage drains the remaining backward passes. The bubble in 1F1B is the same size as in GPipe for the same p and m; the schedule simply reorders the work. What changes is memory. Since activation memory no longer grows with m, the number of micro-batches can be increased to shrink the bubble without running out of memory. That decoupling is what made pipeline parallelism practical at scale. Interleaving: more, smaller stages The bubble fraction depends on the ratio of pipeline depth to micro-batch count. Another way to shrink it is to make each stage's share of the work smaller, so that ramp-up and drain take less time. Narayanan and colleagues, now working with Nvidia, introduced the interleaved schedule in their 2021 paper at the SC conference on training Megatron-LM at scale. Instead of giving each device one contiguous block of layers, give it several smaller, non-contiguous blocks, called virtual stages or model chunks. With four devices and a 32-layer model, device 1 might hold layers 1 to 4 and 17 to 20, device 2 layers 5 to 8 and 21 to 24, and so on. A micro-batch now travels around the ring of devices twice. Each pass through a device takes half as long, and the bubble shrinks by a factor equal to the number of chunks per device. The cost is communication. A micro-batch now crosses stage boundaries twice as many times, so point-to-point traffic doubles. Since pipeline traffic is light, this is usually a good trade. The same paper reported training a trillion-parameter model across 3,072 A100 GPUs at 502 petaflops per second, about 52 percent of theoretical peak, combining 8-way tensor parallelism inside each server, interleaved pipeline parallelism across servers, and data parallelism across pipeline replicas. That combination, often called 3D parallelism, became the template for frontier training, and Chapter 6 examines it in detail. A worked example: cheap traffic, expensive waiting The contrast between pipeline parallelism's light communication and its heavy idle time is easiest to see with numbers drawn from a configuration like Llama 3 405B: a model with a hidden dimension of 16,384, divided into sixteen pipeline stages, processing micro-batches of one sequence of 8,192 tokens. Start with the communication. At each stage boundary, the activation tensor for one micro-batch is 8,192 tokens times 16,384 values, at two bytes each: about 268 megabytes. Across an inter-server link at 50 gigabytes per second, it takes a little over five milliseconds to send. If the stage boundary falls inside a tensor-parallel group of eight GPUs, each GPU can send one-eighth of the tensor over its own network card, cutting the time to well under a millisecond. Now the computation for that micro-batch on one stage. The stage holds a sixteenth of the model, about 25 billion parameters. Its forward pass for 8,192 tokens costs about 2 × 25 billion × 8,192, or 4 × 10^14 operations, spread across the eight GPUs of the tensor-parallel group. At 400 trillion operations per second per GPU, that is about an eighth of a second, and the backward pass takes about twice as long. Against roughly a third of a second of computation per micro-batch, a millisecond or less of transfer is negligible. Pipeline parallelism's communication is, by this measure, nearly free. The bubble is another matter. With sixteen stages and sixteen micro-batches per step, the simple formula for a non-interleaved schedule gives a bubble fraction of 15 / 31, almost half of all device time wasted. That is clearly unacceptable, and it shows why a configuration like Llama 3's could not have used a plain 1F1B schedule. With interleaving, each device holds several non-contiguous chunks and the bubble shrinks in proportion: four chunks per device would reduce the idle time to roughly a quarter of the useful computation, eight chunks to roughly an eighth, at the cost of four or eight times as many point-to-point transfers, each still small. Here the trade is laid bare. Pipeline parallelism has bandwidth to spare and time to lose, and interleaving spends the first to recover the second. Zero bubble and DualPipe For several years the interleaved 1F1B schedule was the state of the art. Then, in 2023 and 2024, two lines of work showed that the bubble could be nearly eliminated by looking more closely at what the backward pass actually does. The backward pass of a layer computes two different things. It computes the gradient with respect to the layer's input, which must be passed back to the previous layer so that its backward pass can proceed. And it computes the gradient with respect to the layer's weights, which is needed only for the optimiser update at the end of the step. The first is on the critical path of the pipeline. The second is not. Qi and colleagues at Sea AI Lab exploited this in a paper presented at ICLR in 2024 titled "Zero Bubble Pipeline Parallelism." They split each backward pass into its two parts, often labelled B for the input gradient and W for the weight gradient. B must happen promptly, because the previous stage is waiting for it. W can be deferred and scheduled into whatever gaps remain. By placing the weight-gradient computations into the slots that would otherwise be bubble, and by rearranging the synchronisation around the optimiser step, they constructed schedules with essentially no bubble under realistic conditions, at the cost of somewhat more memory to hold the deferred work. DeepSeek's DualPipe, described in the V3 report of December 2024, pushes in a related direction with a different emphasis. DualPipe feeds micro-batches into the pipeline from both ends at once, so that the first and last stages, which are normally idle during ramp-up and drain, have work from the start. It also splits each chunk of computation into finer pieces, separating attention, the dispatch and combine steps of the mixture-of-experts layer, and the expert computation itself, and arranges them so that the all-to-all communication of one micro-batch overlaps with the computation of another. DeepSeek reported that this achieved near-zero visible overhead from all-to-all communication and reduced the pipeline bubble compared with earlier methods, at the cost of keeping two copies of the model's parameters on each device, which their memory budget allowed because they used no tensor parallelism. The value of DualPipe lies less in any single trick than in a principle: in a pipeline, the bubble and the communication are both forms of idle time, and a sufficiently fine-grained schedule can fill each with the other's work. The practical limits Pipeline parallelism's elegance on paper comes with real friction. The first is balance. A pipeline runs at the speed of its slowest stage. Transformer layers are nearly identical, which helps, but the first stage also holds the token embedding and the last stage holds the output projection and loss calculation, which for a large vocabulary is substantial. Llama 3's vocabulary of 128,000 tokens made the output layer heavy enough that Meta deliberately placed one fewer Transformer layer on the first and last stages to compensate. Any imbalance becomes bubble. The second is inflexibility with batch size. The bubble shrinks only when the number of micro-batches per step is large compared with the number of stages. But the micro-batch count is fixed by the global batch size divided by the data-parallel degree and by the micro-batch size, and Chapter 2 showed that the global batch cannot grow indefinitely. On a very large cluster, the data-parallel degree is large, so each pipeline replica gets few micro-batches. Llama 3's engineers reported modifying their pipeline schedule so that the number of micro-batches need not be a multiple of the number of stages, precisely to keep this constraint from forcing awkward choices. The third is complexity. A pipeline-parallel training loop is no longer a simple forward call followed by a backward call. It is a carefully choreographed schedule of sends, receives, partial forward passes and partial backward passes, different on every stage. Bugs in such schedules produce deadlocks or silently wrong gradients. Debugging requires tooling that understands the schedule. For these reasons, pipeline parallelism is often the last dimension added, when a model is too large for tensor parallelism and sharding alone, rather than the first. The fourth is memory imbalance across stages. In 1F1B, the first stage must hold activations for as many micro-batches as there are stages, while the last stage holds activations for only one. Stages near the front of the pipeline therefore run out of memory first. Schedules such as Zero Bubble and DualPipe have their own memory profiles, and choosing a schedule is partly a question of which device's memory is binding. Despite these frictions, pipeline parallelism is indispensable at the largest scales, because it is the only established way to spread a single replica of a dense model across many servers without flooding the inter-server network. It takes the model that tensor parallelism has spread across a server and spreads it further, server to server, sending only thin activations across the slow links. The next chapter shows how it fits together with the other strategies, and how two newer dimensions, for long sequences and for mixture-of-experts models, complete the picture. Chapter 6: Composing the Dimensions No single strategy from the preceding chapters can train a frontier model on its own. Data parallelism is bounded by the batch size and does nothing for memory. Sharding fixes the memory of model state but still divides the batch. Tensor parallelism divides each layer but cannot leave the fast interconnect. Pipeline parallelism spreads the model across servers but needs many micro-batches to keep its bubble small. Each relieves a different constraint and each hits a different limit. The answer, worked out in practice over roughly 2020 to 2022 and refined since, is to use them all at once, each along its own axis of a multi-dimensional grid of devices, with each axis mapped onto the part of the physical network that suits its traffic. The grid Picture the GPUs of a cluster arranged as a multi-dimensional block. Along one axis, groups of eight GPUs form tensor-parallel groups; each group jointly holds one slice of each layer. Along a second axis, sixteen such groups form a pipeline; each group holds a different contiguous chunk of the model's layers. Together, the 8 × 16 = 128 GPUs hold one complete replica of the model. Along a third axis, many such replicas form data-parallel groups, each processing different data, synchronising gradients once per step, and usually sharding their optimiser state ZeRO-1 style. Every GPU thus has coordinates: a tensor-parallel rank, a pipeline rank and a data-parallel rank. It belongs to one tensor-parallel group, one pipeline group and one data-parallel group, and it communicates with each group in a different pattern. With its tensor-parallel group, it performs frequent all-reduces or all-gathers and reduce-scatters of activations, inside every layer. With its pipeline neighbours, it exchanges activations and their gradients point-to-point, once per micro-batch per stage boundary. With its data-parallel group, it reduces gradients and gathers updated weights, once per step. The principle that organises the grid is simple. Order the dimensions by how much and how often they communicate, and map them onto the network from fastest to slowest. Tensor parallelism, with the heaviest and most latency-sensitive traffic, takes the innermost position, within a server or rack connected by NVLink. Context parallelism, discussed below, comes next. Pipeline parallelism, with its light point-to-point traffic, crosses server boundaries. Data parallelism, with its large but infrequent and overlappable gradient exchange, takes the outermost position and spans the whole cluster. The differences among the strategies that justify this ordering are summarised in Table 3, which compares them on what each divides, which collective it relies on, how often it communicates and where it is usually placed. Table 3. The main parallelism strategies compared. Strategy What is divided Main communication Frequency Usual placement Data (with ZeRO-1) The batch; optimiser state Reduce-scatter and all-gather of gradients and weights Once per step Outermost, across the cluster Fully sharded data (ZeRO-3, FSDP) The batch; all model state All-gather of weights, reduce-scatter of gradients Every layer, twice per step Often within a group of servers Tensor Each weight matrix All-reduce, or all-gather and reduce-scatter, of activations Several times per layer Within the NVLink domain Pipeline The layers Point-to-point activations and gradients Per micro-batch per stage boundary Across servers Context Each sequence Exchange of keys and values Every attention layer Within or near the NVLink domain Expert The experts of each MoE layer All-to-all of tokens Twice per MoE layer Within or across a few servers A worked example: Llama 3 405B The clearest published account of how these dimensions fit together on a real frontier run is the infrastructure section of Meta's 2024 Llama 3 paper. Its configurations are set out in Table 4. Table 4. Parallelism layouts used to train Llama 3 405B (Llama Team, Meta, 2024). GPUs Tensor Context Pipeline Data Sequence length Utilisation (BF16 MFU) 8,192 8 1 16 64 8,192 43% 16,384 8 1 16 128 8,192 41% 16,384 8 16 16 8 131,072 38% Read along a row and the logic of the grid becomes concrete. Eight-way tensor parallelism matches the eight GPUs in each H100 server, so all tensor-parallel traffic stays on NVLink. Sixteen pipeline stages spread each model replica across sixteen servers, 128 GPUs in all. That leaves 64 or 128 data-parallel replicas, depending on cluster size. Each step processed 16 million tokens. With 128 replicas, each replica handled 16 sequences of 8,192 tokens per step, which served as the micro-batches for its pipeline. Doubling the cluster from 8,192 to 16,384 GPUs doubled the data-parallel degree and halved the micro-batches per replica, and utilisation slipped from 43 to 41 percent, the pipeline bubble and the larger data-parallel collectives taking a small extra toll. The third row shows a different configuration for the later phase of training, when the context length was extended to 128,000 tokens. The data-parallel degree collapses from 128 to 8, and a new dimension, context parallelism, takes the difference, at 16-way. The total is still 16,384 GPUs. What has changed is that each sequence is now so long that a single tensor-parallel group cannot hold its activations, so the sequence itself must be divided. And since each step still processes a bounded number of tokens, the number of separate sequences per step falls, freeing devices from the data-parallel axis to work on context. This is the general pattern. The total number of devices is the product of the parallel degrees. Increasing one degree for a constraint, whether memory, sequence length or model size, means decreasing another to keep the product fixed, and every reallocation shifts traffic between links of different speeds. Configuring a large training run is a search through this space, guided by the constraint that whatever is chosen must fit in memory and keep the slowest link from dominating. Checking that it fits A layout like this can be sanity-checked against the memory arithmetic of Chapter 1. With eight-way tensor parallelism and sixteen pipeline stages, each GPU holds one-hundred-and-twenty-eighth of the model's parameters: about 3.2 billion of Llama 3 405B's 405 billion. At two bytes each for 16-bit weights and two for gradients, that is about 12.7 gigabytes. The remaining twelve bytes per parameter of optimiser state would add another 38 gigabytes if held in full, but sharding it across the data-parallel group, as ZeRO stage 1 does, divides that by the data-parallel degree. With 128 replicas, the optimiser state per GPU falls to well under a gigabyte. So model state occupies something like 13 to 14 gigabytes of each H100's 80. The rest is available for activations, which is where the pipeline schedule and the micro-batch size come in. Under 1F1B, the first stage holds activations for up to sixteen micro-batches at once; with sequence parallelism dividing those activations by eight and selective recomputation discarding the most expensive intermediates, they fit within the remaining memory, with room for communication buffers and workspace. The calculation is approximate, and real systems lose memory to fragmentation and framework overhead, but it shows that the layout is not arbitrary. Each degree was chosen so that the product of the constraints, memory per device, NVLink domain size, pipeline bubble and batch size, could all be satisfied at once. Change one parameter and the others must adjust. Drop pipeline parallelism from sixteen to eight stages and each GPU's share of the model doubles, to about 25 gigabytes of weights and gradients, while the first stage holds activations for half as many micro-batches but twice as many layers each, so its activation memory barely moves; the saving has to come from somewhere else, or the configuration no longer fits. Raise the sequence length from 8,192 to 131,072, as in the third row of Table 4, and activations per sequence grow sixteenfold, overwhelming any fixed layout. The only way to absorb that growth without exceeding memory is to divide the sequence itself, which is what context parallelism does. Context parallelism The quadratic cost of attention means that long sequences create pressure that none of the earlier strategies addresses. FlashAttention removes the quadratic memory for the attention scores, but activations for other operations still scale linearly with sequence length, and at a hundred thousand tokens or more they exceed what a tensor-parallel group can hold, even with sequence parallelism and recomputation. Context parallelism divides each sequence along its length across several devices. Most operations in a Transformer layer act on each token independently, so splitting the tokens is straightforward. Attention is the exception: every token's query must, in principle, attend to the keys and values of every earlier token in the sequence, including tokens that now live on other devices. The best-known solution is Ring Attention, published by Hao Liu, Matei Zaharia and Pieter Abbeel at Berkeley in 2023. The devices are arranged in a ring, each holding one block of the sequence. Each device begins by computing attention between its own queries and its own keys and values. It then passes its keys and values to its neighbour and receives the neighbour's block, and computes attention between its queries and the newly arrived keys and values, combining the results using the same running-softmax technique that FlashAttention uses within a single device. After going around the full ring, every device has attended over the entire sequence, and the transfer of each block can overlap with computation on the previous one. Because a causal language model attends only to earlier tokens, naive block assignment leaves devices with the early tokens doing much less work than those with late tokens; practical implementations assign each device a balanced mixture of early and late chunks to even out the load. Meta's Llama 3 team chose a different approach, an all-gather-based method in which each device collects the keys and values for the whole sequence and then computes attention for its own local queries. This uses more memory and does not overlap as neatly, but it is simpler, adapts more easily to different attention masks, and the authors judged that at their scale, where the keys and values were small relative to the queries thanks to grouped-query attention, the communication cost was acceptable. Other variants, such as the DeepSpeed-Ulysses approach published by Microsoft in 2023, swap between sequence-split and head-split layouts with all-to-all exchanges. The diversity of approaches reflects how new the problem is: models with context windows of a hundred thousand tokens or more became standard only around 2024. Expert parallelism The final dimension arises from a change in the model rather than the data. A mixture-of-experts model replaces some or all of its feed-forward blocks with a set of parallel expert networks and a small router. For each token, the router selects a few experts, typically between one and eight out of dozens or hundreds, and only those experts process the token. The model's total parameter count can therefore be very large while the computation per token stays modest. Google's GShard of 2020 and Switch Transformer of 2021, by Lepikhin and Fedus and their colleagues, established the approach at scale, and by 2025 many of the strongest openly documented models, including DeepSeek-V3, Mixtral and Qwen's larger models, used it. Mixture-of-experts layers are natural candidates for a new kind of division: put different experts on different devices. This is expert parallelism. The communication pattern is distinctive. After the router decides which experts each token should go to, every device must send each of its tokens to the devices holding the chosen experts, which is an all-to-all exchange. After the experts compute, the results must be sent back to the devices the tokens came from, another all-to-all. Unlike an all-reduce, whose volume is fixed and predictable, an all-to-all's traffic depends on the router's decisions, which change with every batch. If many tokens favour the experts on one device, that device and its links become a hot spot. DeepSeek-V3 illustrates how far the engineering of expert parallelism has gone. The model has 671 billion total parameters but activates 37 billion per token. DeepSeek trained it on 2,048 H800 GPUs using 16-way pipeline parallelism, 64-way expert parallelism spanning eight servers, and ZeRO-1 data parallelism, with no tensor parallelism at all. To keep all-to-all traffic off the slower InfiniBand network, the routing algorithm was constrained so that each token could be sent to experts on at most four servers; within a server, tokens were forwarded on to the specific GPUs over NVLink. This node-limited routing is a direct example of shaping a model's design around the network's physical hierarchy. The DualPipe schedule described in Chapter 5 then overlapped the remaining all-to-all communication with computation. The whole pre-training run took about 2.7 million H800 GPU-hours; the full process including context extension and post-training took about 2.8 million. Searching the space The number of possible configurations grows quickly with the number of dimensions. With five dimensions, a cluster of sixteen thousand GPUs and a model of a given size, there are typically dozens of valid factorisations of the device count, and for each one, further choices of micro-batch size, recomputation policy, pipeline schedule, number of virtual stages and sharding stage. Some will not fit in memory. Among those that do, utilisation can differ by a large margin. Practitioners use a combination of analytical models, which estimate memory and communication for each configuration from first principles, and short trial runs on the real cluster. Research systems such as Alpa, described by Zheng and colleagues in 2022, attempted to automate the search entirely, formulating it as an optimisation problem over intra-operator and inter-operator parallelism. In practice, frontier teams still largely configure by hand, guided by rules of thumb that follow from the bandwidth hierarchy: set tensor parallelism to the size of the NVLink domain or smaller; add pipeline stages until the model state fits; add context parallelism only when sequences demand it; place expert parallelism where the all-to-all can stay on fast links; and give everything else to data parallelism. That last rule hides a constraint worth stating outright. The data-parallel degree must divide the global batch into per-replica shares large enough to keep each pipeline full. On the largest clusters, this pushes in the opposite direction from the other dimensions. More devices mean more data-parallel replicas, fewer micro-batches per replica, and a larger bubble. One reason frontier teams have pursued larger NVLink domains, deeper pipelines with better schedules, and context parallelism is that each lets more devices work on each sequence, keeping the data-parallel degree, and thus the division of the batch, within bounds. A multi-dimensional layout is the device-level expression of the book's central trade-off. Each dimension spends a different resource. Tensor and context parallelism spend fast bandwidth. Pipeline parallelism spends schedule time. Data parallelism spends batch size. Sharding spends a modest amount of extra communication. Expert parallelism spends unpredictable all-to-all traffic. A good configuration spends each where it is cheapest. The next chapter turns from the logical layout to the physical network it runs on, and to the techniques for making that network's limited bandwidth go further. Chapter 7: The Communication Bottleneck Every chapter so far has treated communication as a cost to be accounted for: so many bytes per step, crossing such-and-such a link. This chapter looks at the network itself, the physical and software machinery that carries those bytes, and at the three broad families of techniques for coping with its limits. Communication can be hidden, by overlapping it with computation so that the arithmetic never waits. It can be shrunk, by sending fewer or smaller bytes. And it can be avoided, by arranging the computation and even the model so that heavy traffic never needs to cross the slow links. The most efficient large training systems use all three. The physical network A modern training cluster has at least two networks, built for different jobs. The first is the scale-up network, the NVLink fabric that connects GPUs within a server or a rack. Its defining feature is that every GPU can reach every other GPU in the domain directly, at full bandwidth, through NVSwitch chips. In an H100 server, eight GPUs share such a fabric. In Nvidia's GB200 NVL72 rack, seventy-two do, each with 1.8 terabytes per second of NVLink bandwidth. Within this domain, collectives are fast and nearly topology-free: an all-reduce across eight GPUs on NVSwitch behaves as though the GPUs were all connected to a single, very fast hub. The second is the scale-out network, which connects servers to each other across the whole cluster. Each GPU typically has its own network interface card, usually at 400 gigabits per second in clusters built from 2023 to 2025, moving to 800 gigabits with newer hardware. These cards connect to a hierarchy of switches. The standard design is a fat-tree or Clos network: servers connect to leaf switches, leaf switches to spine switches, and in large clusters spine switches to a further tier. A well-built fat-tree provides full bisection bandwidth, meaning that if half the cluster sends to the other half, every link can run at full speed. In practice, large clusters often accept some oversubscription in the upper tiers to reduce cost, which means traffic that crosses many switches may contend for bandwidth. There are two dominant technologies for the scale-out network. InfiniBand, long the standard in supercomputing and now supplied mainly by Nvidia following its acquisition of Mellanox, was designed from the start for low-latency, lossless communication with remote direct memory access, which lets one machine's network card write directly into another's memory without involving either CPU. Ethernet with RDMA over Converged Ethernet, known as RoCE, offers the same direct memory access over standard Ethernet hardware, with additional mechanisms to approximate InfiniBand's lossless behaviour. Meta trained Llama 3 405B on a RoCE cluster of about 24,000 GPUs organised as a three-layer Clos network, with 400-gigabit links between GPUs, while its smaller models were trained on an InfiniBand cluster of similar size; Meta has described tuning both fabrics so that neither became the bottleneck for its largest jobs. The industry consortium known as the Ultra Ethernet Consortium, formed in 2023, is working to extend Ethernet specifically for AI and high-performance workloads. Both technologies also support GPUDirect RDMA, which lets the network card read and write GPU memory directly, bypassing CPU memory. Without it, every inter-server transfer would have to be copied from GPU to CPU memory, sent, and copied back, across the comparatively slow PCIe bus. Collective libraries Between the training framework and the hardware sits the collective communication library. For Nvidia GPUs this is almost always NCCL, pronounced "nickel." AMD's equivalent is RCCL, and Google's TPUs use collectives built into the XLA compiler and the inter-chip interconnect of the TPU pods. NCCL's job is to take a request such as "all-reduce this 500-megabyte buffer across these 1,024 GPUs" and execute it as efficiently as the topology allows. It detects which GPUs share NVLink, which share a network card, and how the network cards are connected, and builds rings or trees accordingly. For a large all-reduce spanning many servers, a common strategy is hierarchical: reduce within each server over NVLink first, then perform the cross-server portion with one participant per GPU position, each handling a slice of the data in parallel across the multiple network cards, and finally broadcast back within each server over NVLink. This keeps most of the bytes on the fast links and makes use of every network card at once. The choice of algorithm depends on message size. For large messages, rings maximise bandwidth use. For small messages, where the fixed cost of each step dominates, trees finish in fewer steps. NCCL also offers algorithms that use the switches themselves to do arithmetic. Nvidia's SHARP technology, in InfiniBand switches and in NVSwitch, can perform the summation of an all-reduce inside the network: each GPU sends its data once, the switches add the contributions as they pass through, and the result is sent back. This roughly halves the traffic each GPU must send compared with a ring, and reduces latency. At the largest scales, teams have found reasons to modify the library itself. Meta's Llama 3 paper describes NCCLX, an internal fork of NCCL tuned for its network, with changes to how data is chunked and transferred, how traffic is prioritised, and how small control messages are handled on high-latency multi-hop paths. Such customisation is a sign that, at sixteen thousand GPUs, the defaults that serve a thousand-GPU job leave meaningful performance unclaimed. Hiding communication The single most effective technique in distributed training is overlap: arranging the work so that while some bytes are crossing the network, the arithmetic units are busy with something that does not depend on them. Earlier chapters described the main instances. Data-parallel gradient reduction overlaps with the backward pass, bucket by bucket. FSDP prefetches the next layer's weights while the current layer computes. Tensor-parallel collectives can be decomposed into chunks that overlap with chunked matrix multiplications. Context parallelism overlaps the passing of key and value blocks with attention computation. DualPipe overlaps one micro-batch's all-to-all with another's computation. Overlap is not free. It requires the communication and computation to run concurrently on the same GPU, and they compete for its resources. Communication kernels need streaming multiprocessors to drive the transfers and memory bandwidth to read and write the buffers. When a collective runs alongside a matrix multiplication, both can slow down. DeepSeek's choice to limit its communication kernels to 20 of 132 multiprocessors was an explicit decision about how much computation to give up for communication. Hardware designers have responded by moving more communication work off the multiprocessors: copy engines, network-card offloads and in-switch reduction all aim to let the network move data without borrowing the GPU's arithmetic units. Overlap also has a ceiling. It can hide communication only up to the amount of independent computation available. If a step contains a hundred milliseconds of computation and a hundred and fifty milliseconds of unavoidable communication, the best possible step time is a hundred and fifty milliseconds, no matter how cleverly the work is scheduled. When communication exceeds computation, the only remedies are to shrink it or avoid it. Shrinking communication The most direct way to shrink communication is to send fewer bits per number. This is closely tied to the broader move towards lower precision in training. The shift from 32-bit to 16-bit training around 2018 halved the size of gradients and activations, and with them the volume of most collectives. The next step is 8-bit floating point, FP8. A 2022 paper by Micikevicius and colleagues from Nvidia, Arm and Intel proposed two FP8 formats, one with more exponent bits for range and one with more mantissa bits for precision, and Nvidia's Hopper and Blackwell GPUs run matrix multiplications in FP8 at roughly twice the bfloat16 rate. FP8 training with conventional per-tensor scaling proved fragile at first, because a single outlier value forces a coarse scale on the whole tensor. DeepSeek-V3 was among the first frontier-scale models trained largely in FP8, using fine-grained scaling: activations were scaled in tiles of 1 × 128 elements and weights in blocks of 128 × 128, so that each small region had its own scale factor, and partial sums were periodically promoted to 32-bit precision to limit accumulated rounding error. The report states that the run experienced no irrecoverable loss spikes and required no rollbacks. DeepSeek also cached and dispatched activations in FP8 for its mixture-of-experts all-to-all communication, directly reducing the traffic of its heaviest collective. Blackwell GPUs add hardware support for still smaller block-scaled formats, including 4-bit variants, and how far these can be pushed into training is under active investigation. More aggressive compression has a long research history. Gradient quantisation schemes such as 1-bit SGD, from Seide and colleagues at Microsoft in 2014, and later 1-bit Adam, sent only the sign of each gradient component plus error-feedback mechanisms that carried the quantisation error forward to the next step. Sparsification schemes send only the largest gradient components. These can reduce traffic by an order of magnitude or more. They are less common in frontier pre-training than their papers might suggest, partly because modern collectives on fast networks are often not the bottleneck once overlap is in place, partly because compression interacts badly with the reduce-scatter and all-reduce algorithms that collective libraries optimise, and partly because any change to the numerical behaviour of a run costing tens of millions of dollars must earn its risk. A different form of shrinking comes from reducing how often devices communicate. Local SGD and its descendants let each data-parallel replica take several optimiser steps independently before synchronising. Google DeepMind's DiLoCo, published by Douillard and colleagues in 2023, applies this to language model training across poorly connected clusters: each replica trains for hundreds of steps on its own, then the replicas exchange and average the change in their parameters using an outer optimiser. The authors reported performance comparable to fully synchronous training while communicating about five hundred times less. Work of this kind matters most where bandwidth between groups of devices is very low, such as training across multiple data centres, a scenario that has moved from curiosity to serious engineering concern as single-site power limits have become binding for the largest clusters. Avoiding communication The most powerful technique is not to need the communication at all. This means designing the layout, and sometimes the model, around the network. The parallelism ordering of Chapter 6 is the primary example. Placing tensor parallelism inside the NVLink domain is a choice to avoid sending tensor-parallel traffic over the slow network. Hybrid sharding avoids sending parameter all-gathers across servers. Node-limited routing in DeepSeek-V3 constrained the model itself so that no token's expert traffic could fan out to more than four servers. Topology-aware placement extends this to the scale-out network. In a fat-tree, GPUs under the same leaf switch can communicate without touching the spine. A scheduler that places each pipeline's stages, or each data-parallel group, under as few switches as possible reduces contention in the upper tiers. Meta's Llama 3 paper describes both halves of this: a job scheduler aware of the network topology, and an ordering of the parallelism dimensions, tensor innermost and data outermost, chosen so that the most communication-intensive groups sat closest together. Model architecture choices also bear on communication, often in ways not obvious from a description of the model. Grouped-query attention, which shares key and value projections across several query heads, reduces the size of the keys and values exchanged in context parallelism. A larger hidden dimension with fewer layers changes the ratio of tensor-parallel communication to computation. Mixture-of-experts architectures trade dense computation for all-to-all traffic. Increasingly, frontier model designs are chosen jointly with their parallel layout and the hardware they will run on, an approach the DeepSeek team has described explicitly as hardware and model co-design. Diagnosing a communication-bound job Knowing the techniques is only half the work. The other half is recognising, in a running job, whether communication is the problem at all, and if so which communication. The first tool is the profiler trace. Frameworks such as PyTorch can record a timeline of every kernel launched on every GPU, on both the compute stream and the communication streams. Viewed for a single step, a healthy trace shows long, dense blocks of matrix multiplications on the compute stream, with collectives running underneath them on the communication stream, starting and finishing within the shadow of the computation. A communication-bound trace shows gaps on the compute stream: stretches where no arithmetic kernel is running because the next one is waiting for a collective to finish. The length and placement of those gaps identify the culprit. Gaps at the end of the backward pass usually indicate data-parallel gradient reduction that was not fully overlapped. Gaps at the start of every layer suggest that sharded weight gathering is not being prefetched early enough. Short, regular gaps inside every layer point to tensor-parallel collectives. The second tool is a scaling experiment. Run the same configuration on progressively larger numbers of devices, keeping the per-device work constant, and measure throughput per device. Perfect weak scaling holds per-device throughput flat. A drop at a particular size often corresponds to a change in network topology: the point at which a data-parallel group first spans more than one leaf switch, for instance, or a pipeline first crosses a spine. Llama 3's slip in utilisation from 43 to 41 percent when the cluster doubled is a mild version of the same signal. The third tool is measuring the collectives in isolation. Benchmarks such as Nvidia's nccl-tests measure the bandwidth actually achieved by each collective at each message size across a given set of GPUs. Comparing these numbers with the headline link speeds reveals how much of the theoretical bandwidth the system is delivering. It is common for achieved bandwidth in a large collective to fall well short of the link speed because of switch contention, poorly balanced routing, or a single degraded cable. On large clusters, operators routinely run such tests across every new batch of hardware before admitting it to production jobs, because one slow link can drag down an entire training run. The diagnosis then suggests the remedy. If the gaps come from data-parallel reduction, larger buckets, earlier launches, or hierarchical collectives may help. If they come from tensor parallelism, the degree may be too high for the link, or the overlapping of collectives with chunked matrix multiplications may need to be turned on. If they come from the network itself, the fix may be physical: reseating a cable, replacing a transceiver, or rebalancing the routing tables. The roofline view A useful way to bring these ideas together is the roofline model, a standard tool of performance analysis. For any operation, compare the arithmetic it needs with the bytes it must move. If the ratio is high, the operation is limited by arithmetic and runs near peak. If low, it is limited by bandwidth, and adding arithmetic capability does nothing. Applied to distributed training, every parallel strategy has its own ratio, and every link its own bandwidth. Data parallelism's ratio grows with the batch per replica. Tensor parallelism's ratio falls as its degree rises. Pipeline parallelism's ratio is high but it has a bubble. The task of configuring a run is to keep every strategy on the arithmetic-bound side of its roofline for the link it uses. Each new hardware generation tends to raise arithmetic throughput faster than bandwidth, which pushes strategies towards the bandwidth-bound side and forces another round of hiding, shrinking and avoiding. This is why the communication bottleneck has not been solved and is unlikely to be: it is renewed with every generation of faster chips. Chapter 8: When Something Is Always Broken A distributed training job of the kind this book describes is, in a precise sense, a single program. Sixteen thousand GPUs execute one synchronised computation. Every data-parallel step waits for every replica; every pipeline waits for every stage; every tensor-parallel layer waits for every member of its group. The parallelism that makes the computation fast also makes it fragile. If any one device stops, the whole computation stops with it. And when the computation runs on tens of thousands of devices for months, devices stop all the time. This is the least glamorous part of large-scale training and, for the teams who run it, often the most consuming. The first seven chapters were about making a training step fast. This one is about keeping the steps coming. The arithmetic of failure Suppose a single GPU server is very reliable, failing on average once every few years. That sounds excellent. Now put two thousand such servers in a single synchronous job. If failures are independent, the expected time between failures for the job as a whole is the individual figure divided by two thousand. A few years becomes less than a day. At sixteen thousand GPUs, with every network card, cable, switch, power supply and memory module also able to fail, the interval shrinks to hours. Meta's Llama 3 paper gives the most detailed public account of what this looks like. During a 54-day snapshot of pre-training on up to 16,384 H100 GPUs, the job had 466 interruptions. Forty-seven were planned, for maintenance and configuration changes. The other 419 were unexpected. Of those, 58.7 percent were attributed to GPU issues, with faulty GPUs themselves accounting for 30.1 percent and their HBM3 memory for 17.2 percent. Software bugs, network problems and host failures made up much of the rest. Despite this, Meta reported more than 90 percent effective training time, meaning that for more than nine-tenths of the period the cluster was doing useful training work. Only three of the 419 interruptions required significant manual intervention; the rest were handled by automation. Google's Gemini report described the same problem in terms of goodput, the fraction of time spent making real forward progress. Through a series of improvements, the goodput of its largest training job rose from 85 to 97 percent. Earlier and smaller runs had it harder. Meta's logbook for the OPT-175B model, published openly in 2022 alongside the model, recorded a two-month run on 992 A100 GPUs punctuated by dozens of manual restarts and the cycling of more than a hundred hosts out of the cluster. These numbers make clear that reliability is a first-order concern in the economics of training. On a cluster that costs hundreds of millions of dollars, the difference between 85 percent and 97 percent goodput is the difference between wasting roughly one-seventh of the investment and wasting one-thirtieth. Checkpoints and their cost The basic defence against failure is the checkpoint. Periodically, the training state is written to durable storage. When a failure occurs, the job stops, the faulty hardware is removed or replaced, the job restarts on healthy hardware, loads the latest checkpoint and resumes. The work lost is everything computed since that checkpoint, plus the time to detect the failure, reallocate hardware, restart all the processes and reload the state. The cost of a checkpoint is the time spent writing it, during which, in a simple implementation, training pauses. Recall the size of the training state from Chapter 1: around sixteen bytes per parameter. For a 405-billion-parameter model, that is several terabytes, which must be written across the cluster to a parallel file system or object store. Even at high aggregate storage bandwidth, a synchronous checkpoint takes a meaningful amount of time. There is a classical trade-off here, studied for decades in high-performance computing. Checkpoint too often, and the time spent writing checkpoints dominates. Checkpoint too rarely, and each failure throws away a large amount of work. A widely cited approximation, due to John Young in 1974 and refined by John Daly in 2006, gives the optimal interval as roughly the square root of twice the checkpoint cost multiplied by the mean time between failures. If a checkpoint takes one minute and the job fails every three hours, the optimal interval is around nineteen minutes. As clusters grow, the mean time between failures falls and the optimal interval shrinks, which raises the overhead of checkpointing unless the checkpoint itself gets faster. So it has. The main techniques are now standard. Sharded checkpointing. Each device writes only its own shard of the state, in parallel. With ZeRO-style sharding, the optimiser state is already divided, and every device writes its own slice simultaneously, so the aggregate write bandwidth scales with the number of devices. Asynchronous checkpointing. Rather than pausing training while the checkpoint is written to remote storage, the device copies its state to CPU memory, which is fast, and then resumes training while a background process writes from CPU memory to storage. The pause shrinks to the time of a copy from GPU to host. PyTorch's distributed checkpointing supports this, as do the checkpointing systems described by ByteDance and others. In-memory redundancy. Google's Gemini report described keeping redundant copies of the model state in the memory of other devices, so that on an unplanned failure, the job could recover directly from an intact replica rather than reading from storage. Data parallelism, with its multiple replicas, provides some of this redundancy naturally, though ZeRO-style sharding removes much of it, and systems must deliberately keep extra copies of each shard. Fast restart. A large share of the time lost to a failure is spent not on recomputation but on restarting: detecting the failure, tearing down the job, scheduling a replacement, initialising tens of thousands of processes, establishing all the communication groups and loading the checkpoint. Each of these steps, trivial on a small cluster, can take minutes at scale. Production systems invest heavily in shortening them, with warm spare servers, cached initialisation and parallel loading. Detecting what is wrong Restarting from a checkpoint presupposes knowing that something has failed and which component failed. Neither is as simple as it sounds. A crash is the easy case: a process exits, a GPU reports an error, the job stops, and logs indicate where. The harder cases are hangs and slowdowns. If one GPU stops responding in the middle of a collective, every other GPU in that collective waits indefinitely. From the outside, sixteen thousand processes are all simply waiting, and the one that caused the wait looks no different from the rest. Llama 3's engineers described building tooling into their NCCLX library to record the state of every collective, so that when a hang occurred, they could identify which rank had failed to participate and in which operation. ByteDance's MegaScale paper similarly emphasised deep-stack observability, collecting fine-grained timing data from every device and every collective and visualising it so that operators could see, for instance, that one particular server was consistently arriving late. Stragglers are the subtlest problem. A device that is not broken but merely slow, because of thermal throttling, a degraded network link, a background process or a weak memory module, does not cause a failure. It slows every step, because synchronous training proceeds at the pace of the slowest participant. On a large cluster, the probability that at least one device is running a few percent slow at any given moment is high. Meta noted that a daily cycle of 1 to 2 percent variation in throughput tracked the time of day, which it attributed to temperature effects on GPU clock speeds. Finding stragglers requires comparing timings across devices step by step, identifying outliers, and either fixing or removing them. The MegaScale team described this as a central part of keeping utilisation high over long runs. Silent data corruption is the most insidious failure of all. A processor that computes a wrong answer without raising an error, a flipped bit in an arithmetic unit or a defect that appears only under certain temperatures or voltages, can corrupt the training state invisibly. Researchers at Meta, in a 2021 paper by Dixit and colleagues, and at Google, in a paper by Hochschild and colleagues the same year provocatively titled "Cores That Don't Count," documented that such defects occur at measurable rates in large fleets of processors. Google's Gemini report stated that at its scale, silent data corruption events could be expected to affect training every week or two. Defences include running deterministic computations twice and comparing results, checking that replicas that should be identical actually are, and screening hardware with targeted tests before admitting it to a production job. Keeping bad hardware out Detection after the fact is expensive, because every failure costs a restart and the work since the last checkpoint. The cheaper strategy is to stop bad hardware from joining the job in the first place, and to remove degrading hardware before it fails outright. Large operators therefore run new and returning servers through burn-in and qualification. A server is stressed with intensive computation, memory tests and collective benchmarks for hours or days before it is admitted to a production pool. Its results are compared against a fleet-wide baseline, and machines that run slower, hotter or less consistently than their peers are held back for investigation. The same tests run periodically on servers already in service, and between training jobs, to catch components that have begun to degrade. Health checks also run continuously during training. Each server's GPUs report error-correcting-code events in their memory, link errors on NVLink and the network, temperature excursions and clock throttling. A rising count of corrected memory errors, for instance, often precedes an uncorrectable one. Production systems watch these signals and proactively drain a server from the job at the next convenient checkpoint, rather than waiting for it to crash in the middle of a step. All of this depends on having spare capacity. A job that needs exactly sixteen thousand GPUs on a cluster of exactly sixteen thousand GPUs cannot replace a faulty server without waiting for a repair. Large clusters are therefore built with a pool of warm spares, servers kept ready to step in. The Llama 3 job ran on part of a cluster of roughly 24,000 GPUs, and the headroom serves this purpose among others. An alternative to spares is elasticity: allowing the job to continue on fewer devices when some fail, and to grow again when they return. PyTorch's elastic launcher and several research systems support this, adjusting the data-parallel degree on the fly. Elasticity is attractive in principle but awkward in practice for frontier training, because changing the number of data-parallel replicas changes the global batch size or the number of micro-batches per replica, which alters the optimisation or the pipeline schedule mid-run. Most frontier runs prefer to keep the layout fixed and swap hardware instead. Loss spikes and numerical failure Not every failure is a hardware failure. Large training runs also suffer from sudden numerical instabilities, where the loss, after falling smoothly for weeks, jumps sharply upward. Sometimes it recovers; sometimes it diverges entirely. Google's PaLM paper described about twenty such spikes during the training of its 540-billion-parameter model, despite gradient clipping. Its remedy was pragmatic: restart from a checkpoint about a hundred steps before the spike and skip roughly two hundred to five hundred batches of data around the point where the spike occurred. The authors found that the spikes were not caused by bad data as such but by a particular interaction between data and model state at that point in training. The OPT logbook records similar episodes, handled by restarting from earlier checkpoints with lowered learning rates. Loss spikes sit at the boundary between the systems problem and the modelling problem. Some are caused by numerical precision, which interacts with distributed training through the choice of formats for gradients and optimiser states. Some are caused by architectural features, and techniques such as normalising query and key vectors in attention have been introduced partly to suppress them. From the operational point of view, they look much like hardware failures: training must be rolled back to a checkpoint and resumed, losing the work in between. They add to the case for frequent, fast checkpoints. DeepSeek's statement that its V3 run required no rollbacks at all was notable precisely because rollbacks had come to be expected. Power, heat and the facility At the largest scale, the training job interacts with the physical facility in ways that would surprise anyone who has only trained on a single machine. Synchronous training is bursty. During computation, every GPU draws near its maximum power. During a collective or a checkpoint, many GPUs briefly idle. When sixteen thousand GPUs move between these phases together, the power draw of the whole facility swings by a large amount in a fraction of a second. Meta reported swings of tens of megawatts, enough to stress the power grid connection. Such swings can be mitigated in software, by keeping GPUs partly busy during idle phases, and in hardware, with energy storage that smooths the draw. Heat follows power. GPUs that run hot throttle their clocks, becoming stragglers. Modern rack-scale systems such as the GB200 NVL72 are liquid-cooled, partly because air cooling cannot remove the heat of seventy-two high-power GPUs in a single rack. And as single training jobs have grown to consume a substantial share of a data centre's power, the question of where a large enough site can be built, with enough power and cooling, has become a constraint on how big a single synchronous training job can grow. This is one of the forces pushing interest in training across multiple data centres, with the reduced-communication methods described in Chapter 7. Reliability as part of the design The common thread is that at large scale, reliability cannot be added afterwards. The choice of parallelism layout affects how a failure propagates and how quickly the job can recover. The choice of checkpoint strategy depends on how state is sharded. The ability to find stragglers depends on instrumenting every collective. The willingness to adopt a new optimisation, such as FP8 or gradient compression, depends on how much it threatens numerical stability over months of training. A training system is therefore judged not only by its peak throughput in a clean benchmark but by its goodput over the life of a real run: the fraction of all the hardware-hours paid for that turned into useful progress on the model. The difference between those two measures, peak and sustained, is the operational counterpart of the gap between theoretical and achieved utilisation that ran through the earlier chapters. Both are measures of the same thing: how much of a very expensive machine's capacity actually reaches the model. Conclusion The techniques in this book can look, on first encounter, like a catalogue of unrelated tricks: a ring algorithm here, a sharding scheme there, a pipeline schedule with a clever name. Having followed them through, it is possible to see that they are all answers to the same question, asked under different constraints. Where should the work go, given that memory is scarce on every device, arithmetic is plentiful but must be kept fed, and bandwidth falls by an order of magnitude at every step outward from the chip? Data parallelism answers by copying the model and dividing the data, spending batch size. Sharding answers by lending the model's state rather than copying it, spending a little extra communication. Tensor parallelism divides each layer, spending the fastest bandwidth in the system. Pipeline parallelism divides the depth, spending schedule time. Context parallelism divides the sequence; expert parallelism divides the experts and spends unpredictable all-to-all traffic. The multi-dimensional layouts used for frontier models put each of these on the part of the network that can best afford it. The reliability machinery of the last chapter protects the whole arrangement against the certainty that some part of it is broken at any moment. Every one of these is a trade among memory, arithmetic and bandwidth. That is the sense in which training at scale is a communication problem: the arithmetic is the easy part, and nearly all the engineering lies in arranging for the bytes to be in the right place when the arithmetic needs them. What follows from this Several conclusions follow that are not obvious from any single technique. The first is that the network shapes the model, not only the other way round. The rule that tensor parallelism stays within the NVLink domain has, for years, nudged model designers towards hidden dimensions and head counts that divide neatly by eight. DeepSeek's node-limited routing constrained a model's mixture-of-experts layer so that its traffic would fit the network. Grouped-query attention reduced the size of what context parallelism must exchange. As the cost of training has grown, model architecture has become something designed jointly with the hardware and the parallel layout rather than handed to systems engineers to accommodate. Anyone evaluating a new architecture at scale should ask not only how well it learns but how it communicates. The second is that hardware progress makes the problem harder, not easier. Each generation of accelerators has raised arithmetic throughput faster than memory bandwidth, and memory bandwidth faster than network bandwidth. FP8 and smaller formats raise effective arithmetic throughput further still. The result is that the amount of computation each device must do per byte communicated, to stay efficient, keeps rising. Strategies that were comfortably compute-bound on one generation drift towards being bandwidth-bound on the next. This is why the techniques for hiding, shrinking and avoiding communication are not a one-time fix but a permanent part of the field, renewed with every product cycle. The third is that the scale-up domain is the decisive unit. For years, the boundary between fast and slow communication was the eight-GPU server, and almost every configuration choice deferred to it. Rack-scale systems with seventy-two GPUs on a single NVLink fabric move that boundary outward by an order of magnitude, and announced successors propose to move it further. A larger fast domain relaxes nearly every constraint in this book: it permits wider tensor parallelism, keeps more expert traffic local, shortens pipelines and reduces the number of data-parallel replicas competing for the slow network. How much of the scale-out network's burden migrates into ever-larger scale-up domains is likely to be the most consequential architectural question for training systems over the next several years. The fourth is that utilisation and goodput are economic quantities, not engineering vanity metrics. A well-run frontier job achieving around 40 to 55 percent model FLOPs utilisation and 90 percent or more goodput still leaves roughly half the theoretical capacity of a very expensive machine unused. Each improvement, whether a better pipeline schedule, a faster checkpoint or a more efficient collective, translates directly into a cheaper model or a larger one for the same budget. DeepSeek-V3's widely discussed training cost was not the product of a single breakthrough but of many such improvements compounded: FP8, DualPipe, node-limited routing, carefully engineered communication kernels and no rollbacks. What remains unsettled Several questions are open, and the reader should treat confident claims about them with care. It is not yet clear how far synchronous training can scale on a single site. Power and cooling limits are already shaping where the largest clusters can be built, and the methods for training across multiple data centres, such as the reduced-communication approaches discussed in Chapter 7, have been demonstrated at meaningful but not yet frontier scale in public. Whether frontier training moves to a looser, partly asynchronous regime, or whether single sites keep growing, is an open question with large consequences for how training systems are built. It is not settled how low precision can go. FP8 training is now demonstrated at frontier scale. Four-bit formats have hardware support and active research behind them. Each step down reduces memory and communication but adds numerical risk, and the boundaries are being found empirically, run by run. Nor is it clear how training systems will cope with changing workloads. Reinforcement learning stages after pre-training, now a large and growing share of total compute for many leading models, mix training with large-scale generation, which has a very different communication profile. Long-context training continues to push context parallelism. Mixture-of-experts models make expert parallelism and its all-to-all traffic a central concern. The parallelism strategies described here will remain the building blocks, but the way they are composed will keep changing. What to do differently For an engineer or researcher beginning to work at scale, a few practical habits follow from the argument of this book. Do the arithmetic first. Before choosing a framework or a configuration, calculate the training state in bytes, the activation memory for the intended sequence length and micro-batch, and the total compute from 6ND. These numbers decide which constraints bind and therefore which strategies are needed. Most configuration mistakes are visible on the back of an envelope. Start simple and add dimensions only when forced. Data parallelism with sharded optimiser state solves a remarkable range of problems. Add full sharding when memory requires it, tensor parallelism when single layers or activations will not fit, pipeline parallelism when the model must span servers, and context or expert parallelism when the data or the architecture demands them. Each added dimension brings complexity and new failure modes. Map the logical layout to the physical network deliberately. Know the size of the scale-up domain, the bandwidth of each link, and how the scale-out network is built. Put the most frequent, least overlappable traffic on the fastest links. Measure utilisation and goodput honestly, and instrument everything. A training run without per-device, per-collective timing is a run in which stragglers, hangs and slow links will remain invisible until they have cost a great deal. Treat reliability as a design requirement from the outset. Plan checkpoint frequency against realistic failure rates, test the restart path before it is needed, and assume that at scale every component will fail during the run. The models that dominate public attention are, in the end, mathematical objects: arrays of numbers adjusted by gradient descent. But the ability to produce them at the sizes now possible rests on a large body of distributed-systems engineering, most of it invisible in the finished model. The engineers who build that machinery work under constraints that will tighten rather than loosen as hardware improves. Understanding those constraints, memory, arithmetic and bandwidth, and the trades among them, is the foundation for understanding how large models are made, and what it will take to make the next ones. Hashtags: #DistributedDeepLearning #DeepLearning #DistributedTraining #LargeScaleModelTraining #MassiveAIModels #ArtificialIntelligence #MachineLearning #LargeLanguageModels #NeuralNetworks #HighPerformanceComputing #GPUComputing #MultiGPUTraining #DataParallelism #ModelParallelism #TensorParallelism #PipelineParallelism #SequenceParallelism #ContextParallelism #ExpertParallelism #DistributedDataParallel #FullyShardedDataParallel #FSDP #DeepSpeed #ZeROOptimizer #MegatronLM #GradientSynchronization #AllReduce #MixedPrecisionTraining #TrainingOptimization #ScalableAI

Download the Book (PDF): Introduction In June 2017 eight researchers, most of them at Google, posted a paper with a title that sounded more like a slogan than a scientific claim: "Attention Is All You Need." Its proposal was narrow. For the task of translating sentences from English into German and French, it replaced the recurrent neural networks that then dominated the field with a stack of layers built almost entirely from one operation, which the authors called scaled dot-product attention. The model trained faster than its competitors and translated better. On the standard English-to-German benchmark it scored 28.4 BLEU, ahead of every previously published system, including ensembles. Nothing in that result announced a revolution. Translation benchmarks had been broken many times before. What followed was different. Within eighteen months the same architecture, lightly modified, had become the basis of BERT, which reset the state of the art across a broad suite of language-understanding tasks, and of GPT and GPT-2, which showed that a model trained only to predict the next word could produce coherent paragraphs. Within three years it had been applied to images, by cutting them into patches and treating the patches as words. Within five it was folding proteins, transcribing speech, writing code, and serving as the core of conversational systems used by hundreds of millions of people. Today, nearly every large model that attracts public attention, whether it generates text, images, audio, or video, has a transformer, or something designed explicitly as a transformer's successor, at its centre. This booklet explains how that architecture works and why it changed in the ways it did. It is not a history of the companies or a survey of applications. It is an account of one mechanism and the pressures that shaped it. The controlling idea The argument of this booklet can be stated in a sentence. The transformer is best understood as a solution to a problem of information routing, and nearly every important change to it since 2017 has been a negotiation between the expressive power of letting every position consult every other position and the cost, in arithmetic and above all in memory, of doing so. Before the transformer, sequence models moved information along a chain. To get a fact from the first word of a paragraph to the last, a recurrent network had to carry it through every intermediate step, compressed into a fixed-size vector that was overwritten a little at each position. Attention replaced the chain with a switchboard. At every layer, every position can draw directly on every other, with the connections chosen by the content of the positions themselves rather than by their distance apart. That one change is the source of the transformer's strengths: it trains in parallel across a whole sequence, it can relate distant words as easily as adjacent ones, and it scales smoothly as models grow. The same change is also the source of its central weakness. A switchboard that connects every position to every other has a number of connections that grows with the square of the sequence length. Double the context and the attention computation quadruples. More subtly, a model that generates text one token at a time must keep a record, for every past token, of what that token offers to future queries. This record, the key-value cache, grows linearly with context and dominates the memory budget of long-context inference. Much of the architectural history of the past several years, from multi-query attention to FlashAttention to state-space models, makes sense only once you see which of these costs it was trying to reduce, and what it gave up in exchange. A second, quieter thread runs through the same story. Attention, by itself, has no sense of order. It treats its input as a set. The sentence "the dog bit the man" and "the man bit the dog" contain the same words, and pure attention cannot tell them apart. Every transformer therefore needs some way to inject position, and the choice of method turns out to govern whether a model can handle sequences longer than those it was trained on. The progression from fixed sinusoids to learned embeddings to rotary embeddings to the interpolation tricks that stretch a model's context from four thousand tokens to a million is the second half of the long-context story. What the reader will need The booklet assumes curiosity and a willingness to read an equation slowly, not a background in machine learning. The mathematics is mostly linear algebra at the level of matrix multiplication and dot products, plus the softmax function, which is defined when it first appears. Equations are written inline in plain text, such as Attention(Q, K, V) = softmax(QK^T / sqrt(d_k)) V, where the caret marks a transpose and sqrt is the square root. Every equation is also explained in words, and where a calculation matters to the argument, it is worked through with actual numbers. A few conventions recur. The letter n stands for the length of a sequence, measured in tokens; a token is a word or a fragment of a word, and in English text a typical modern tokenizer produces roughly three tokens for every four words. The letter d, often written d_model, stands for the width of the model, the number of components in the vector that represents each token as it passes through the network. The letter L counts layers, and h counts attention heads. Sizes of models are given in parameters, the individual numbers learned during training. The path through the material The chapters build in order. The first describes the problem the transformer solved: the sequential bottleneck of recurrent networks and the earlier form of attention that was invented to patch it. The second derives scaled dot-product attention from first principles, including the reason for the square root that gives it its name. The third assembles the full transformer block, explaining why there are many attention heads rather than one and what the feed-forward layers, residual connections, and normalization are for. The fourth chapter treats position: how a set-processing operation is made to respect order, and why rotary embeddings became the default. The fifth follows the single block as it divided into three families, encoder-only, decoder-only, and encoder-decoder, and explains why the decoder-only design won. The sixth asks what attention heads actually learn, drawing on interpretability research that has identified specific circuits, such as induction heads, inside trained models. The seventh chapter turns to scale: the empirical laws that relate model size, data, and compute to performance, and the mixture-of-experts designs that decouple parameter count from computation. The eighth confronts the quadratic wall directly, counting the floating-point operations and bytes of memory that attention consumes and showing where the true bottleneck lies. The ninth describes the engineering that made exact attention fast enough to survive, including FlashAttention and the progressive shrinking of the key-value cache. The tenth examines the many attempts to approximate attention more cheaply and explains why most of them did not displace the original. The eleventh considers the strongest challengers, state-space models and their relatives, along with the hybrid designs that combine them with attention, and asks what "long context" actually delivers when it is measured carefully. The conclusion draws these threads into an assessment of where the architecture stands and what remains genuinely unresolved. A note on currency The field moves fast, and specific model names and context lengths will date. The underlying arithmetic will not. The cost of attention grows quadratically with sequence length for reasons that follow from its definition. The key-value cache grows linearly for reasons that follow from autoregressive generation. Position must be supplied from outside because attention is permutation-invariant. These facts are the fixed points around which the architecture has moved, and a reader who understands them will be able to make sense of designs that have not yet been published. That, more than any catalogue of current systems, is what this booklet tries to provide. Chapter 1. The Sequential Bottleneck Language arrives in order. A sentence is a sequence of words, a paragraph a sequence of sentences, and meaning depends on that order at every scale. Any machine that processes language must decide how to represent a sequence and how to let information flow along it. The transformer's answer is easiest to appreciate against the answer it replaced, and against the specific failures of that answer that a decade of clever engineering could not fully repair. Reading one word at a time The natural way to build a neural network for sequences is to mimic a human reader who moves left to right, keeping a running summary in mind. This is the recurrent neural network, or RNN. At each position t, the network takes two inputs: the current token, represented as a vector x_t, and its own summary from the previous step, a vector h_(t-1) called the hidden state. It combines them to produce a new hidden state: h_t = tanh(W_h h_(t-1) + W_x x_t + b) Here W_h and W_x are matrices of learned weights, b is a learned bias vector, and tanh is a function that squashes each component into the range from minus one to one. The same weights are used at every position. After the last token, the final hidden state h_n is meant to contain everything the network needs to know about the whole sequence. This design has real elegance. It handles sequences of any length with a fixed number of parameters. It has a built-in notion of order, since the network literally processes tokens in sequence. And it has a plausible story about memory: the hidden state is a working buffer, updated as new evidence arrives. The trouble begins with training. Neural networks learn by gradient descent: the error at the output is propagated backward through every operation that produced it, telling each weight how to change. In a recurrent network, the path from an early token to the final output passes through the recurrence once for every intervening position. The gradient that flows back along that path is multiplied, at each step, by the matrix W_h and by the derivative of tanh. If the relevant quantities are on average a little smaller than one, the product shrinks exponentially with distance; if a little larger, it explodes. Sepp Hochreiter identified this problem in his 1991 diploma thesis, and Yoshua Bengio, Patrice Simard and Paolo Frasconi analysed it formally in 1994. The practical consequence was blunt. A plain RNN could learn dependencies spanning a few steps but struggled with anything spanning dozens. Gates and the long short-term memory The most influential repair was the long short-term memory network, or LSTM, introduced by Hochreiter and Jürgen Schmidhuber in 1997 and refined by others over the following years, notably by the addition of a "forget gate" by Felix Gers and colleagues in 2000. The LSTM adds a second internal vector, the cell state c_t, which is updated additively rather than by repeated matrix multiplication. Three gates, each a vector of numbers between zero and one computed from the current input and previous hidden state, control the flow. The forget gate decides how much of the old cell state to keep. The input gate decides how much new information to write. The output gate decides how much of the cell state to expose as the hidden state. The key equation is the cell update: c_t = f_t c_(t-1) + i_t g_t where f_t is the forget gate, i_t the input gate, g_t a candidate update, and the asterisk means element-wise multiplication. When the forget gate is close to one and the input gate close to zero, the cell state passes through unchanged, and so does its gradient. The LSTM therefore has a protected channel along which information, and the error signal needed to learn it, can travel across long stretches. A closely related simplification, the gated recurrent unit or GRU, was proposed by Kyunghyun Cho and colleagues in 2014 and performs comparably on many tasks. Gated recurrent networks worked. From roughly 2013 to 2017 they were the dominant architecture for speech recognition, handwriting recognition, language modelling and machine translation. Google's production translation system, described in a 2016 paper by Yonghui Wu and colleagues, used a stack of eight LSTM layers in its encoder and eight in its decoder. But the gates mitigated the long-range problem rather than solving it, and they did nothing about a second limitation that would turn out to matter even more. The parallelism problem A recurrent network cannot compute h_t until it has computed h_(t-1). This is true during training as well as during use. Given a training sentence of fifty tokens, the network must perform fifty sequential steps, each waiting on the last, before it can compute the loss and begin backpropagation, which then also runs sequentially in reverse. This matters because of the hardware on which neural networks are trained. A graphics processing unit is built to perform a very large number of identical arithmetic operations simultaneously. Its natural workload is a large matrix multiplication, in which thousands of independent dot products can be computed at once. A recurrent network offers the GPU a long series of small matrix multiplications, each dependent on the previous one. Within a single step there is parallelism across the hidden state's components and across the examples in a batch, but along the sequence there is none. The longer the sequence, the more the GPU waits. Researchers worked around this with large batches, careful kernel engineering, and architectures that limited recurrence to a few layers. But the fundamental constraint remained: the amount of sequential computation grew linearly with sequence length, and no amount of hardware could remove it. When the transformer paper later compared architectures, it singled out this property. A recurrent layer requires a number of sequential operations proportional to n. A self-attention layer requires a constant number, independent of n. That single line in the paper's comparison of layer types turned out to be among the most consequential observations in it. Convolutional sequence models, such as ByteNet and ConvS2S, both published in 2016 and 2017, tackled the parallelism problem from another direction. A convolution looks at a fixed window of neighbouring tokens and can be applied at every position simultaneously. Stacking convolutions widens the window. These models trained much faster than LSTMs and matched or beat them on translation. But relating two tokens far apart still required many layers, a number growing linearly with distance for ordinary convolutions or logarithmically for dilated ones. The path between distant tokens was shorter than in an RNN but still not direct. The encoder-decoder and its fixed-size bottleneck Machine translation exposed a third limitation, and it was the one whose solution became the transformer's central idea. In 2014 two groups independently proposed the sequence-to-sequence framework for translation. Ilya Sutskever, Oriol Vinyals and Quoc Le at Google, and Kyunghyun Cho and colleagues at the University of Montreal, built systems with two recurrent networks. The encoder read the source sentence and produced a final hidden state. The decoder took that state as its starting point and generated the target sentence one word at a time, each step conditioned on its own previous outputs. The design was simple and, with enough data and some tricks, effective. Sutskever and his colleagues found, for instance, that reversing the order of the source sentence improved results, because it placed the first words of the source close to the first words of the target and shortened the path between them. That such a trick helped was itself a diagnostic. The entire meaning of the source sentence, however long, had to pass through a single vector of fixed size, the encoder's final hidden state. A vector of a thousand numbers can hold a great deal, but it cannot hold an arbitrarily long sentence without loss, and Cho's group showed that translation quality fell off sharply as sentence length grew beyond about twenty words. Attention as a patch The fix came from Dzmitry Bahdanau, working with Cho and Bengio, in a paper posted in September 2014 and presented at the 2015 International Conference on Learning Representations. Its title stated the idea: "Neural Machine Translation by Jointly Learning to Align and Translate." Instead of compressing the source sentence into one vector, Bahdanau's encoder kept every hidden state, one per source word. When the decoder was about to produce a target word, it compared its current state against each of those encoder states and computed a score for each. The scores were converted into weights that summed to one, and the decoder received a weighted average of the encoder states, called the context vector. Different target words could thus draw on different parts of the source. When translating a French adjective that follows its noun, the decoder could look back at the English adjective that precedes it. The conversion from scores to weights used the softmax function, which will appear constantly in what follows. Given a list of real numbers s_1 through s_n, softmax returns a list of the same length whose i-th entry is exp(s_i) divided by the sum of exp(s_j) over all j. Every output is positive, the outputs sum to one, and larger inputs get disproportionately larger outputs. Softmax turns arbitrary scores into a probability distribution that concentrates on the highest-scoring items while giving some weight to the rest. In Bahdanau's design the score between decoder state s and encoder state h_j was computed by a small neural network: v^T tanh(W s + U h_j), with v, W and U learned. This is now called additive attention. A year later, Minh-Thang Luong, Hieu Pham and Christopher Manning at Stanford compared several scoring functions and found that a simpler one, the dot product between the decoder state and the encoder state, possibly with a learned matrix in between, worked as well or better and was cheaper to compute. This is multiplicative or dot-product attention, and it is the direct ancestor of the transformer's version. The effect of attention on translation was large. It removed the fixed-size bottleneck, improved quality on long sentences, and produced, as a by-product, alignment weights that could be inspected. Plotting the weight each target word placed on each source word yielded a matrix that, for language pairs with similar word order, lay close to the diagonal, with deviations exactly where the languages reorder words. It was one of the first times a neural translation system offered an interpretable glimpse of what it was doing. From patch to foundation In these systems attention was an addition to a recurrent architecture, not a replacement for it. The encoder and decoder were still LSTMs, still sequential, still subject to the parallelism limit. Attention connected the decoder to the encoder, but within each, information still travelled step by step. Several lines of work in 2016 began to use attention within a single sequence. Jianpeng Cheng, Li Dong and Mirella Lapata proposed an LSTM variant that attended over its own previous states, which they called intra-attention. Ankur Parikh and colleagues at Google built a model for natural-language inference that used attention between two sentences and almost no recurrence at all, and achieved strong results with an order of magnitude fewer parameters than competing systems. Zhouhan Lin and colleagues proposed a "structured self-attentive sentence embedding." The idea that a sequence might attend to itself, and that this might substitute for recurrence, was in the air. The transformer paper's contribution was to take that idea to its conclusion. If attention lets the decoder reach any encoder position directly, and if a sequence can attend to itself, then perhaps recurrence is unnecessary. Every layer could consist of self-attention, in which each position computes a weighted average over all positions in the same sequence, followed by a simple position-wise transformation. Information would travel between any two positions in a single layer. All positions could be computed at once. The path length between distant tokens, which had been the core problem since Hochreiter's thesis, would be constant. The transformer's authors made the comparison explicit with three measures for each layer type: the total computation per layer, the number of operations that must happen sequentially, and the maximum path length between any two positions. For self-attention, computation per layer grows as n squared times d, sequential operations are constant, and maximum path length is one. For a recurrent layer, computation grows as n times d squared, sequential operations grow as n, and path length grows as n. For a convolution with kernel width k, computation grows as k times n times d squared, sequential operations are constant, and the path length grows as the logarithm of n to the base k when dilated convolutions are used. Two features of that comparison set up everything that follows. The first is how decisively self-attention wins on the second and third columns. It is the only layer type that is both fully parallel and gives every pair of positions a direct connection. The second is the quiet presence of n squared in the first column. When sentences were thirty tokens long and the model width was 512, n squared times d was smaller than n times d squared, so self-attention was cheaper than recurrence as well as faster. The authors noted that this held as long as n was smaller than d, "which is most often the case" for the sentence-level tasks of the time. They also suggested that for very long sequences attention could be restricted to a neighbourhood around each position. That caveat was reasonable in 2017. It would become the defining engineering problem of the architecture as context windows grew from hundreds of tokens to hundreds of thousands. Before turning to it, though, we need to understand exactly what the attention computation does, and why its particular form, including a scaling factor that might look like an afterthought, is what it is. Chapter 2. Scaled Dot-Product Attention The heart of the transformer is a single formula: Attention(Q, K, V) = softmax(QK^T / sqrt(d_k)) V It fits on one line and contains four operations: a matrix multiplication, a division, a softmax, and another matrix multiplication. Everything else in the architecture exists to feed this formula good inputs, to make use of its outputs, or to make it cheaper to compute. This chapter takes it apart piece by piece, works through it with numbers, and explains the one detail, the division by sqrt(d_k), that gives it its name. A soft dictionary The easiest way into attention is through an ordinary data structure. A dictionary, or hash map, stores pairs of keys and values. You present a query; the dictionary finds the key that matches it exactly and returns the associated value. If you look up "Paris" in a dictionary of capitals keyed by city, you get "France." If the query matches no key, you get nothing. Attention is a dictionary in which matching is graded rather than exact. Every stored item has a key and a value, both vectors. A query is also a vector. Instead of finding the one key that equals the query, attention measures how similar the query is to every key, converts those similarities into weights that sum to one, and returns the weighted average of all the values. If one key is much more similar to the query than the others, the result is close to that key's value, and attention behaves like an ordinary lookup. If several keys are similar, the result blends their values. The lookup is soft, and because every step is a smooth function, the whole operation can be differentiated and learned by gradient descent. In self-attention, every token in a sequence plays all three roles. Each token emits a query, which expresses what it is looking for. Each token emits a key, which advertises what it contains. Each token emits a value, which is what it will contribute if selected. Each token's query is compared with every token's key, and each token receives a weighted average of every token's value. The output for each position is therefore a mixture of information from the whole sequence, with the mixture chosen by content. Statisticians will recognise the structure. A weighted average of values, with weights determined by a similarity kernel between a query point and stored points, is the Nadaraya-Watson kernel regression estimator from the 1960s. Attention is a kernel smoother whose kernel is the exponential of a scaled dot product, and whose queries, keys and values are all learned projections of the data. Where queries, keys and values come from Suppose a sequence has n tokens, and each has been turned into a vector of d_model numbers. Stack those vectors as the rows of a matrix X, which therefore has n rows and d_model columns. The transformer learns three weight matrices, W_Q, W_K and W_V. Multiplying X by each gives: Q = X W_Q, K = X W_K, V = X W_V If W_Q and W_K have d_model rows and d_k columns, then Q and K have n rows and d_k columns: one query vector and one key vector of length d_k per token. Similarly V has n rows and d_v columns. In the original transformer, d_model was 512, and with eight attention heads, each head had d_k = d_v = 64. The reason for heads comes in the next chapter. For now, consider a single one. The separation into three projections is not decoration. If queries and keys were the same vectors, the similarity between token i and token j would be symmetric: i would attend to j exactly as strongly as j attends to i. Language is not like that. A pronoun needs to find its antecedent, but the antecedent has no particular need to find the pronoun. Separate query and key projections allow asymmetric relationships. The separate value projection lets what a token contributes differ from what it advertises: a token might be found because it is a noun in subject position, but contribute information about its meaning. The four steps With Q, K and V in hand, attention proceeds as follows. First, compute every pairwise similarity at once with the matrix product QK^T. Q has n rows and d_k columns; K^T, the transpose of K, has d_k rows and n columns. Their product is an n-by-n matrix, often called the attention score matrix, whose entry in row i and column j is the dot product of query i with key j. The dot product of two vectors is the sum of the products of their corresponding components; it is large and positive when the vectors point in similar directions, near zero when they are unrelated, and negative when they point in opposite directions. Second, divide every entry by sqrt(d_k). This is the scaling, discussed below. Third, apply softmax to each row separately. Row i holds the scores of query i against all keys; softmax turns it into n positive weights summing to one. The resulting n-by-n matrix is the attention weight matrix, often written A. Its entry A_ij is how much position i attends to position j. Fourth, multiply A by V. Row i of the product is the sum over j of A_ij times value vector j: the weighted average of the values, using query i's weights. The output is an n-by-d_v matrix, one new vector per position. A worked example Numbers make this concrete. Take a sequence of three tokens with d_k = d_v = 2, and suppose the projections have produced: Queries: q1 = (1, 0), q2 = (0, 1), q3 = (1, 1) Keys: k1 = (1, 0), k2 = (0, 1), k3 = (1, 1) Values: v1 = (1, 0), v2 = (0, 2), v3 = (3, 1) Consider the first token. Its query (1, 0) dotted with the three keys gives 1, 0 and 1. Dividing by sqrt(2), approximately 1.414, gives 0.707, 0 and 0.707. Exponentiating gives about 2.028, 1.000 and 2.028, which sum to 5.056. Dividing each by the sum gives weights of 0.401, 0.198 and 0.401. The first token attends equally to itself and to the third token, and about half as much to the second. Its output is 0.401 times (1, 0) plus 0.198 times (0, 2) plus 0.401 times (3, 1). The first component is 0.401 + 0 + 1.203 = 1.604. The second is 0 + 0.396 + 0.401 = 0.797. The new representation of token one is (1.604, 0.797), a blend dominated by the first and third values. The same procedure applied to the third token, whose query (1, 1) gives dot products of 1, 1 and 2, yields scaled scores of 0.707, 0.707 and 1.414, weights of roughly 0.248, 0.248 and 0.503, and an output that leans toward value three. Every token gets its own mixture, and all three mixtures can be computed simultaneously because the whole operation is two matrix products and a row-wise softmax. This example also shows what attention cannot do by itself. Nothing in the computation refers to the order of the tokens. If we swapped tokens one and two, swapping their queries, keys and values together, the outputs would swap in exactly the same way and otherwise be unchanged. Attention is permutation-equivariant: reorder the input and the output is reordered identically. This is why the transformer needs position information injected from outside, the subject of Chapter 4. Why divide by the square root of d_k The scaling factor is the one part of the formula that looks arbitrary, and the transformer paper's explanation of it is a short footnote. The reasoning behind it is worth spelling out, because it is an instance of a pattern that recurs throughout deep learning: keeping the numbers inside a network at a scale where gradients can flow. Suppose the components of a query vector and a key vector are independent random variables, each with mean zero and variance one. This is roughly the situation at the start of training, when weights are random, and something like it persists afterward because of normalization layers. The dot product q · k is a sum of d_k terms, each the product of two independent unit-variance variables. Each such product has mean zero and variance one. A sum of d_k independent terms with variance one has variance d_k, so the dot product has standard deviation sqrt(d_k). With d_k = 64, the typical dot product therefore has magnitude around 8, and in a row of many scores it is common for the largest to exceed the others by 10 or 20. Consider what softmax does to a row of scores 8, 0 and minus 8. The exponentials are about 2981, 1, and 0.000335. The weights are about 0.99966, 0.00034, and 0.0000001. The softmax has become nearly a hard maximum, putting essentially all its weight on one entry. That would be harmless if it reflected a genuine preference the model had learned. The problem is what it does to learning. The derivative of softmax output i with respect to input j is p_i times (1 minus p_i) when i equals j, and minus p_i times p_j otherwise. When one probability is nearly one and the rest nearly zero, every one of these terms is nearly zero. The gradient vanishes. A model whose attention saturates at initialisation cannot learn to adjust it, because the error signal that would tell it how cannot pass back through the softmax. Dividing the scores by sqrt(d_k) restores them to unit variance, regardless of the head dimension. With d_k = 64 the factor is 8, and the scores 8, 0 and minus 8 become 1, 0 and minus 1, which softmax maps to about 0.665, 0.245 and 0.090. The distribution is still peaked toward the highest score but far from saturated, and gradients flow. The authors observed that, without scaling, dot-product attention performed worse than additive attention at large d_k, and attributed this to exactly the saturation effect described here. The scaling can be read another way. Dividing logits by a constant before softmax is equivalent to raising the softmax's temperature, a term borrowed from statistical physics. High temperature flattens a distribution; low temperature sharpens it. The factor 1/sqrt(d_k) sets the temperature so that, at initialisation, attention is neither uniform nor one-hot. The model can then learn to sharpen it where sharpness helps, by growing the norms of particular queries and keys. This choice is a convention, not a law. Some later work has replaced the fixed factor with a learned temperature, or normalised queries and keys to unit length before taking the dot product and then multiplied by a learned scale, a technique called QK-normalisation that has been used to stabilise training of very large models whose attention logits otherwise grow without bound. These variants all address the same concern the square root addresses: controlling the magnitude of the softmax's inputs. Masking Two practical additions complete the operation. Sequences in a batch often differ in length, and shorter ones are padded with placeholder tokens. Attention should ignore these. The standard method is to add a large negative number, conceptually minus infinity, to the scores for padded positions before the softmax. Since exp of minus infinity is zero, those positions receive zero weight. The second form of masking is more fundamental. A language model that generates text predicts each token from the tokens before it. During training, the model is shown a whole sequence and asked to predict every token from its predecessors at once, which is what makes training parallel. But if position i could attend to position i + 1, it could simply copy the answer. The solution is a causal mask: add minus infinity to every score where the key position comes after the query position. In the n-by-n score matrix this blanks out everything above the main diagonal. Row one can attend only to position one; row two to positions one and two; row n to everything. A single matrix computation then produces n correctly causal predictions at once. This trick, which lets a model trained on a document of a thousand tokens learn from a thousand prediction problems in one parallel pass, is a large part of why transformer language models train so efficiently. The shape of the cost The worked example hid the most important property of attention under its small size. The score matrix QK^T is n by n. For three tokens it has nine entries. For a thousand tokens it has a million. For a hundred thousand tokens it has ten billion, per head, per layer. Computing it requires, for each of the n squared entries, a dot product of length d_k, so about n squared times d_k multiply-add operations. Multiplying the weight matrix by V costs about the same. Storing the matrix, if it is materialised in memory, costs n squared numbers. Both grow with the square of the sequence length. By contrast, the projections that produce Q, K and V cost about n times d_model times d_k each: linear in n. So does every other part of the transformer layer. Attention is the only component whose cost grows faster than the sequence. At short lengths this hardly matters, because the linear parts, multiplied by large model widths, dominate. At long lengths the quadratic part takes over. Where the crossover occurs, and why the memory cost of attention has turned out to matter even more than its arithmetic, is the subject of Chapter 8. For now it is enough to see the trade the formula makes. In exchange for an n-by-n matrix, every position gets direct, content-addressed access to every other position, in a computation that is entirely parallel. That trade made the transformer possible, and its price has shaped every revision since. Chapter 3. Many Heads and the Transformer Block A single attention operation gives each position one weighted average over the sequence. That is powerful but restrictive. A word in a sentence typically stands in several relationships at once. In "The engineer who built the bridge retired last spring," the verb "retired" needs to find its subject, "engineer," skipping over the relative clause; it also has a tense relationship with "last spring"; and the model may benefit from knowing what the nearest preceding word is, for reasons of local syntax. One softmax distribution must spread its weight across all of these needs, and a weighted average of three unrelated values is a blur rather than a combination. The transformer's solution is to run several attention operations in parallel, each with its own projections, and concatenate the results. This is multi-head attention. Around it, the transformer builds a block with three more components, each solving a specific problem: a position-wise feed-forward network, residual connections, and layer normalization. Stacking that block is the whole architecture. Multi-head attention Multi-head attention with h heads uses h separate sets of projection matrices: W_Q^i, W_K^i and W_V^i for head i. Each head computes its own queries, keys and values from the same input, and runs scaled dot-product attention independently: head_i = Attention(X W_Q^i, X W_K^i, X W_V^i) The h outputs, each n by d_v, are concatenated side by side into an n by (h times d_v) matrix and multiplied by one more learned matrix, W_O, which maps the result back to width d_model: MultiHead(X) = Concat(head_1, ..., head_h) W_O The trick that makes this affordable is to shrink each head. In the original transformer, d_model was 512 and h was 8, and each head had d_k = d_v = 512 / 8 = 64. The total width of all queries across heads equals d_model, so the projections cost the same as one full-width head. Eight heads of width 64 do roughly the same arithmetic as one head of width 512, but they produce eight independent attention patterns instead of one. What does that buy? Each head computes its own n-by-n weight matrix, so each can attend to a different set of positions for a different reason. Head three might learn to attend from each verb to its subject; head five might attend to the previous token; head seven might spread attention broadly as a kind of summary. Each head works in its own 64-dimensional subspace, so the information it retrieves is drawn from a different projection of the values. The output projection W_O then mixes these retrieved pieces back together. The authors of the original paper tested the choice directly. Holding total computation constant, a single head of width 512 performed noticeably worse on translation than eight heads of width 64. Very many heads also hurt: with 32 heads of width 16, quality fell again, presumably because each head's queries and keys had become too small to express useful similarity. The sweet spot lay in the middle, and nearly every subsequent model has used head dimensions between 64 and 256. Contemporary large language models commonly use 128. There is a subtlety in how multi-head attention should be pictured. It is tempting to see the heads as a committee of independent specialists. Mathematically, the output projection W_O can be split into h horizontal slabs, one per head, so the multi-head output equals the sum over heads of head_i times its slab of W_O. Each head therefore writes its result directly into the full model width, and the heads' contributions are added. This additive view, emphasised by interpretability researchers, turns out to be the more useful one, and it connects to the residual stream described below. Later research has asked whether all those heads are needed. In 2019 Paul Michel, Omer Levy and Graham Neubig showed that, in trained translation and BERT models, a large fraction of heads could be removed at test time with little loss, and that many layers could be reduced to a single head. Elena Voita and colleagues found in the same year that a small number of heads in a translation encoder did most of the identifiable work, attending to positional neighbours, syntactic relations, or rare words, and that the rest could be pruned. These findings do not mean the extra heads are useless during training. Redundancy may make optimisation easier. But they show that the specialisation story is real for some heads and absent for others. The feed-forward network Attention moves information between positions. It does not, by itself, transform that information very much. Its output for each position is a weighted average of value vectors, which are linear projections of the input; the only nonlinearity is the softmax that chooses the weights. A stack of attention layers alone would be a poor function approximator. The second sublayer supplies the missing computation. After attention, each position's vector passes through the same small neural network, independently of every other position: FFN(x) = max(0, x W_1 + b_1) W_2 + b_2 The first matrix W_1 expands the vector from width d_model to a larger inner width d_ff; max(0, ...) is the rectified linear unit, or ReLU, which zeroes negative entries; and W_2 projects back down to d_model. In the original transformer d_ff was 2048, four times d_model, and that four-to-one ratio became a default for years. The feed-forward network looks unremarkable, but it holds most of the model's parameters. In each layer, the four attention projections (queries, keys, values, output) have 4 times d_model squared parameters. The two feed-forward matrices have 2 times d_model times d_ff, which with d_ff = 4 d_model is 8 times d_model squared. Two-thirds of each layer's weights are in the feed-forward network. What do those weights do? A useful picture comes from work by Mor Geva and colleagues, published in 2021, which interpreted feed-forward layers as key-value memories. Each column of W_1 acts as a key, detecting a pattern in the input; the ReLU passes through only those keys that are activated; and the corresponding row of W_2 is a value added to the output. They found individual units that fired on recognisable textual patterns and whose output rows promoted plausible next tokens. Later work by Kevin Meng and colleagues located factual associations, such as the country of a landmark, in mid-layer feed-forward weights, and showed that editing those weights could change the fact the model would report. The division of labour that emerges is suggestive though not absolute: attention routes information between positions, and feed-forward layers store and apply knowledge at each position. Modern models have replaced ReLU with smoother functions. The most widespread variant is the gated linear unit family, popularised for transformers by Noam Shazeer in 2020. In the SwiGLU version, the input is projected twice, once through a gate passed through the Swish (or SiLU) activation and once without, and the two are multiplied element-wise before the down-projection: FFN_SwiGLU(x) = (Swish(x W_1) * (x W_3)) W_2 This uses three matrices instead of two, so to keep parameter count constant the inner width is usually reduced to about two-thirds of four times d_model, roughly 8/3 times d_model. Shazeer reported consistent quality gains, and SwiGLU or close relatives became standard in the LLaMA family, PaLM, and most open models that followed. Residual connections and the residual stream The third component is a shortcut. Each sublayer's output is added to its input rather than replacing it: x = x + Sublayer(x) This is a residual connection, introduced for image recognition by Kaiming He and colleagues in 2015 to train networks with more than a hundred layers. Its first purpose is to help optimisation. Because the identity function passes through untouched, the gradient of the loss flows straight back from the output to every layer along the addition path, avoiding the multiplicative decay that crippled deep plain networks and recurrent networks alike. Its second consequence is conceptual, and has become central to how researchers think about transformers. With residual connections, the vector for each position is a running sum. It starts as the token's embedding plus position information. Every attention sublayer and every feed-forward sublayer reads from it and adds something to it. The final vector, which is used to predict the output, is the original embedding plus the sum of every sublayer's contribution. Researchers at Anthropic, in a 2021 framework for analysing transformer circuits by Nelson Elhage, Neel Nanda, Catherine Olsson and colleagues, named this the residual stream and argued that it is best seen as a shared communication channel. Each layer reads a subspace, computes, and writes to a subspace; later layers can read what earlier layers wrote. Attention heads are the only components that move information between positions' streams. Feed-forward layers act on one stream at a time. This picture explains several otherwise puzzling observations. Removing a single middle layer from a trained transformer often degrades it only mildly, because later layers can still read most of what they need from the stream. Different heads in different layers can compose: a head in layer five can use, as its query or key, information written into the stream by a head in layer two. Chapter 6 shows how this composition produces specific, identifiable algorithms. Layer normalization, and where to put it The fourth component keeps the numbers in a sensible range. Layer normalization, introduced by Jimmy Lei Ba, Jamie Ryan Kiros and Geoffrey Hinton in 2016, takes each position's vector, subtracts its mean, divides by its standard deviation, and then multiplies by a learned scale and adds a learned shift, both per component. The result has controlled magnitude regardless of what the previous layers did. The original transformer applied normalization after each residual addition: x = LayerNorm(x + Sublayer(x)). This is called post-norm. It works, but it is fragile. Post-norm transformers needed a learning-rate warm-up, a period at the start of training when the learning rate is increased gradually from near zero, or they diverged. Ruibin Xiong and colleagues analysed the problem in 2020 and showed that in post-norm models the gradients at the last layers are large at initialisation, which makes early large updates destabilising. The alternative is pre-norm, which normalizes the input to each sublayer but leaves the residual path untouched: x = x + Sublayer(LayerNorm(x)). The residual stream is then a pure sum, and the identity path carries gradients cleanly from output to input. Pre-norm transformers train stably with less careful warm-up and at greater depth, and they became standard after GPT-2 adopted the arrangement in 2019. A final normalization is applied after the last block, before the output projection. A further simplification is RMSNorm, proposed by Biao Zhang and Rico Sennrich in 2019. It skips subtracting the mean and simply divides by the root-mean-square of the components, then applies a learned scale. It is cheaper and, empirically, works as well. LLaMA, Mistral and many other recent models use pre-norm RMSNorm. Assembling the block Putting the pieces together, a modern decoder block, of the kind used in most current language models, computes: h = x + MultiHeadAttention(RMSNorm(x)) y = h + FFN(RMSNorm(h)) with causal masking inside the attention. The full model is an embedding layer that turns tokens into vectors, a stack of L such blocks, a final normalization, and an output layer that maps each position's final vector to scores over the vocabulary, from which a softmax gives the probability of each possible next token. The original 2017 transformer differed in several respects: it was an encoder-decoder (Chapter 5), it used post-norm, ReLU, and sinusoidal positions (Chapter 4). Its two published configurations show how modest the starting point was by later standards, as Table 1 records. Table 1. The two configurations of the original transformer (Vaswani et al., 2017). Setting Base Big Layers (encoder and decoder each) 6 6 Model width d_model 512 1024 Feed-forward width d_ff 2048 4096 Attention heads h 8 16 Head width d_k 64 64 Parameters about 65 million about 213 million Training 100,000 steps, about 12 hours on 8 P100 GPUs 300,000 steps, about 3.5 days on 8 P100 GPUs English-German BLEU 27.3 28.4 The big model's parameter count is smaller than that of many models that now run on mobile phones, and its training budget would be a modest experiment today. Yet the block is recognisably the same one used in models a thousand times larger. That continuity is itself a finding. When researchers scaled the transformer up by several orders of magnitude, the parts they changed were mostly the ones discussed in this chapter's margins: where normalization goes, which activation the feed-forward network uses, how the residual stream is initialised. The core, multi-head scaled dot-product attention feeding a position-wise network on a residual stream, survived. Counting parameters and operations Two rules of thumb follow from the block's structure and will be useful later. Parameters per layer: four attention matrices of d_model by d_model, plus feed-forward matrices totalling 8 d_model squared with the classic four-to-one ratio, for 12 d_model squared in all, ignoring small bias and normalization terms. A model with L layers therefore has about 12 L d_model squared parameters, plus the embedding table. For GPT-3, with L = 96 and d_model = 12,288, this gives 12 times 96 times about 151 million, or about 174 billion, close to its stated 175 billion. Arithmetic per token: each parameter in a matrix multiplication contributes one multiplication and one addition per token, so a forward pass costs about 2 floating-point operations per parameter per token, and training, which needs a backward pass roughly twice as costly as the forward pass, costs about 6. This "6N" rule, where N is the parameter count, underlies the scaling-law calculations of Chapter 7. It omits the attention score computation itself. For a context of n tokens, computing one token's scores against every key and then its weighted sum of values takes two products of about n times d_model multiply-adds each, or roughly 4 times n times d_model floating-point operations per token per layer in the forward pass (causal masking halves this on average). At short contexts that term is small next to the 24 d_model squared operations the layer's weights cost per token; the two become equal when n reaches 6 d_model. For a model with d_model = 4096, that crossover arrives around 25,000 tokens. Beyond it, the quadratic term of the previous chapter starts to dominate the arithmetic, and the argument of the later chapters begins. Chapter 4. Encoding Position Chapter 2 ended with an awkward fact. Self-attention is permutation-equivariant: shuffle the tokens and the outputs are shuffled in the same way, but otherwise identical. The feed-forward layers act on each position separately, so they are equally blind to order. A transformer with nothing added would treat "the dog bit the man" and "the man bit the dog" as the same bag of words, differing only in which output belongs to which token. Order must therefore be supplied from outside, by modifying either the token representations or the attention computation so that positions are distinguishable. This turns out to be one of the most consequential design choices in the architecture. The method chosen determines not only whether the model can tell word order, but how it represents distance, whether it can generalise to sequences longer than those seen in training, and how cheaply its context window can later be extended. This chapter follows the main methods in roughly the order they appeared, ending with the rotary embedding that now dominates and the interpolation techniques built on it. Absolute positions: sinusoids and learned tables The original transformer added a position vector to each token's embedding before the first layer. If the token at position pos has embedding e, the model's input there is e + PE(pos), where PE(pos) is a vector of the same width d_model. The authors chose a fixed, non-learned PE built from sine and cosine waves. For each pair of dimensions 2i and 2i + 1, with i running from 0 to d_model/2 minus 1: PE(pos, 2i) = sin(pos / 10000^(2i / d_model)) PE(pos, 2i + 1) = cos(pos / 10000^(2i / d_model)) Each pair of dimensions is a sine-cosine pair oscillating at its own frequency. The first pair cycles once every 2 pi positions, about 6.3 tokens; the frequencies decrease geometrically across dimensions until the last pair has a wavelength of roughly 10,000 times 2 pi, over sixty thousand positions. The result resembles a binary counter written in smooth waves: low dimensions change rapidly from one position to the next, high dimensions change slowly, and together they give every position a distinct fingerprint. The choice of sines and cosines was deliberate. For any fixed offset k, the encoding of position pos + k is a linear function of the encoding of position pos: each sine-cosine pair at pos + k equals the pair at pos rotated by an angle proportional to k. The authors hypothesised that this would let the model learn to attend by relative position, since "the token three to my left" corresponds to a fixed linear transformation regardless of where one is in the sequence. They also hoped the smooth functional form would let the model handle positions beyond the training length. They compared this with the simpler alternative of learning a separate vector for each position, a lookup table of positional embeddings trained alongside everything else, and found nearly identical results on translation. BERT and the GPT models adopted learned absolute embeddings, with tables of 512 positions for BERT and 1,024 for GPT-2. A learned table is flexible, but it has a hard limit. Position 1,025 simply has no entry. Neither method delivered meaningful extrapolation in practice: models trained on sequences of one length degraded sharply when run on longer ones, because the combinations of position signals and content they encountered were outside anything seen in training. Adding position to the content embedding has a further drawback. Position and meaning are mixed in the same vector from the first layer, and every subsequent layer must disentangle them. When the query and key are computed from a sum e + p, the dot product between positions i and j expands into four terms: content with content, content with position, position with content, and position with position. Only the last, and in a limited way the middle two, carry ordering information, and the model must learn to use them through the same projections it uses for meaning. Relative positions An alternative line of work started from the observation that what usually matters in language is not a word's absolute position, but how far apart two words are. Whether a determiner precedes its noun by one position is the same fact at position 7 and at position 700. In 2018 Peter Shaw, Jakob Uszkoreit and Ashish Vaswani proposed modifying attention so that each query-key score includes a learned term depending only on the clipped distance j minus i between the two positions. Distances beyond some maximum, such as 16 in either direction, were treated as equal. This improved translation slightly and removed absolute positions from the input entirely. Transformer-XL, by Zihang Dai and colleagues in 2019, refined the approach with a sinusoidal relative encoding and separate learned biases for content and position, and combined it with a segment-level recurrence that let a language model attend into a cached previous segment, an early attempt at longer context. The T5 model from Google, described by Colin Raffel and colleagues in 2019, simplified relative position to its bare minimum: a single learned scalar bias, added to the attention score, for each bucket of relative distance. Nearby distances got their own buckets; farther ones were grouped logarithmically, so that 64 to 127 tokens apart might share one bucket. The bias was shared across layers and differed per head. This was cheap and effective, though computing the bias for every pair adds overhead and the bucketing is coarse at long range. In 2021 Ofir Press, Noah Smith and Mike Lewis proposed ALiBi, short for Attention with Linear Biases. ALiBi adds no learned position parameters at all. Instead, to each attention score between query i and key j it adds minus m times (i minus j), a penalty that grows linearly with distance, where m is a fixed slope that differs by head: one head has a steep slope and attends mostly locally, another a shallow slope and attends broadly. Press and his colleagues called their paper "Train Short, Test Long," and their headline result was that a model trained with ALiBi on sequences of 1,024 tokens could be evaluated on 2,048 tokens with no loss in perplexity, which sinusoidal and learned embeddings could not manage. ALiBi was adopted in BLOOM and MPT. Its weakness is the flip side of its strength: by construction it biases every head toward recency. A model penalised linearly for attending far away extrapolates gracefully partly because it largely ignores the distant tokens it was never trained to use. Rotary position embedding The method that won was proposed by Jianlin Su and colleagues in a 2021 paper titled "RoFormer: Enhanced Transformer with Rotary Position Embedding." Rotary embeddings, usually called RoPE, were adopted in GPT-J and GPT-NeoX in 2021, PaLM in 2022, and then LLaMA in 2023, after which they became the default in nearly every open-weight language model. Understanding why requires a short excursion into rotation. Consider a vector with two components, (x, y). Rotating it by an angle theta about the origin gives (x cos theta minus y sin theta, x sin theta plus y cos theta). Rotation changes the vector's direction but not its length. Now take two vectors, rotate the first by angle alpha and the second by angle beta, and take their dot product. Because the dot product depends only on the lengths of the vectors and the angle between them, the result equals the dot product of the original vectors with the first rotated by alpha minus beta. The absolute angles drop out; only their difference survives. RoPE uses exactly this property. It splits each query and key vector of width d_k into d_k/2 pairs of components. For a token at position m, it rotates pair i by an angle m times theta_i, where the frequencies theta_i follow the same geometric schedule as the sinusoidal encoding: theta_i = base^(-2i / d_k), with base originally set to 10,000. A query at position m and a key at position n are each rotated according to their own positions. When their dot product is taken, the rotations combine into a single rotation by (m minus n) times theta_i for each pair. The attention score therefore depends on the content of the query and key and on their relative offset, m minus n, but not on their absolute positions. In symbols: if R(m) denotes the block-diagonal rotation for position m, then the score between query q at position m and key k at position n is (R(m) q) · (R(n) k) = q · (R(n minus m) k), with the sign convention depending on which vector one regards as rotated relative to the other. Whatever the convention, absolute position cancels. This is an attractive combination. Position is injected at every layer, directly into the query-key comparison, rather than once at the input. Content and position are not added together in the residual stream; values are not rotated at all, so what a token contributes is free of positional contamination. The method is relative in effect, but it is applied to each token independently, so it costs only a cheap element-wise operation on queries and keys and does not require computing an n-by-n bias matrix. And because rotation preserves length, it does not distort the magnitude of the scores. The frequencies give RoPE a natural multi-scale structure. The highest-frequency pairs rotate by about one radian per token and complete a full turn every six or seven tokens; they are sensitive to fine positional detail but wrap around quickly, so two tokens 100 apart and 106 apart look similar to them. The lowest-frequency pairs, with base 10,000 and d_k = 128, rotate so slowly that their wavelength is around 54,000 tokens; they change little across a short context and serve almost as position-free channels that heads can use for content-based matching. Su and colleagues also showed that, averaged over content, the RoPE score tends to decay as the relative distance grows, a mild built-in preference for nearby tokens that resembles, in softer form, ALiBi's explicit penalty. Extending the context window RoPE does not by itself extrapolate well. A model trained with RoPE on sequences of 2,048 tokens and run on 4,096 encounters rotation angles in its low-frequency dimensions that it never saw during training, and its perplexity rises sharply. What RoPE offered was not extrapolation but something more useful: a clean, parametric description of position that could be manipulated after training. Between 2023 and 2024 a series of techniques exploited this to stretch context windows by large factors with little additional training. The first was position interpolation, published by Shouyuan Chen and colleagues at Meta in June 2023, and discovered independently around the same time by a hobbyist developer who posted it online. The idea is to squeeze rather than stretch. To run a model trained on 2,048 positions over 8,192, divide every position index by four before computing the rotations. Position 8,000 is rotated as if it were position 2,000. The model never sees an angle outside its training range; it simply sees positions packed more densely. With a short period of fine-tuning, around a thousand steps, Chen's team extended LLaMA models from 2,048 to 32,768 tokens with good results. Interpolation has a cost. Dividing all positions by four also compresses the high-frequency dimensions, which had no extrapolation problem and which the model relies on to distinguish adjacent tokens. Squeezing them blurs local order. A series of refinements addressed this by treating frequencies differently. "NTK-aware" scaling, again first circulated informally in 2023, increases the RoPE base rather than dividing positions, which spreads the adjustment unevenly: low frequencies are slowed substantially while high frequencies are left nearly untouched. YaRN, published by Bowen Peng, Jeffrey Quesnelle, Honglu Fan and Enrico Shippole in 2023, divided the frequency spectrum into bands, leaving the highest frequencies alone, fully interpolating the lowest, and blending in between, and added a small correction to the attention temperature to compensate for the change in score distribution at long range. YaRN extended LLaMA 2 models to 128,000 tokens with modest fine-tuning. Model developers also simply raised the base. Meta's Llama 3, released in 2024, used a RoPE base of 500,000 rather than 10,000, which slows every frequency and makes long wavelengths far longer from the start; Llama 3.1 then applied a frequency-dependent rescaling of the YaRN type to reach a 128,000-token window. Code Llama, a year earlier, had used a base of one million for the same reason. LongRoPE, from Microsoft researchers in 2024, used a search procedure to find a separate rescaling factor for each frequency and reported extending a model's window beyond two million tokens. Each of these is a way of reconciling two demands. The model's positional machinery must assign distinguishable signals to every position in the new, longer window, and those signals must remain close enough to the ones seen in training that the learned weights still apply. RoPE made this reconciliation tractable because position enters through a small number of known frequencies that can be adjusted analytically, rather than through a learned table that has no meaningful value outside its range. Do models need explicit position at all? One last result complicates the story. In 2022 Adi Haviv and colleagues reported that decoder-only language models trained with no position encoding at all, a configuration they called NoPE, achieved perplexity close to that of models with explicit encodings. Amirhossein Kazemnejad and colleagues followed in 2023 with evidence that NoPE could, on some synthetic tasks, generalise to longer sequences better than models with RoPE or ALiBi. How can a model with no position signal know order? The answer is the causal mask. In a decoder, position one can attend only to itself, position two to two tokens, position n to n tokens. A head that attends uniformly over everything it is allowed to see produces an average whose properties, such as how much any single token contributes, depend on how many tokens are visible, and therefore on position. The mask leaks order into the computation, and gradient descent learns to exploit the leak. This does not work for encoder models, which have no causal mask, and NoPE has not displaced RoPE in production models. Some recent architectures do combine the two, interleaving layers with RoPE and layers without any position encoding, on the reasoning that the position-free layers may be better at retrieving information regardless of distance. The broader lesson is that position in a transformer is not a single feature added at the input but a property that can enter through several channels, and the choice among them shapes what the model finds easy and what it finds hard, especially at the edge of its context. Chapter 5. Encoders, Decoders, and the Triumph of Prediction The 2017 transformer was a translation machine with two halves. Within two years, researchers had split it apart and found that each half was useful on its own. Three families emerged: encoder-only models, which read text and produce representations of it; decoder-only models, which generate text one token at a time; and encoder-decoder models, which keep the original pairing. For a few years the three coexisted, each dominant in its niche. Then, as models grew, the decoder-only family absorbed most of the field. Understanding why tells us something about what the transformer is good at, and it also explains why the cost problems of later chapters are framed almost entirely in terms of autoregressive generation. The original pairing In the translation transformer, the encoder is a stack of blocks with unmasked self-attention: every source token can attend to every other source token, in both directions. Its output is a sequence of vectors, one per source token, each enriched with context from the whole sentence. The decoder is a stack of blocks with three sublayers instead of two. The first is masked self-attention over the target tokens generated so far, using the causal mask from Chapter 2. The second is cross-attention, in which queries come from the decoder's own stream while keys and values come from the encoder's final output. This is the direct descendant of Bahdanau's attention: at each target position, the decoder asks which source tokens are relevant. The third is the feed-forward network. During training, the decoder sees the whole correct target sentence, shifted right by one position, and predicts every next token in parallel. During translation, it generates one token at a time, feeding each output back in as the next input. The design separates understanding from production. The encoder can look at the whole input at once, which suits comprehension; the decoder must be causal, because it produces output in order. That asymmetry seemed natural for translation, summarisation, and any task framed as transforming one text into another. Encoder-only: BERT and representation learning In October 2018 Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova at Google released BERT, for Bidirectional Encoder Representations from Transformers. BERT was the transformer's encoder alone, trained on a large unlabelled corpus of books and English Wikipedia, around 3.3 billion words, and then adapted to specific tasks. The difficulty with training an encoder on unlabelled text is choosing what to predict. A causal language model predicts the next word, but an encoder that sees the whole sentence could simply look at the answer. BERT's solution was masked language modelling. During training, 15 percent of input tokens were selected; most of these were replaced with a special [MASK] symbol, some with a random token, and a few left unchanged. The model had to predict the original tokens from their context on both sides. A second objective, next-sentence prediction, asked whether two segments were consecutive in the original text; later work, notably RoBERTa from Facebook AI in 2019, found it unnecessary and dropped it while training longer on more data. BERT's results were striking. Its large version, with 24 layers, width 1,024, 16 heads and about 340 million parameters, set new records on the GLUE benchmark of language understanding, on the SQuAD question-answering benchmark, and on several others. The method of use was as influential as the scores: take the pre-trained encoder, add a small task-specific output layer, and fine-tune the whole thing for a few epochs on labelled data. Tasks that had each required bespoke architectures now shared one recipe. Encoder-only models remain widely used, though rarely in the headlines. Search engines, recommendation systems, and retrieval components of larger AI systems depend on embedding models, which map a text to a single vector so that similar texts land near each other. Most embedding models are encoders in the BERT lineage. Their bidirectional attention lets every token's representation reflect the full input, which suits tasks where the whole text is available at once and nothing needs to be generated. Decoder-only: GPT and the next token Four months before BERT, in June 2018, Alec Radford and colleagues at OpenAI described a model now known as GPT. It was the transformer's decoder without cross-attention: a stack of masked self-attention and feed-forward blocks trained to predict the next token on a corpus of about seven thousand unpublished books. Like BERT, it was then fine-tuned on specific tasks, and it improved the state of the art on many of them, though BERT's bidirectional encoder soon surpassed it on understanding benchmarks. The difference between the two objectives is worth stating precisely. A causal language model is trained to maximise the probability of each token given all the tokens before it. If the sequence is t_1, t_2, ..., t_n, the training loss is the average over positions of minus log P(t_i | t_1, ..., t_(i-1)). This is an ordinary likelihood objective: the model assigns a probability to every possible text, and training makes the real texts more probable. Because of the causal mask, a single forward pass over a sequence of n tokens yields n predictions, each a full training signal. Every token in the corpus is a target. Masked language modelling, by contrast, provides a learning signal only at the masked positions, about 15 percent of tokens. It also does not define a straightforward probability distribution over whole texts, so a masked model cannot easily generate text. Its advantage is bidirectional context for every representation. The decisive move came with GPT-2 in February 2019. Radford and colleagues trained a 1.5-billion-parameter decoder on WebText, about 40 gigabytes of text from web pages linked from Reddit posts with some minimum level of engagement. They did not fine-tune it. Instead they showed that the pre-trained model could perform tasks if the task was written as text to be continued. Appending "TL;DR:" to an article prompted something like a summary. Presenting an English sentence followed by an equals sign and a French sentence, a few times, and then a new English sentence with an equals sign, prompted a translation. The results were well below task-specific systems, but the principle was established: a model trained only to predict the next token had acquired, as a by-product, some ability to perform tasks it had never been explicitly trained on. OpenAI initially withheld the full model, citing concerns about misuse, and released it in stages over nine months. GPT-3, described by Tom Brown and colleagues in May 2020, scaled the same design to 175 billion parameters and about 300 billion training tokens. The paper's title, "Language Models are Few-Shot Learners," named its finding. With a handful of examples placed in the prompt, and no change to the model's weights, GPT-3 performed competitively on many tasks for which earlier systems had needed fine-tuning. This behaviour, called in-context learning, became the defining capability of large language models, and Chapter 6 discusses a mechanism that partly explains it. Encoder-decoder: T5 and the unification attempt The third family kept both halves. T5, the Text-to-Text Transfer Transformer from Raffel and colleagues at Google, cast every task as converting an input text to an output text: "translate English to German: That is good." becomes "Das ist gut."; "cola sentence: The course is jumping well." becomes "not acceptable." Its pre-training objective was span corruption: replace random spans of the input with sentinel tokens and train the decoder to output the missing spans. The T5 paper was an unusually systematic study. Its authors compared architectures, objectives, data sets and scales, and found that encoder-decoder models with a span-corruption objective were the strongest of the configurations they tested at matched computational cost. Encoder-decoder models remained competitive for several years. BART, from Facebook AI, used the pairing for generation tasks such as summarisation. Google's speech recognition and translation systems, and OpenAI's Whisper speech model, released in 2022, use encoder-decoder transformers, which suit tasks where a fixed input, such as an audio clip, is transformed into an output sequence. Table 2 sets the three families side by side. Table 2. The three transformer families compared. Family Attention pattern Typical pre-training objective Representative models Typical uses Encoder-only Bidirectional self-attention Masked token prediction BERT, RoBERTa, DeBERTa Classification, retrieval, embeddings Decoder-only Causal self-attention Next-token prediction GPT series, PaLM, LLaMA Open-ended generation, chat, code, general assistants Encoder-decoder Bidirectional encoder, causal decoder, cross-attention Span corruption or denoising T5, BART, Whisper Translation, summarisation, speech-to-text Why the decoder won By 2023 almost every frontier language model was decoder-only. Several reasons combined, and none is sufficient alone. The first is simplicity of the objective at scale. Next-token prediction turns every token of raw text into a training example and requires no decisions about masking rates or span lengths. When the bottleneck on progress became the amount of computation and data that could be poured into one model, an objective that used all of the data with no wasted positions and no design choices was attractive. The second is generality of the interface. A decoder can generate text, and once it can generate text, nearly any task can be posed as a prompt and answered as a continuation. Classification becomes generating a label word; retrieval-augmented question answering becomes generating an answer after the retrieved passages; translation becomes generating the target after the source. The encoder-decoder model can do this too, but it must divide the context into an input part and an output part, and the dividing line is not always natural in a multi-turn conversation or an extended document where the model's own previous outputs become input. The third is efficiency of reuse during generation. A decoder-only model generating a response processes the prompt and its own outputs with the same stack, and caches keys and values from every previous token so they need not be recomputed. Chapter 8 discusses this cache at length. In an encoder-decoder model, the encoder runs once on the input, which is efficient, but any new input in a conversation requires either re-encoding or awkward bookkeeping. The fourth is empirical, and it is the most debated. In 2022 Thomas Wang and colleagues ran a controlled comparison of architectures and objectives at a scale of about five billion parameters. They found that encoder-decoder models trained with masked objectives did best after multitask fine-tuning, but that decoder-only models trained with plain next-token prediction did best at zero-shot generalisation, performing tasks straight from a prompt. Since the strategic bet of the large labs was on zero-shot and few-shot use, the decoder was the natural choice. It is not clear that encoder-decoder models are fundamentally worse at large scale; there is simply far less evidence about them, because few groups trained them at the frontier after 2021. Finally, decoder-only models combined naturally with the next phase of training. After pre-training, modern assistants are adapted by supervised fine-tuning on examples of desired behaviour and then by reinforcement learning, from human feedback or from automatically checkable rewards, to follow instructions and produce useful answers. These methods operate on generated text, and fit an architecture whose native operation is generation. The consequence for cost The triumph of the decoder had a direct effect on where the transformer's costs fell. An encoder processes a fixed input in one parallel pass, and its quadratic attention cost is paid once. A decoder generating text pays in two phases with very different profiles. In the first phase, called prefill, the model processes the prompt. All prompt tokens are known, so this proceeds exactly as in training: a fully parallel pass with causal masking, costing quadratic attention arithmetic over the prompt length. The GPU is well used. In the second phase, decode, the model generates one token at a time. Each new token needs a query, which must be compared against the keys of every previous token, and a weighted sum of their values. Recomputing all those keys and values every step would be wasteful, so they are stored, layer by layer and head by head, in the key-value cache. Each step then computes only one new query, key and value per head, appends the new key and value to the cache, and attends over the whole cache. The arithmetic for that step is linear in the context length. But every step must read the entire cache from memory, and generating a thousand tokens means doing so a thousand times. That memory traffic, rather than the arithmetic, is what limits generation speed in practice, and it grows with every token of context. Much of the long-context engineering of Chapters 8 and 9 is an attack on the cache. It is a problem that belongs specifically to autoregressive decoders, and it became central precisely because the decoder won. Chapter 6. What Attention Learns The formula for attention says how information can move. It does not say what information does move in a trained model. The weights that determine queries, keys and values are learned, and what they learn is not written down anywhere. For several years this left a gap between a precise understanding of the mechanism and a vague one of its use. Since about 2019, a body of research has begun to close that gap. It has found that some attention heads implement identifiable, almost algorithmic functions, that these functions compose across layers, and that one of them, the induction head, appears to underlie a central capability of large language models. It has also found that attention weights are a less reliable window into a model than they first appeared. Both findings matter for understanding why the architecture works and what is lost when it is approximated. Reading the attention matrix The most direct way to inspect a head is to look at its weight matrix A for some input: the n-by-n array whose row i shows how much position i attends to each other position. Researchers did this from the beginning. The transformer paper's appendix included examples in which some heads in the encoder of a translation model appeared to resolve the antecedents of pronouns and others to track sentence structure. A systematic study by Kevin Clark, Urvashi Khandelwal, Omer Levy and Christopher Manning in 2019 examined all 144 heads of BERT-base, twelve layers of twelve heads each. They found several recurring patterns. Some heads attended almost entirely to the previous token or the next token. Some attended broadly across the whole sentence. A strikingly large share of attention, in many heads, went to the special separator token [SEP] or to punctuation such as commas and full stops. Most interestingly, specific heads corresponded closely to specific syntactic relations. One head attended from verbs to their direct objects with high accuracy; another from nouns to their determiners; another from possessive pronouns to the nouns they modified; another linked coreferent mentions. No head was a complete parser, but for several standard dependency relations there was a head that recovered it far better than a simple positional baseline. John Hewitt and Christopher Manning, also in 2019, approached the same question from the representations rather than the attention weights. They showed that a linear transformation of BERT's hidden vectors could recover the distances between words in a syntax tree. The model had not been told about grammar. It had learned an internal geometry that encoded it, because doing so helped predict masked words. The trouble with attention as explanation The apparent readability of attention weights led to a common practice of presenting them as explanations: highlight the input words a model attended to, and say that these were the reasons for its prediction. In 2019 Sarthak Jain and Byron Wallace challenged this in a paper titled "Attention is not Explanation." Working with recurrent models with attention on text classification tasks, they showed that attention weights often correlated poorly with other measures of word importance, and that one could often construct quite different attention distributions that yielded the same prediction. Sarah Wiegreffe and Yuval Pinter replied, in "Attention is not not Explanation," that the adversarial distributions were not ones the model would naturally produce and that attention did carry useful information under appropriate tests. The exchange did not end with a clean verdict, but it established a caution that has held up. Transformers make the caution stronger. The value a head retrieves from position j is not token j's original embedding. It is a projection of the residual stream at j, which after a few layers already blends information from many positions. A head in layer ten attending strongly to the word "it" may actually be retrieving information about the noun that "it" refers to, written there by an earlier head. Samira Abnar and Willem Zuidema proposed in 2020 to address this with "attention rollout," multiplying attention matrices across layers to trace flow back to the input, but this too is an approximation that ignores the feed-forward layers and the value contents. A related observation reveals something about how attention is used, not just how it should be read. Many heads place a large share of their weight on the first token of the sequence, or on a delimiter, regardless of content. Kevin Clark and colleagues saw this in BERT; Guangxuan Xiao and colleagues studied it in large decoder models in 2023 and called these positions "attention sinks." The explanation is a quirk of softmax. Its outputs must sum to one, so a head that has nothing useful to retrieve for a given query cannot simply attend to nothing. It needs somewhere harmless to put its weight, and the model learns to use an early, always-visible token whose value contributes little. Xiao's group showed that this had practical consequences: if a long-running generation discarded the earliest tokens from its cache to save memory, quality collapsed, but keeping just the first four tokens alongside a recent window restored it. They called the resulting method StreamingLLM. Other researchers have since experimented with adding an explicit learnable "sink" or allowing the softmax to sum to less than one, to give heads a sanctioned way to abstain. Circuits The most productive approach to understanding attention has been mechanistic: treat a trained transformer as a program to be reverse-engineered, and identify the specific combinations of components, called circuits, that implement particular behaviours. The framework from Anthropic's 2021 paper by Elhage and colleagues, mentioned in Chapter 3, made this tractable by rewriting an attention head as two nearly independent circuits. The query-key circuit, determined by the product of W_Q and W_K^T, decides where each position attends: it is a bilinear form saying how much a token with property a should attend to a token with property b. The output-value circuit, determined by the product of W_V and the head's slice of W_O, decides what information moves once a position is attended to and how it is written into the residual stream. A head can thus be analysed as answering two separate questions: "Which tokens should I look at?" and "What should I copy from them?" The authors studied very small attention-only transformers, with no feed-forward layers, so that every computation could be traced. In a one-layer model, they found heads that implemented skip-trigram patterns: if token A appears somewhere earlier and the current token is B, raise the probability of C. A head might learn that after "keep" and later "in," the token "mind" is likely. These patterns are useful but limited: one layer of attention can look at the current token and at earlier tokens, but cannot condition its search on relationships between earlier tokens. Two layers change this, because a head in the second layer can use, in its query, key or value, information that a first-layer head has written into the stream. The authors called this composition. If a first-layer head writes into each position "the previous token was X," then a second-layer head can use that as its key, and so search for positions whose previous token matches something. Induction heads That composition yields the circuit that has become the best-known finding in transformer interpretability. Consider text in which a pattern repeats: a name like "Mrs Dursley" appears, and later "Mrs" appears again. A good predictor should guess that "Dursley" follows. More generally, if the sequence contains "A B ... A," predict "B." A two-layer transformer can implement this with two heads. In the first layer, a previous-token head at each position attends to the position immediately before it and copies that token's identity into the stream. Now every position carries a note saying which token preceded it. In the second layer, an induction head at the current position, whose token is A, issues a query looking for positions whose "previous token" note says A. It finds the position just after the earlier A, which holds B. Its output-value circuit copies B's identity forward, raising the probability that the model predicts B next. The mechanism is precise: it uses relative position, through the previous-token head, and content matching, through the query-key circuit, to look up what followed the last occurrence of the current token. Catherine Olsson and colleagues at Anthropic, in a 2022 paper titled "In-context Learning and Induction Heads," made a broader argument. They tracked many models, small and large, through training and observed that at a specific point, early in training, these heads formed abruptly. At the same moment, the models' in-context learning ability improved sharply. They measured in-context learning by how much better the model predicted tokens late in a long context than early in it, that is, how much it benefited from having more text to draw on. This improvement appeared in a sudden phase change, visible as a bump in the training loss curve, coinciding with the formation of induction heads. In small attention-only models, where interventions could be precise, removing the induction heads removed most of the in-context learning. In larger models, the authors offered several lines of correlational evidence that the same mechanism, or more abstract generalisations of it, were responsible for a large part of in-context learning. The induction head is a specific, mechanistic link between the architecture and its most celebrated emergent capability. Few-shot prompting, in which a model is shown a handful of input-output examples and then a new input, is a sophisticated version of "A B ... A, predict B." Larger models appear to contain fuzzier induction heads that match not exact tokens but similar meanings, so that the pattern "translation of a word follows the word" can be continued for a word never seen in the prompt. The induction head also makes a point about architecture that matters for later chapters. It depends on the ability of a single position to retrieve a specific token from an arbitrary earlier location in the context, chosen by content. This is exactly what full attention offers and what compressed alternatives, which summarise the past into a fixed-size state, find hardest. When Chapter 11 discusses why state-space models lag transformers at certain recall tasks, the reason is visible here. Other identified mechanisms Induction heads are the clearest case, but not the only one. In 2022 Kevin Wang and colleagues reverse-engineered how GPT-2 small completes sentences of the form "When Mary and John went to the store, John gave a drink to ..." with "Mary." They identified a circuit of 26 heads in several functional classes: heads that detect the duplicated name, heads that suppress it, and "name mover" heads that copy the remaining name to the output. They also found backup name movers that took over when the primary ones were ablated, a vivid example of redundancy in a trained network. Research on factual recall, including the Meng study mentioned in Chapter 3 and follow-up work by Mor Geva and colleagues in 2023, has traced how a model answers "The Eiffel Tower is in the city of": early feed-forward layers at the subject tokens enrich the representation of "Eiffel Tower" with attributes, and later attention heads at the final position extract the relevant attribute, the city, from the subject's position. Attention here acts as a retrieval mechanism over an enriched representation, not as a store of the fact itself. Limits of the circuit picture Two difficulties prevent this approach from yet giving a full account of large models. The first is superposition. Models have far more features they would like to represent than dimensions in their residual stream, and research by Elhage and colleagues in 2022 showed that small networks learn to store more features than they have dimensions, by encoding them as nearly orthogonal directions that interfere slightly. Individual neurons, and individual dimensions, then respond to many unrelated features, which makes them hard to interpret one at a time. Work since 2023 on sparse autoencoders, which attempt to decompose activations into a larger set of sparsely active and more interpretable features, has made progress on this, and in 2024 Anthropic and others reported extracting millions of interpretable features from production-scale models. But mapping features into complete circuits at that scale remains laborious. The second difficulty is scale itself. A model with 80 layers and 64 heads per layer has over five thousand heads, each of which may play several roles depending on context. The successes so far are islands of understanding. They are enough to show that attention heads in trained models do implement legible operations, that those operations compose through the residual stream, and that a key capability of large models can be traced to a specific form of content-addressed retrieval. They are not yet a complete map. What the circuit work establishes for the argument of this booklet is concrete. Full attention's ability to fetch any earlier token by content, at any distance, is not an abstract capacity that models leave unused. Trained transformers rely on it for in-context learning, for copying, for resolving references, and for retrieving facts. Any proposal to make attention cheaper must be judged by whether it preserves that ability, and the record of the next several chapters is largely a record of how hard that is. Chapter 7. Scale, Laws, and Sparse Experts The transformer did not become dominant only because it was better than recurrent networks at a given size. It became dominant because it kept getting better as it grew, smoothly and predictably, across many orders of magnitude. That property turned model building from a craft of architectural invention into something closer to an engineering discipline governed by budgets and forecasts. It also changed the economics of the architecture's costs, because once the main lever of progress was scale, every inefficiency was multiplied by enormous sums. This chapter describes the empirical laws of scaling, the correction that reshaped them, and the mixture-of-experts designs that decouple a model's size from its per-token cost. These belong in an account of attention because they explain why the attention bottlenecks of the following chapters became so pressing: scale made context valuable, and context is where attention is expensive. Smooth power laws In January 2020 Jared Kaplan, Sam McCandlish and colleagues at OpenAI published "Scaling Laws for Neural Language Models." They trained many decoder-only transformers, ranging from under a thousand to over a billion non-embedding parameters, on varying amounts of text, and measured the test loss: the average negative log probability the model assigned to held-out tokens. Lower is better. They found that the loss followed a power law in each of three quantities when the others were not limiting: the number of parameters N, the number of training tokens D, and the amount of compute C used in training. A power law has the form L = (X_0 / X)^alpha: double X and the loss falls by a constant factor. On a plot with logarithmic axes it is a straight line. For parameters, they estimated an exponent of about 0.076; for data, about 0.095; for compute, about 0.050. These are small exponents. Halving the loss attributable to model size would require increasing N by a factor of roughly 2^(1/0.076), more than nine thousand. But the trends held cleanly over six or seven orders of magnitude, with no sign of levelling off within the tested range. Just as notable was what did not matter much. Within broad limits, the shape of the transformer, whether it was deep and narrow or shallow and wide, how many heads it had, and the ratio of feed-forward to model width, had only small effects on loss at a given parameter count. Performance depended mostly on scale. This was a powerful simplification, and also a challenge to architecture research. A new design had to beat not a fixed baseline but a baseline that improved predictably with resources. Kaplan and colleagues drew a practical conclusion about allocation. If a fixed compute budget C can be spent on a larger model trained on fewer tokens or a smaller model trained on more, which is better? Using the 6ND approximation for training compute from Chapter 3, they estimated that as the budget grows, most of the increase should go to model size, with data growing more slowly; specifically, optimal model size should scale roughly as C^0.73. This analysis guided the construction of GPT-3 and other models of 2020 and 2021, which were very large relative to the data they were trained on. GPT-3's 175 billion parameters saw about 300 billion tokens, under two tokens per parameter. The Chinchilla correction In March 2022 Jordan Hoffmann and colleagues at DeepMind published "Training Compute-Optimal Large Language Models," which revised that conclusion. They trained over four hundred models, from 70 million to 16 billion parameters, on 5 to 500 billion tokens, and fitted the loss with three independent methods. All three agreed: model size and training tokens should be scaled in roughly equal proportion. Double the compute, and both N and D should increase by a factor of about the square root of two. The compute-optimal ratio came out at roughly twenty training tokens per parameter. A key reason for the disagreement with the earlier work was the learning-rate schedule. Kaplan's experiments had used a fixed schedule length, so models trained on fewer tokens than the schedule assumed were evaluated before their learning rate had decayed fully, which understated the benefit of more data. To test the conclusion, DeepMind trained Chinchilla, a 70-billion-parameter model, on 1.4 trillion tokens, using about the same compute as its earlier 280-billion-parameter model, Gopher, which had seen 300 billion tokens. Chinchilla outperformed Gopher on almost every benchmark they reported, despite being a quarter of the size. It was also four times cheaper to run. That last point exposed a gap in "compute-optimal." The Chinchilla analysis minimises the cost of training a model to a given loss. But a deployed model is trained once and then run, often billions of times. If inference cost matters, it is worth training a smaller model on far more tokens than the Chinchilla ratio suggests: the extra training compute is paid once, while the smaller model's lower inference cost is collected on every query. After 2022, developers pushed past the Chinchilla ratio by large factors. Meta's LLaMA models of February 2023 trained a 7-billion-parameter model on a trillion tokens. Llama 3, in 2024, trained an 8-billion-parameter model on about 15 trillion tokens, nearly two thousand tokens per parameter, and Meta reported that the models were still improving when training stopped. This trend matters for the argument here. Smaller models trained on more data shift the balance of a model's costs. When the weights are relatively small, the memory and bandwidth consumed by the key-value cache for long contexts becomes a larger fraction of the total cost of serving the model. A trend that made models cheaper per parameter made context relatively more expensive. What the laws do not say Scaling laws describe the average loss on next-token prediction. They do not directly predict specific capabilities, such as solving arithmetic problems or following multi-step instructions. A 2022 study led by Jason Wei catalogued "emergent abilities," tasks on which performance stayed near chance as models grew and then rose sharply above some scale. In 2023 Rylan Schaeffer, Brando Miranda and Sanmi Koyejo argued that many of these apparent discontinuities were artefacts of the metrics chosen: an all-or-nothing score, such as exact-match accuracy on a multi-digit sum, can jump abruptly even when the underlying per-token probabilities improve smoothly. The debate has not entirely settled, but its practical lesson is that smooth loss curves can conceal threshold effects in what users experience. The laws also do not establish that the transformer is the only architecture that scales. They show that it scales reliably, and that matching it requires a design that scales at least as well. Several challengers discussed in Chapter 11 have published their own scaling curves, and the question that matters is whether their lines lie above or below the transformer's at the scales people actually train. Mixture of experts Scaling laws suggest that more parameters help. The 6N rule says that more parameters cost more compute per token. Mixture-of-experts architectures attempt to break that link: give the model many parameters, but use only a fraction of them for any given token. The idea is older than the transformer. Robert Jacobs, Michael Jordan, Steven Nowlan and Geoffrey Hinton described mixtures of experts in 1991. In 2017 Noam Shazeer and colleagues at Google applied a sparsely gated version to large recurrent language models, with thousands of experts, and showed large gains in capacity at modest computational cost. The transformer version replaces the dense feed-forward network in some or all blocks with a set of E expert networks, each an ordinary feed-forward network, and a router. For each token, the router, a small learned linear layer followed by softmax, scores the experts; the token is sent to the top k of them, typically one or two; and their outputs are combined, weighted by the router scores. If a model has 8 experts per layer and routes each token to 2, it holds roughly four times the feed-forward parameters of a dense model with the same per-token computation. Since feed-forward layers hold most of a transformer's parameters, the effect on total size is large. The Switch Transformer, by William Fedus, Barret Zoph and Shazeer in 2021, simplified routing to a single expert per token and scaled to over a trillion parameters. They reported reaching the same quality as a dense T5 baseline up to seven times faster in pre-training time at equal computation per token. Mixtral 8x7B, released by Mistral AI in December 2023, brought the approach to widely used open models. It has eight experts per layer and routes each token to two, giving about 47 billion total parameters but about 13 billion active per token, and it matched or exceeded the dense 70-billion-parameter Llama 2 on most of the benchmarks its developers reported. DeepSeek-V3, released in December 2024, pushed the ratio further, with 671 billion total parameters and 37 billion activated per token, using many small, finely divided experts plus a shared expert that every token uses. Mixture-of-experts models have real difficulties. Routers tend to collapse onto a few favoured experts unless training includes an auxiliary loss or other mechanism that encourages balanced use. Experts on different devices require tokens to be shuffled across the network, adding communication costs. And while computation per token falls, memory does not: all the experts must be held somewhere, so a mixture-of-experts model needs as much memory for its weights as a dense model of the same total size. Where attention fits Notice what mixture of experts changes and what it leaves alone. It makes the feed-forward layers sparse. It does nothing to attention. Every token still attends to every previous token, with the same quadratic score computation and the same key-value cache. Indeed, by making the feed-forward part cheaper per token, mixture of experts increases the share of total computation and memory that attention consumes. This pattern recurs throughout the architecture's evolution. Scale made models more capable, and more capable models made longer contexts worth having: an assistant that can read a whole codebase, a contract, or a book is more useful than one limited to a few pages. Context windows that were 2,048 tokens for GPT-3 in 2020 had become 128,000 tokens or more in widely available models by 2024, and a million tokens or more in some. The feed-forward layers, being position-wise, cost the same per token at any context length. Attention does not. By the time scaling had made models good enough to use long contexts well, attention had become the part of the transformer that most needed fixing. The next chapter measures exactly how much it costs. Chapter 8. The Quadratic Wall and the Memory Wall It is common to say that attention is quadratic in sequence length and leave it there. That statement is true and incomplete. It describes the arithmetic of one phase of computation. It does not describe the cost that actually limits long-context models in use, which is mostly a matter of moving bytes rather than multiplying them, and which grows linearly rather than quadratically. To understand which of the architecture's later changes addressed which problem, one has to count both arithmetic and memory, separately, for each phase. This chapter does that counting with real model dimensions. Two resources: arithmetic and bandwidth A modern accelerator, such as an NVIDIA H100 GPU, has two relevant limits. One is how fast it can do arithmetic. An H100 in its SXM form can perform on the order of a thousand trillion floating-point operations per second on the 16-bit matrix multiplications that dominate neural network computation. The other is how fast it can move data between its main memory, called high-bandwidth memory or HBM, and the compute units. For the same chip, that is about 3.35 terabytes per second. The ratio of these two numbers, roughly 300 operations per byte, determines which limit binds for a given computation. If a computation performs many operations for each byte it reads from memory, it is compute-bound: the arithmetic units are the bottleneck and bandwidth sits partly idle. If it performs few operations per byte, it is memory-bound: the arithmetic units wait for data. The number of operations per byte is called the computation's arithmetic intensity. This framing, formalised as the "roofline model" by Samuel Williams, Andrew Waterman and David Patterson in 2009, is the most useful single tool for reasoning about transformer performance. A large matrix multiplication has high arithmetic intensity. Multiplying two n-by-n matrices reads about 2n squared numbers and performs about 2n cubed operations, so its intensity grows with n. Element-wise operations, such as applying softmax or adding a residual, have low intensity: each number read is used a handful of times. Transformer computation is a mix, and which kind dominates depends on the phase. The chip also has a much smaller, much faster memory on the processor itself, called SRAM, amounting to a few tens of megabytes on an H100, much of it divided into small per-processor slices of a couple of hundred kilobytes each, with bandwidth many times that of HBM. Keeping data in SRAM, rather than writing it out to HBM and reading it back, is the central idea of the most important attention optimisation, described in Chapter 9. Counting the prefill Take a concrete model. Llama 2 70B, released by Meta in 2023, has 80 layers, model width d_model = 8,192, 64 query heads of dimension 128, and 8 key-value heads (an arrangement explained in Chapter 9). Consider a prompt of n tokens processed in a single prefill pass. The weight multiplications cost about 2 times the parameter count per token, about 140 billion operations per token, or 140 billion times n in total. This is linear in n. The attention score computation, per layer, computes QK^T for 64 heads, each an n by 128 matrix times a 128 by n matrix: about 2 times 64 times 128 times n squared, or roughly 16,000 times n squared operations. The weighted sum with V costs the same again. Across 80 layers, the total is about 2.6 million times n squared operations. With causal masking, about half of these can be skipped, giving roughly 1.3 million times n squared. Setting the two equal: 140 billion times n equals 1.3 million times n squared, which gives n of about 100,000. Below a prompt of roughly a hundred thousand tokens, the linear weight multiplications dominate prefill arithmetic for this model. Above it, attention dominates, and grows quadratically. At 4,000 tokens attention is about 4 percent of prefill arithmetic; at 32,000 tokens about a quarter; at a million tokens about 90 percent. The general rule from Chapter 3 gives the same picture: attention arithmetic per token per layer is about 4 times n times d_model, the weights cost about 24 times d_model squared, and the crossover sits near n = 6 d_model, here about 50,000 tokens before accounting for causal masking. Prefill is compute-bound in both regimes, because both the weight multiplications and the attention products are large matrix multiplications. The quadratic term therefore appears directly as wall-clock time. On a node of eight H100s, a million-token prefill for a model of this size takes minutes rather than seconds, and doubling the prompt more than doubles the wait. The memory of naive attention The arithmetic is not the only quadratic cost in prefill. A straightforward implementation computes the full score matrix S = QK^T / sqrt(d_k), writes it to HBM, reads it back to compute the softmax, writes the probabilities P, then reads P back to multiply by V. For one head and n = 32,000 tokens, S has about a billion entries, which at two bytes each is 2 gigabytes. For all heads in one layer simultaneously, it would exceed the memory of any single accelerator. Moreover, every one of those reads and writes is an element-wise pass with low arithmetic intensity. The softmax over an n-by-n matrix performs a few operations per entry and moves every entry in and out of HBM. In 2022, Tri Dao and colleagues measured that in standard implementations of attention for typical sequence lengths, much of the run time was spent not in the two matrix multiplications but in these memory-bound intermediate steps. The quadratic memory traffic of the score matrix, not its quadratic arithmetic, was the practical bottleneck. Chapter 9 explains how FlashAttention removed it without changing the result. Counting the decode Generation is a different story. After prefill, the model produces one token per step. At each step, for each layer, it computes the new token's query, key and value, appends the key and value to the cache, computes the query's scores against all n cached keys, takes a softmax, and forms the weighted sum of the n cached values. The arithmetic per step is modest: the weight multiplications for one token, about 140 billion operations for the 70B model, plus attention over n cached positions, about 4 times n times 8,192 times 80, or 2.6 million times n operations. At n = 100,000, that is about 260 billion operations for attention, comparable to the weights. But consider the memory traffic. To process one token, the model must read every weight from HBM: 70 billion parameters at two bytes each is 140 gigabytes. It must also read the entire key-value cache. The arithmetic intensity of the weight multiplications is about one operation per byte for a single sequence, three hundred times below the H100's balance point. Decode is overwhelmingly memory-bound. Generating one token cannot take less time than reading the weights and cache from memory. This is why serving systems batch many users' requests together. The weights are read once per step and used for every sequence in the batch, multiplying arithmetic intensity by the batch size. A batch of 64 sequences raises the weight multiplications to about 64 operations per byte, still memory-bound but far better. The key-value cache, however, cannot be shared this way. Each sequence has its own cache, and each must be read in full at every step. Batching amortises weights but not the cache. The key-value cache in numbers The size of the cache follows directly from the architecture. For each layer and each key-value head, for each token, it stores one key vector and one value vector of dimension d_head. In bytes: KV cache = 2 × L × h_kv × d_head × n × bytes per number where the leading 2 counts keys and values. Table 3 works this out at 16-bit precision for three published model configurations. Table 3. Key-value cache size per sequence at 16-bit precision, computed from published architecture settings. Model Layers KV heads × head dim Cache per token Cache at 4,096 tokens Cache at 128,000 tokens GPT-3 175B (full multi-head) 96 96 × 128 4.7 MB 19 GB 604 GB Llama 2 70B (grouped-query) 80 8 × 128 0.33 MB 1.3 GB 42 GB Llama 3 8B (grouped-query) 32 8 × 128 0.13 MB 0.5 GB 16.8 GB The configurations are those published by OpenAI for GPT-3 and by Meta for the Llama models; the cache sizes are computed from the formula above. GPT-3 never ran at 128,000 tokens; the column shows what its full multi-head design would have required. Several facts stand out. First, under full multi-head attention, a large model's cache for a single long sequence would exceed the memory of an entire eight-GPU server, which holds 640 gigabytes. Second, the reduction from 96 key-value heads to 8, in otherwise comparable designs, cuts the cache by more than an order of magnitude; Chapter 9 explains how that reduction is achieved without destroying quality. Third, even with that reduction, a 128,000-token context for Llama 2 70B needs 42 gigabytes per sequence, about a third of the model's own weights. Serve thirty such sequences at once and the caches outweigh the model several times over. For the small model the situation is starker. Llama 3 8B's weights occupy about 16 gigabytes at 16-bit precision. Its cache for one 128,000-token sequence is about the same size. In long-context serving, the cache is often the largest single consumer of accelerator memory, and since every byte of it must be read at every decoding step, it is the dominant cost of generation. Where the walls actually stand The analysis yields a clearer statement of the transformer's long-context problem than "attention is quadratic." During training and prefill, attention's arithmetic is quadratic in n. For large models it overtakes the cost of everything else only at contexts of tens of thousands to a hundred thousand tokens, depending on model width. Below that, it is a significant but not dominant cost. Above it, it dominates, and the time to process a long prompt grows with its square. In naive implementations, the memory traffic of materialising the n-by-n score matrix was a separate and more immediate problem, binding at much shorter lengths. This was an implementation problem rather than a mathematical one, and it has largely been solved. During generation, the binding cost is linear, not quadratic: the size of the key-value cache, which must be stored for every active sequence and read in full at every step. It limits how many users a server can handle at once and how fast each one's text appears. Shrinking the cache is therefore the single most valuable thing an architecture can do for long-context serving. These three costs have different remedies. Faster kernels address the second. Changes to the attention heads address the third. Only changes to the attention computation itself, replacing exact all-pairs comparison with something cheaper, address the first. The next chapter describes the first two kinds of remedy, which kept exact attention competitive. The chapter after that describes the third, and explains why it has been so much harder to make work. Chapter 9. Keeping Exact Attention: Kernels, Heads, and Caches Faced with the costs set out in the previous chapter, there are two broad strategies. One is to change what attention computes, replacing the full comparison of every query with every key by something cheaper. The other is to keep the computation, or something very close to it, and change how it is carried out and what it stores. In practice the second strategy has done far more for the long-context transformers in production use. This chapter covers its three main lines: kernels that compute exact attention without materialising the score matrix, changes to the head structure that shrink the key-value cache, and systems techniques that manage, compress, and distribute that cache. FlashAttention In May 2022 Tri Dao, Daniel Fu, Stefano Ermon, Atri Rudra and Christopher Ré posted "FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness." The key word was IO-awareness. Attention's arithmetic was not changed at all. What changed was the order in which it was performed, so as to minimise traffic between the GPU's large, slow HBM and its small, fast SRAM. The obstacle to computing attention in small pieces is the softmax. Each output row requires normalising over all n scores in that row, which seems to require all of them at once. FlashAttention solves this with a technique known as online softmax, described by Maxim Milakov and Natalia Gimelshein at NVIDIA in 2018 and used for attention by Markus Rabe and Charles Staats in 2021. The idea is to process the keys in blocks while maintaining two running statistics for each query: the largest score seen so far, m, and the sum of exponentials of scores seen so far, each measured relative to m, called l. Along with these, keep a running unnormalised output vector o. When a new block of keys and values arrives, compute its scores, find the block's maximum, and update m to the larger of the old and new maxima. Rescale the old l and o by exp(old m minus new m), to express them relative to the new maximum. Add the new block's exponentials to l and their weighted values to o. After the last block, divide o by l. The result is exactly the softmax-weighted average, computed without ever holding a whole row of scores. Subtracting the running maximum keeps every exponential at or below one, so there is no risk of numerical overflow. FlashAttention combines this with tiling. It loads a block of queries into SRAM, then streams blocks of keys and values through, computing scores, updating the running statistics, and accumulating outputs, all within SRAM. Only the final output for that block of queries is written to HBM. The n-by-n score matrix never exists in slow memory. Memory use for attention falls from quadratic to linear in n, and HBM traffic falls by a large factor. For the backward pass, which in a naive implementation reads the stored probability matrix, FlashAttention stores only the per-row statistics m and l and recomputes the scores block by block when gradients are needed. This trades some extra arithmetic for far less memory traffic, a good trade on hardware where arithmetic is cheap and bandwidth is not. The original paper reported speedups of around three times on GPT-2 training at sequence length 1,024 compared with standard implementations, and made it feasible to train with sequence lengths of 16,000 and 64,000 tokens on hardware where standard attention ran out of memory. FlashAttention-2, in 2023, reorganised the parallelism across GPU thread blocks and reduced non-matrix operations, roughly doubling speed again. FlashAttention-3, in 2024, targeted the H100 specifically, overlapping data movement with computation using the chip's asynchronous hardware units and supporting 8-bit floating-point arithmetic, and reached around three-quarters of the chip's theoretical peak in 16-bit precision. The significance of FlashAttention for the argument of this booklet is that it removed one of the three walls identified in Chapter 8, the quadratic memory traffic of the materialised score matrix, entirely, without approximation. Exact attention became fast enough that, for contexts up to tens of thousands of tokens, approximations offered little practical advantage. FlashAttention did not change the quadratic arithmetic of prefill, and it did not shrink the key-value cache. Those problems remained. Fewer keys and values: multi-query and grouped-query attention The cache stores keys and values for every head. If several query heads could share the same keys and values, the cache would shrink proportionally. Noam Shazeer proposed exactly this in a 2019 paper with the self-explanatory title "Fast Transformer Decoding: One Write-Head is All You Need." In multi-query attention, or MQA, the model keeps many query heads but uses a single key head and a single value head shared by all of them. Each query head still computes its own attention pattern, because its query projection differs, but all heads look up the same keys and retrieve from the same values. The cache shrinks by a factor equal to the number of heads, 32 or 64 or more. Shazeer showed that decoding became much faster with only a small loss in quality. PaLM, Google's 540-billion-parameter model of 2022, used MQA. The quality loss was real, though, and some models trained with MQA were less stable. In 2023 Joshua Ainslie and colleagues at Google proposed grouped-query attention, or GQA, as an intermediate. The query heads are divided into G groups, and each group shares one key head and one value head. With 64 query heads and 8 groups, every 8 query heads share a key-value pair, and the cache is 8 times smaller than full multi-head attention. Ainslie's team found GQA with 8 groups recovered nearly all the quality of multi-head attention while keeping most of the speed of MQA. They also showed that an existing multi-head model could be converted to GQA by averaging its key and value heads within each group and continuing training for a small fraction of the original compute. GQA became the standard. Llama 2's 70B model used it, as Table 3 showed; so did all sizes of Llama 3, Mistral, and many others. The reduction from 64 key-value heads to 8 is why a 70-billion-parameter model of 2023 needs fourteen times less cache memory per token than GPT-3 did, a larger gain than most changes to attention's mathematics have achieved. Compressing keys and values: multi-head latent attention DeepSeek-V2, released by DeepSeek-AI in May 2024, introduced a further step called multi-head latent attention, or MLA. Instead of reducing the number of key and value heads, MLA compresses keys and values into a shared low-dimensional latent vector per token. For each token the model computes a single compressed vector c, of dimension much smaller than the total width of all keys and values, and caches only that. At attention time, keys and values for every head are reconstructed from c by learned up-projections. That might seem to require reconstructing full keys and values at each step, which would cost arithmetic. But because the up-projection for keys and the query projection are both linear, they can be multiplied together in advance and absorbed: the query can be projected directly into the latent space and compared with the cached compressed vectors, and the same absorption can be applied on the value side via the output projection. The model attends over compressed vectors without ever expanding them. A complication is RoPE. The rotation depends on position and sits between the query and key projections, so it prevents the absorption trick. DeepSeek's solution was to split each head's query and key into two parts: a larger part with no positional rotation, which goes through the compressed path, and a small additional part that carries RoPE and is cached separately and shared across heads. DeepSeek reported that MLA reduced the cache by about 93 percent relative to the multi-head attention of their earlier 67-billion-parameter model while improving quality on their benchmarks. DeepSeek-V3 retained the design. MLA shows that the head-sharing logic of MQA and GQA, which reduces the cache by restricting how many distinct keys and values there are, can be generalised into a learned compression that adapts to what the model needs. Restricting the window A different way to bound the cache is to stop attending to everything. In sliding-window attention, each position attends only to the previous w tokens. The cache never holds more than w entries per layer, and the arithmetic per token is bounded by w instead of n. The idea goes back to the local attention suggested in the original transformer paper and to models such as Longformer, by Iz Beltagy, Matthew Peters and Arman Cohan in 2020, which combined a sliding window with a few "global" tokens that attended everywhere. A sliding window does not make distant information invisible. Because layers are stacked, information can propagate: a token at position 10,000 in layer two can attend to position 6,000, whose layer-one representation attended to position 2,000. With L layers and window w, the receptive field reaches about L times w. Mistral 7B, released in September 2023, used a window of 4,096 tokens across 32 layers, giving a theoretical reach of about 131,000 tokens. But information passed through many hops is attenuated and compressed, much as in a recurrent network. Precise retrieval of a specific distant token, the kind of operation induction heads perform, becomes unreliable. Recent models have therefore mixed local and global layers. Google's Gemma 2, in 2024, alternated sliding-window layers with a 4,096-token window and full-attention layers. Gemma 3, in 2025, used five local layers for every global one with a window of 1,024 tokens, so that most layers have small caches and only a sixth of the layers need cache proportional to the full context. The global layers preserve exact long-range retrieval; the local layers handle the bulk of the computation cheaply. This is a pragmatic acknowledgement that most of what most layers do is local, while some of what some layers do requires the whole context. Managing the cache as a systems problem Even with smaller caches, serving many users with different, growing context lengths raises problems of allocation. Early serving systems reserved a contiguous block of memory for each sequence's maximum possible length, wasting most of it. In 2023 Woosuk Kwon and colleagues at the University of California, Berkeley, introduced PagedAttention and the vLLM serving system, which borrowed the idea of virtual memory from operating systems. The cache is divided into fixed-size blocks, each holding keys and values for a small number of tokens, and a block table maps each sequence's logical positions to physical blocks, which need not be contiguous. Memory is allocated as sequences grow, fragmentation nearly disappears, and sequences that share a common prefix, such as the same system prompt, can share the physical blocks for it. Kwon's team reported throughput improvements of two to four times over the serving systems they compared against. Prefix sharing has become an economic feature in its own right. Several commercial providers now offer reduced prices for prompts whose beginnings match a previously processed prompt, because the cache for that prefix can be reused instead of recomputed. Other approaches compress what is stored. Keys and values can be quantised from 16 bits to 8 or fewer with modest quality loss. Some methods evict cache entries judged unimportant: the H2O method of Zhenyu Zhang and colleagues in 2023 keeps tokens that have received the most cumulative attention, the "heavy hitters," along with a recent window. StreamingLLM, described in Chapter 6, keeps attention-sink tokens plus a recent window. These eviction schemes trade exactness for memory, and they suffer from a basic difficulty: a token that seems unimportant now may be exactly the one a later query needs. Distributing a single long sequence For contexts of a million tokens or more, even an optimised cache and FlashAttention on a single chip are not enough, because one device's memory cannot hold the activations. Ring Attention, by Hao Liu, Matei Zaharia and Pieter Abbeel in 2023, splits the sequence into blocks across a ring of devices. Each device holds the queries for its block and computes attention against keys and values that are passed around the ring, one block at a time, using the same online-softmax accumulation as FlashAttention. Communication of the next block overlaps with computation on the current one. The maximum context length then scales with the number of devices. Similar techniques, under names such as context parallelism and sequence parallelism, are now standard in training systems for long-context models. What this approach achieved, and what it left Taken together, these techniques explain how context windows grew by roughly two orders of magnitude between 2020 and 2024 while the core attention computation stayed exact. FlashAttention made exact attention efficient in memory traffic and memory footprint. Grouped-query and latent attention cut the cache by factors of eight to more than ten. Sliding-window and hybrid layer schedules cut it further in most layers. Paged memory management and prefix sharing made the remaining cache cheaper to serve. Ring-style parallelism spread the longest contexts across many devices. None of this changed the asymptotic facts. Prefill arithmetic is still quadratic in the prompt length for every layer with full attention. The cache in every such layer still grows linearly with context, and must be read in full at every step of generation. Exact attention survived because engineering pushed back the point at which those facts become intolerable. It did not abolish them. The question of whether attention itself could be replaced by something with better asymptotics was pursued throughout the same period, and its answer is the subject of the next two chapters. Chapter 10. The Search for Cheaper Attention From 2019 onward, a large literature set out to replace attention's n-squared cost with something that grows linearly, or nearly so, in sequence length. The proposals were ingenious and drew on sparse matrices, hashing, low-rank approximation, and random features from kernel theory. Many reported large speedups on long sequences. Very few made it into the leading models. This chapter describes the main families, the mathematics each relies on, and why, in the main, they fell short. The reasons are instructive, because they show what full attention provides that approximations struggle to keep. Sparse patterns The most direct approach is to compute only some of the n-squared scores. If each query attends to a fixed set of keys of size s much smaller than n, the cost falls from n squared to n times s. In 2019 Rewon Child, Scott Gray, Alec Radford and Ilya Sutskever at OpenAI described the Sparse Transformer. It used factorised patterns: one head attends to the previous sqrt(n) positions, another to every sqrt(n)-th position. Any position can reach any other in two hops, and the cost per layer is n times sqrt(n). They used it to model images, audio and text with sequences of tens of thousands of elements. GPT-3 alternated dense layers with locally banded sparse ones, following the same work. Longformer, mentioned in Chapter 9, combined a sliding window with global tokens. BigBird, from Manzil Zaheer and colleagues at Google in 2020, added random connections: each query attended to a local window, a few global tokens, and a few randomly chosen keys. The authors proved that this pattern retained theoretical properties of full attention, including the ability to approximate any sequence-to-sequence function given enough layers, and showed gains on long-document question answering and genomics. Sparse patterns are effective when the important interactions are predictable in advance: mostly local, with a few known hubs. Their weakness is the opposite case. When the token a query needs could be anywhere, and its location depends on content rather than position, a fixed pattern will miss it unless it routes through many hops, and the analysis of Chapter 6 shows that trained models depend on exactly this kind of content-addressed lookup. Content-based sparsity A natural refinement is to choose which keys each query attends to by content. Since softmax concentrates weight on the largest scores, the output is dominated by the keys most similar to the query. If those could be found cheaply, the rest could be ignored. Reformer, by Nikita Kitaev, Łukasz Kaiser and Anselm Levskaya in 2020, used locality-sensitive hashing for this. A hash function of this type maps similar vectors to the same bucket with high probability. Reformer hashed queries and keys, which it tied to be the same vectors, sorted them by bucket, and computed attention only within buckets and neighbouring chunks, giving cost proportional to n log n. The Routing Transformer, by Aurko Roy and colleagues in 2021, used online k-means clustering instead of hashing. These methods were clever but awkward on hardware. Sorting, gathering and scattering tokens by bucket are irregular memory operations that GPUs handle poorly compared with dense matrix multiplication. Hash collisions are probabilistic, so relevant keys sometimes land in the wrong bucket. And at the sequence lengths of practical interest, a well-optimised dense implementation often ran faster than the theoretically cheaper sparse one. Content-based sparsity has returned in recent years in a new form, discussed at the end of this chapter, but its first generation did not displace dense attention. Low-rank projections Another family exploits the observation that attention matrices in trained models are often approximately low-rank. Linformer, by Sinong Wang and colleagues at Facebook AI in 2020, projected the n keys and n values down to a fixed k with learned matrices of size k by n, then computed attention from n queries to k compressed keys. Cost becomes n times k. The authors showed that on BERT-style tasks, k of 128 or 256 sufficed for sequences of several thousand. Linformer's design has a basic incompatibility with causal language models. The projection mixes all n keys, including future ones, into each compressed key, which breaks the causal mask. It also fixes n at training time, since the projection matrix has one column per position. Nyströmformer, in 2021, used a landmark-based approximation from kernel methods with similar strengths and limitations. Low-rank methods suited bidirectional encoders on fixed-length inputs, which was not where the field was heading. Linear attention and the kernel trick The most mathematically consequential family is linear attention, because it connects attention to recurrence and leads directly to the state-space models of Chapter 11. Write attention for a single query q_i, ignoring the scaling, as: output_i = sum over j of sim(q_i, k_j) v_j / sum over j of sim(q_i, k_j) where sim(q, k) = exp(q · k). The difficulty is that exp(q · k) does not factor into a function of q times a function of k, so one must compute it for every pair. Suppose instead that the similarity could be written as a dot product of feature maps, sim(q, k) = phi(q) · phi(k), for some function phi mapping vectors to vectors. Then: output_i = phi(q_i)^T (sum over j of phi(k_j) v_j^T) / phi(q_i)^T (sum over j of phi(k_j)) The sums in brackets do not depend on i. Compute them once, in time linear in n, and every query's output follows from a matrix-vector product. The n-by-n matrix never appears. The cost is n times the square of the feature dimension instead of n squared times d. Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas and François Fleuret developed this in a 2020 paper titled "Transformers are RNNs." They used a simple feature map, phi(x) = elu(x) + 1, which keeps features positive. Their title pointed to the second consequence. In the causal case, each position's sums run only over j up to i. Define a state matrix S_i = sum over j ≤ i of phi(k_j) v_j^T, and a normaliser z_i = sum over j ≤ i of phi(k_j). Then: S_i = S_(i-1) + phi(k_i) v_i^T output_i = phi(q_i)^T S_i / phi(q_i)^T z_i This is a recurrent network. The state S has fixed size, independent of sequence length. At each step, the model writes an outer product of the new key and value into the state and reads from it with the query. Generation needs no growing cache: just the fixed-size state, updated once per token. Training can still be done in parallel, because the cumulative sums can be computed with parallel scan algorithms or in chunks. Choromanski and colleagues at Google followed in 2020 with the Performer, which used random features to approximate the exponential kernel itself, so that phi(q) · phi(k) approximates exp(q · k) in expectation. This made linear attention an approximation of softmax attention, rather than a different operation. Random Feature Attention, by Hao Peng and colleagues in 2021, took a similar approach and added a gating mechanism. Why linear attention lagged Linear attention delivered exactly what it promised on cost. On quality, it fell short, particularly on language modelling. Two explanations, one about capacity and one about selection, account for most of the gap. The capacity argument is about the state. Full attention stores every key and value; its memory of the past is lossless and grows with n. Linear attention compresses the whole past into a matrix S of fixed size, d_phi by d_v. It can store about d_phi independent key-value associations before new writes begin to interfere with old ones. Imanol Schlag, Kazuki Irie and Jürgen Schmidhuber made this precise in a 2021 paper, "Linear Transformers Are Secretly Fast Weight Programmers," connecting linear attention to "fast weight" memories studied in the early 1990s. Writing an outer product into S is like storing an association in a linear associative memory, and such memories have finite capacity. Once the number of stored items exceeds the dimension, retrieval becomes noisy. Schlag and colleagues proposed replacing the additive write with a delta-rule update, which first retrieves what the memory currently associates with the key and writes only the correction, reducing interference. The selection argument is about the softmax. Exponentiation makes softmax attention sharp: a query can place almost all its weight on a single matching key among thousands. The feature maps used in linear attention produce much flatter weightings, because a positive feature map with modest dimension cannot replicate the extreme peakiness of exp. A linear-attention model asked to retrieve one specific token from a long context gets a blur. Random-feature approximations can in principle capture the exponential, but need many features to do so with low variance, eroding the cost advantage. Both problems bear directly on the induction-head mechanism. Copying the token that followed an earlier occurrence of the current token requires finding one exact position among many and retrieving it cleanly. A bounded, blurred memory is poorly suited to that. A survey of these methods published in 2020, "Efficient Transformers: A Survey" by Yi Tay, Mostafa Dehghani, Dara Bahri and Donald Metzler, organised the literature into a taxonomy, and a companion benchmark from the same group, Long Range Arena, compared many of the methods on tasks with sequences of 1,000 to 16,000 elements. No method dominated across tasks, and the gains in speed were often smaller in practice than the asymptotic analysis suggested. The benchmark's tasks were themselves later criticised as not reflecting what matters for language modelling. Table 4 summarises the families described in this chapter. Table 4. Main families of efficient attention approximations. Family Representative methods Cost per layer Causal generation Main limitation Fixed sparse patterns Sparse Transformer, Longformer, BigBird n × sqrt(n) or n × w Yes Misses content-dependent distant links Content-based sparsity Reformer, Routing Transformer n × log n Awkward Irregular memory access; hash misses Low-rank projection Linformer, Nyströmformer n × k No (Linformer) Mixes future tokens; fixed length Kernelised linear attention Linear Transformer, Performer n × d squared Yes, with constant state Bounded memory; blurred selection The second wind of linear models The linear-attention line did not end with these results. It changed course. Researchers stopped trying to approximate softmax attention and instead treated the recurrent form as a model class in its own right, to be improved directly. Three ideas proved important. Adding a decay, so that old associations fade at a learned rate, let the state forget what it no longer needed: S_i = gamma S_(i-1) + phi(k_i) v_i^T, with gamma between zero and one. Making that decay depend on the input, a gate computed from the current token, let the model decide what to forget and when. And replacing the additive update with the delta rule reduced interference. RetNet, from Microsoft researchers in 2023, used a fixed decay. Gated Linear Attention, by Songlin Yang and colleagues in 2023, used data-dependent gates and developed hardware-efficient chunked training algorithms. DeltaNet and Gated DeltaNet, from Yang and colleagues in 2024, combined the delta rule with gating. RWKV, an open-source project led by Bo Peng, developed a similar recurrent formulation through several versions from 2021 onward. These models closed much of the gap with transformers on language modelling perplexity at moderate scale. They converge, from the attention side, on the same territory that state-space models reached from the control-theory side, which the next chapter takes up. Learned sparse attention Meanwhile, content-based sparsity returned in a form designed for hardware. The insight is that sparsity should be chosen at the granularity of blocks, not individual tokens, so the selected keys can be processed with dense matrix multiplication. Native Sparse Attention, from DeepSeek researchers in early 2025, combined three branches: attention over compressed summaries of blocks of past tokens, attention over the top-scoring full blocks selected by those summaries, and a sliding window. The selection is learned during pre-training rather than imposed afterwards. The authors reported quality matching full attention on their benchmarks and substantial speedups at a context of 64,000 tokens. DeepSeek later deployed a related mechanism in a production model. Moonshot AI's Mixture of Block Attention, also from 2025, took a similar approach, routing each query to a few relevant key blocks. These designs differ from the first generation of sparse attention in two ways. They are built around the memory hierarchy of GPUs rather than abstract operation counts, and the decision of where to look is made by the model itself, trained end to end. Whether they will become standard is not yet settled. But they reflect what the preceding decade of work taught: the expensive part of attention is the large number of keys that do not matter to a given query, but the model must be able to find, cheaply and reliably, the few that do. Chapter 11. Challengers, Hybrids, and the Meaning of Long Context By 2023 the arguments of the previous chapters had produced a clear picture. Full attention gives every position lossless, content-addressed access to every earlier position, and trained models depend on that access. Its price is quadratic prefill arithmetic and a key-value cache that grows with every token. Recurrent models with a fixed-size state have neither cost, but their bounded memory struggles with precise retrieval. The question was whether a recurrent architecture could be made good enough to match transformers on language, and if not, whether the two could be combined so that each did what it does best. This chapter follows the most serious challenger, the selective state-space model, through to the hybrid designs that have emerged as a working answer, and then asks what long context actually delivers once it is measured carefully. State-space models State-space models come from control theory and signal processing, where they describe a system whose internal state evolves continuously in response to an input. In continuous time: h'(t) = A h(t) + B x(t) y(t) = C h(t) Here x is the input signal, h is a state vector, y is the output, the prime denotes the time derivative, and A, B and C are matrices. To apply this to a sequence of tokens, the equations are discretised with a step size Delta, which gives a linear recurrence of the familiar form: h_t = A_bar h_(t-1) + B_bar x_t y_t = C h_t where A_bar and B_bar are computed from A, B and Delta. This looks like a recurrent network without the nonlinearity, and that absence is the point. Because the recurrence is linear and its matrices are the same at every step, unrolling it shows that the output is a convolution of the input with a fixed kernel whose entries are C A_bar^k B_bar for k = 0, 1, 2, and so on. A linear time-invariant state-space model can therefore be trained in parallel as a long convolution, computed efficiently with the fast Fourier transform, and run at inference as a recurrence with constant memory per step. Albert Gu, Karan Goel and Christopher Ré made this practical in their 2021 paper on S4, "Efficiently Modeling Long Sequences with Structured State Spaces." The main obstacles had been numerical: computing powers of A for long sequences is unstable for a generic matrix, and a poorly chosen A forgets information quickly. S4 initialised A using a construction from Gu's earlier HiPPO work, designed so that the state optimally summarises the history of the input as a set of coefficients for a family of polynomials, and parameterised A in a structured form that made the convolution kernel cheap to compute. S4 achieved strong results on the Long Range Arena benchmark, including a task with sequences of 16,000 elements on which every transformer variant had performed at chance. S4 and its successors were excellent on continuous signals such as audio and on synthetic long-range tasks, but they lagged transformers on language. In 2022 Daniel Fu, Tri Dao and colleagues diagnosed one reason. Studying synthetic tasks, they found that state-space models struggled with associative recall: given a list of key-value pairs and then a key, return its value. This is the induction-head task in another guise. A time-invariant model applies the same dynamics to every token. It cannot decide, based on the content of the current token, that this token is worth remembering and the last fifty were not. Selectivity: Mamba In December 2023 Albert Gu and Tri Dao released Mamba, described in "Mamba: Linear-Time Sequence Modeling with Selective State Spaces." Its central change was to make the state-space parameters functions of the input. The step size Delta, and the matrices B and C, are computed at each position from the current token through small learned projections. A large Delta for a given token means the state is updated strongly toward that token's input and older content decays quickly; a small Delta means the token is largely ignored and the state is preserved. The model can thus choose, token by token, what to write into its memory and what to let pass. Selectivity breaks the convolution trick, since the dynamics now differ at every step. Gu and Dao replaced it with a hardware-aware parallel scan: the recurrence is computed with an associative scan algorithm, with the expanded state kept in fast on-chip SRAM rather than written to HBM, in the same spirit as FlashAttention. The result trains in time linear in sequence length and generates with a fixed-size state and no growing cache. Mamba's reported results were striking. At sizes up to about three billion parameters, trained on 300 billion tokens, Mamba matched or exceeded transformers of the same size, and in some comparisons transformers twice its size, on language modelling and on standard downstream benchmarks. Its inference throughput was several times higher than a transformer of similar size, because it had no cache to read. Mamba-2, by Dao and Gu in May 2024, made a theoretical connection explicit in a paper titled "Transformers are SSMs." They showed that a selective state-space model with a scalar-times-identity structure for A is mathematically equivalent to a form of linear attention with a particular data-dependent decay mask, and that both can be written as multiplication by a structured matrix they called semiseparable. Softmax attention corresponds to a different, full-rank matrix of the same general shape. This "state space duality" unified the gated linear attention models of Chapter 10 and the selective state-space models into a single family. Its practical consequence was a training algorithm that processes the sequence in chunks, using dense matrix multiplication within each chunk and the recurrence between them, which ran substantially faster than Mamba's original scan on modern GPUs. Where recurrent models fall short As selective state-space models and gated linear attention models were scaled and tested more broadly, a consistent pattern emerged. On average perplexity and on many standard benchmarks, they were close to transformers. On tasks requiring precise recall or copying from a long context, they were worse, sometimes much worse. Simran Arora and colleagues, in the 2023 "Zoology" study, measured this systematically. They found that a large majority of the perplexity gap between attention-free models and transformers on natural text was attributable to tokens that could be predicted by recalling an earlier association in the context, such as a name or phrase used earlier in the document. Samy Jelassi, David Brandfonbrener, Sham Kakade and Eran Malach, in a 2024 paper titled "Repeat After Me: Transformers are Better than State Space Models at Copying," proved that a two-layer transformer can copy strings of length exponential in its size, whereas any model with a fixed-size state cannot copy strings longer than its state can hold, and showed experimentally that transformers learned to copy far more efficiently. Gated recurrent models with large states narrow the gap, and the delta-rule variants narrow it further, but the theoretical limit remains: a fixed-size state is a lossy summary, and information that was not chosen for retention when it streamed past cannot be retrieved later. This is the same trade-off seen at the start of this booklet, now at a much higher level of sophistication. Recurrent networks were displaced because they compressed the past into a fixed vector. Selective state-space models are vastly better at deciding what to compress, but they still compress. Attention does not compress, and pays for it. Hybrids The response, by 2024, was to stop treating the question as a contest. If attention is needed for precise retrieval but a recurrent layer handles most other sequence processing more cheaply, a model can be built from both. AI21 Labs released Jamba in March 2024, interleaving Mamba layers with transformer attention layers in a ratio of about seven to one and adding mixture-of-experts feed-forward layers. It supported a context of 256,000 tokens with a cache a fraction of the size an all-attention model of similar scale would need, since only one layer in eight stored keys and values. In June 2024 researchers at NVIDIA, led by Roger Waleffe, published a controlled comparison at the 8-billion-parameter scale, training pure transformers, pure Mamba and Mamba-2 models, and hybrids on the same data. The pure state-space models fell behind on tasks requiring in-context recall and on few-shot learning. A hybrid composed mostly of Mamba-2 layers with a small fraction of attention layers, around 7 percent of all layers, matched or exceeded the pure transformer on the standard tasks they evaluated while promising much faster generation. The pattern has spread. MiniMax-01, released in January 2025, combined a linear-attention variant with standard softmax attention, using one softmax layer for every seven linear ones, and its developers reported handling contexts of up to four million tokens at inference. Alibaba's Qwen3-Next, released later in 2025, interleaved Gated DeltaNet layers with full attention layers in a three-to-one ratio. Google's Gemma 3, described in Chapter 9, applied the same reasoning within the transformer family, using cheap sliding-window attention for most layers and full attention for a few. The logic is the same in each case. Most of a long context's processing is local or summary-like and can be handled by layers with bounded memory. A minority of operations, such as finding a specific earlier name, quotation, function definition or fact, requires lossless access, and a few full-attention layers can supply it. The cache cost of those few layers still grows with context, but it grows at a fraction of the rate of an all-attention model. This combination is the closest thing the field now has to a settled view of the efficient long-context architecture, though the best ratio, the best recurrent component, and the best placement of attention layers remain under active study. What long context actually delivers Architectural progress pushed context windows from a few thousand tokens to hundreds of thousands and beyond. Google's Gemini 1.5, announced in February 2024, offered a million tokens in production and reported research tests up to ten million. Whether models make good use of such windows is a separate question, and the answer has been more qualified than the headline numbers suggest. The first widely used probe was the needle-in-a-haystack test, popularised by Greg Kamradt in late 2023: insert a single out-of-place sentence somewhere in a long document and ask the model to retrieve it. Many models scored near perfectly. But the test is easy in a specific way. The needle is lexically distinct from the surrounding text, so a single query can find it by matching words, which is exactly the operation induction-style heads perform well. Harder tests produced weaker results. Nelson Liu and colleagues, in a 2023 paper titled "Lost in the Middle," gave models a question and a set of documents, only one of which contained the answer, and varied the position of that document. Performance was highest when the relevant document was at the beginning or end of the context and dropped substantially when it was in the middle, producing a U-shaped curve. In some configurations, models did worse with the relevant document in the middle of the context than when given no documents at all. The RULER benchmark from Cheng-Ping Hsieh and colleagues at NVIDIA in 2024 extended needle tests with multiple needles, variable tracking across the context, and aggregation tasks such as counting the most common words. They found that although nearly all models they tested passed simple needle retrieval, most showed large drops on the harder tasks as context grew, and that only about half of the models claiming context sizes of 32,000 tokens or more maintained satisfactory performance at 32,000 tokens. Why should a model with exact attention to every token fail to use a token it can see? Several causes contribute. Training data contains few documents in which the answer to a question depends on information many thousands of tokens away, so models get little practice. Positional schemes stretched by the interpolation methods of Chapter 4 may degrade the precision of attention at long range. Attention weight is a finite resource: softmax distributes a total of one across all visible tokens, and as the number of distractors grows, a relevant token must have a much higher score to win the same share. And tasks that require combining many pieces of scattered information demand many retrieval steps, each a possible point of failure. Long-context evaluation has therefore become a field in its own right, and developers have increasingly trained specifically on long, structured documents and on synthetic retrieval and reasoning tasks spanning long contexts. The general result is that the usable context of a model, the length at which it reliably performs tasks that genuinely depend on the whole input, is often well below its advertised maximum, and depends heavily on the task. Retrieval as an alternative Finally, there is an alternative to long context that sidesteps the architecture altogether. Retrieval-augmented generation, introduced under that name by Patrick Lewis and colleagues in 2020, keeps a large collection of documents outside the model, uses an embedding model of the kind described in Chapter 5 to find the passages most relevant to a query, and places only those in the model's context. Rather than letting attention search a million tokens, an external index searches them and attention handles the few thousand that are retrieved. Retrieval and long context are complements rather than rivals. Retrieval scales to collections far larger than any context window and costs little per query, but it can fail when relevance is not apparent from the query alone, or when a task requires synthesising information spread thinly across a whole document. Long context handles those cases better, at higher cost. Many deployed systems now combine the two: retrieve generously, then let a long-context model read what was retrieved. In architectural terms, retrieval is one more way of solving the problem that runs through this booklet. Attention lets every position consult every other, and when "every other" becomes too many, something must decide in advance which ones are worth consulting. Conclusion The transformer began as a substitution. Recurrence had carried information along a chain, step by step, through a vector that was overwritten at every position. Attention replaced the chain with direct, content-addressed connections between every pair of positions, computed in parallel. The preceding chapters have followed what that substitution made possible and what it cost, and the shape of the architecture's history now looks less like a series of inventions than like a long negotiation over a single exchange rate. What the evidence shows Several conclusions follow from the material, and they are firmer than the usual commentary about the field suggests. First, the transformer's advantage was never only about long-range dependencies. The original comparison with recurrent and convolutional layers counted three things, and the decisive one in practice was the second: a constant number of sequential operations. That made training parallel, which made it possible to use the hardware efficiently, which made scale affordable. The scaling laws of Chapter 7 are, in an important sense, a consequence of the parallelism argument of Chapter 1. An architecture that could not use a thousand GPUs efficiently could not have been tested at the scales where its predictable improvement became visible. Second, full attention is not merely a generous connectivity pattern that models mostly leave unused. The interpretability research of Chapter 6 found specific circuits, induction heads above all, that depend on fetching an exact earlier token chosen by content from an arbitrary distance. The capacity and recall studies of Chapters 10 and 11 found that the main thing models lose when attention is replaced by a fixed-size state is precisely this operation. These are two independent lines of evidence pointing to the same conclusion. Lossless, content-addressed lookup into the past is what makes attention valuable, and it is what is hardest to keep while making attention cheaper. Third, the cost that constrains long-context transformers in practice is not the one usually named. Quadratic arithmetic in prefill is real, and above a hundred thousand tokens or so it dominates. But for most deployments the binding constraint is the key-value cache, which grows linearly, consumes memory that could otherwise serve more users, and must be read in full for every generated token. The most effective interventions of the past several years, grouped-query attention, latent attention, sliding windows interleaved with global layers, and hybrid stacks in which only a few layers hold a full cache, are all attacks on that linear cost. The sustained effort to approximate the quadratic term, surveyed in Chapter 10, produced fewer production successes, in part because it was aimed at the less pressing problem and in part because its approximations damaged exactly the retrieval capability the circuit work identified. Fourth, position turned out to be an engineering lever rather than a fixed feature. The transition to rotary embeddings did not make models extrapolate better by default. It gave a compact, analytic description of position that could be rescaled after training, which turned context extension from a matter of retraining into a matter of fine-tuning. Much of the growth in advertised context windows rests on that property. What remains unsettled Some questions the evidence does not yet answer. The best architecture for long sequences is not known. Hybrids of attention and recurrent layers are the current working consensus among groups trying to reduce serving cost, but the right ratio, the right recurrent component, and the right placement of attention layers are being explored by trial and error. It is possible that learned sparse attention, which keeps the cache but reads only part of it, will prove a better trade than hybrids, which shrink the cache but lose some layers' access to the past. It is also possible that a recurrent layer with a large enough, cleverly enough updated state will close the recall gap for practical purposes, although the copying results of Chapter 11 set a hard limit on how far any fixed-size state can go. The relationship between advertised and usable context remains poorly understood. Models that can attend to a million tokens often perform tasks requiring genuine use of the whole input much less reliably than tasks requiring a single lookup. Whether this is a matter of training data, positional precision, the dilution of softmax over many distractors, or something more basic about how models combine many retrieved pieces, is not yet clear, and the answer will determine whether the next order of magnitude of context is worth its price. The theory of why the architecture works is also incomplete. Scaling laws are empirical regularities without an agreed derivation. The circuits identified so far are islands in models with thousands of heads. Superposition means that the basic units of analysis, features rather than neurons or heads, are only beginning to be extracted at scale. The transformer is in the unusual position of being the most important computational artefact of its decade while remaining only partly understood by the people who build it. What follows for the reader For anyone who designs, deploys or evaluates these systems, the analysis leads to a few practical habits. When a new architecture is announced, ask which cost it reduces. A design that claims linear complexity may be attacking prefill arithmetic, which matters only for very long prompts, or the key-value cache, which matters for nearly all generation. Ask also what it gives up, and specifically whether it retains the ability to retrieve an exact earlier token chosen by content. A model that does well on average perplexity can still fail badly on recall, and the failures will appear in exactly the tasks for which long context was wanted. When a model advertises a context window, treat the number as a ceiling rather than a capability. Test it on the task at hand, with the relevant information placed in the middle of the context and surrounded by plausible distractors, not merely with a single distinctive needle. Expect usable length to depend on the task. When reasoning about cost, count bytes as well as operations. A generation workload is limited by how fast weights and cache can be read from memory, and the cache scales with layers, key-value heads, head dimension, context length and the number of simultaneous users. The formula in Chapter 8 is simple enough to apply on the back of an envelope, and it explains more about the economics of long-context serving than any count of floating-point operations. When choosing between a longer context and an external retrieval system, recognise that they solve the same problem at different points. Retrieval decides in advance which material deserves attention; long context lets the model decide. The first is cheaper and scales further; the second handles cases where relevance cannot be judged before reading. Most robust systems will use both. The equation at the centre of this booklet, Attention(Q, K, V) = softmax(QK^T / sqrt(d_k)) V, has proved remarkably durable. Its form has barely changed since 2017. What has changed is everything around it: how its inputs are positioned, how many distinct keys and values it stores, how its computation is ordered on the chip, how many layers of a model use it, and what is placed into its context in the first place. That pattern is likely to continue. The question for the next generation of architectures is not whether every position should be able to consult every other, since the evidence says that ability is worth a great deal. It is how few positions need to pay for it. Glossary Arithmetic intensity. The number of arithmetic operations a computation performs for each byte it reads from memory. Low intensity means the computation is limited by memory bandwidth rather than by arithmetic speed. Attention head. One independent instance of scaled dot-product attention within a layer, with its own query, key and value projections. Causal mask. A mask that prevents each position from attending to later positions, so that a model can be trained to predict every next token in parallel without seeing the answer. Context window. The maximum number of tokens a model can take as input at once. Decode. The phase of generation in which a model produces one token per step, attending over the cached keys and values of all previous tokens. Feed-forward network. The position-wise sublayer of a transformer block, applying the same small neural network to each position independently. Grouped-query attention (GQA). A variant of multi-head attention in which groups of query heads share a single key head and value head, shrinking the key-value cache. Induction head. An attention head that, together with an earlier previous-token head, finds an earlier occurrence of the current token and copies the token that followed it. Key-value cache. The stored keys and values of all previous tokens, kept during generation so they need not be recomputed at each step. Prefill. The phase in which a model processes the whole prompt in one parallel pass before generating. Residual stream. The running sum of each position's representation to which every sublayer adds its output. Rotary position embedding (RoPE). A method of encoding position by rotating pairs of query and key components through angles proportional to position, so that attention scores depend on relative offset. Softmax. The function that turns a list of real numbers into positive weights summing to one, by exponentiating each and dividing by the total. State-space model. A sequence model whose hidden state evolves through a linear recurrence, trainable in parallel and runnable with fixed memory per step. Token. The unit of text a model processes, typically a word or part of a word. Further Reading Vaswani, A., et al. "Attention Is All You Need." NeurIPS, 2017. The original paper. Short, clearly written, and still the best statement of the comparison between attention, recurrence and convolution that motivated the design. Bahdanau, D., Cho, K., and Bengio, Y. "Neural Machine Translation by Jointly Learning to Align and Translate." ICLR, 2015. The paper that introduced attention to neural translation, and the clearest view of the problem the transformer later generalised. Su, J., et al. "RoFormer: Enhanced Transformer with Rotary Position Embedding." 2021. The derivation of rotary embeddings. Pair it with Chen et al., "Extending Context Window of Large Language Models via Positional Interpolation" (2023), and Peng et al., "YaRN" (2023), for the context-extension methods built on it. Kaplan, J., et al. "Scaling Laws for Neural Language Models." 2020; and Hoffmann, J., et al. "Training Compute-Optimal Large Language Models." 2022. The two empirical studies that shaped how large models are sized. Reading them together shows how a methodological detail changed the conclusion. Elhage, N., et al. "A Mathematical Framework for Transformer Circuits." Anthropic, 2021; and Olsson, C., et al. "In-context Learning and Induction Heads." Anthropic, 2022. The foundations of mechanistic interpretability for transformers, including the residual-stream view and the induction-head mechanism. Dao, T., et al. "FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness." NeurIPS, 2022. The clearest demonstration that attention's practical cost was a matter of memory traffic, and how to remove it without approximation. Ainslie, J., et al. "GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints." EMNLP, 2023; and Kwon, W., et al. "Efficient Memory Management for Large Language Model Serving with PagedAttention." SOSP, 2023. Two papers on the key-value cache, one from the architecture side and one from the systems side. Tay, Y., Dehghani, M., Bahri, D., and Metzler, D. "Efficient Transformers: A Survey." ACM Computing Surveys, 2022. A taxonomy of the sparse, low-rank and kernel-based approximations, written at the height of that research programme. Katharopoulos, A., et al. "Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention." ICML, 2020. The bridge between attention and recurrence, and the starting point for the gated linear models that followed. Gu, A., and Dao, T. "Mamba: Linear-Time Sequence Modeling with Selective State Spaces." 2023; and Dao, T., and Gu, A. "Transformers are SSMs." ICML, 2024. The leading attention-free architecture and the theory connecting it to linear attention. Waleffe, R., et al. "An Empirical Study of Mamba-based Language Models." NVIDIA, 2024. A controlled comparison of transformers, state-space models and hybrids at matched scale, and the clearest evidence for the hybrid design. Liu, N. F., et al. "Lost in the Middle: How Language Models Use Long Contexts." TACL, 2024; and Hsieh, C.-P., et al. "RULER: What's the Real Context Size of Your Long-Context Language Models?" 2024. Two careful evaluations showing the gap between advertised and usable context. Phuong, M., and Hutter, M. "Formal Algorithms for Transformers." DeepMind, 2022. Precise pseudocode for every major transformer variant, useful for readers who want to check the mathematics against a single consistent notation. Hashtags: #TransformerRevolution #TransformerArchitecture #AttentionMechanisms #SelfAttention #ScaledDotProductAttention #MultiHeadAttention #ArtificialIntelligence #DeepLearning #MachineLearning #NaturalLanguageProcessing #LargeLanguageModels #NeuralNetworks #AttentionIsAllYouNeed #SequenceModeling #PositionalEncoding #RotaryPositionEmbeddings #RoPE #EncoderDecoderArchitecture #DecoderOnlyTransformers #BERT #GPT #GenerativeAI #FlashAttention #KeyValueCache #LongContextModels #MixtureOfExperts #ScalingLaws #InContextLearning #StateSpaceModels #AIArchitectureEvolution

Download the Book (PDF): Introduction On the morning of 17 June 2016, a contract called The DAO began giving its money away. It held roughly a seventh of all the ether then in existence, raised from thousands of contributors who believed they were funding a new kind of investment fund governed by code rather than by managers. Nobody had broken into a server. Nobody had stolen a password. Someone had simply read the contract more carefully than its authors had, noticed that it sent ether to a caller before it updated that caller's balance, and arranged for the caller to ask again while the first payment was still in flight. The contract did exactly what it had been written to do. That was the problem. Almost everything that matters about smart contract security is already present in that incident. The code was public, so the attacker could study it at leisure. The code was immutable, so once the flaw was found there was no quiet patch. The code held real money directly, with no bank, clearing house or fraud department standing between a bug and a loss. And the flaw itself was not exotic. It was an ordering mistake of a kind that working programmers make every week in ordinary software and fix in the next release, where the consequence is a glitch rather than a catastrophe. Ten years later the stakes are larger and the mistakes are, in many cases, the same. Decentralised finance protocols hold tens of billions of dollars in contracts that anyone can call. Bridges move value between chains on the strength of a few signatures. The biggest single theft in the industry's history, the loss of roughly 1.5 billion dollars from the exchange Bybit in February 2025, did not even require a bug in a smart contract; it required persuading the people who controlled one to approve a transaction they could not properly read. Each year brings new incident reports, and each year a large share of them describe vulnerability classes that were documented, named and catalogued long before the affected code was written. The argument of this book This book is written for developers who write Solidity or Vyper and for the auditors who review their work. Its central claim is simple to state and harder to practise: smart contract vulnerabilities are, almost without exception, violated invariants, and auditing is the discipline of stating those invariants explicitly and then attacking them from every direction the execution environment allows. An invariant is a property that must hold whenever the contract is at rest. The total of all user balances equals the token's recorded supply. A vault never owes more than it holds. Only the owner can change the fee. A loan is never left undercollateralised at the end of a transaction. These are not technicalities; they are the business promises the contract makes, written as logic. Reentrancy is what happens when an external call lets someone observe or act on the contract while one of those promises is temporarily broken. Integer overflow is what happens when arithmetic silently produces a number that no promise allowed for. Front-running is what happens when the promise holds for each transaction alone but not for the order in which transactions happen to arrive. An access control failure is a promise about who may act that the code forgot to enforce. Seen this way, the long lists of vulnerability types that circulate in the industry stop being a checklist to memorise and become a set of recurring ways in which a precise statement about state can be falsified. That shift matters, because the next serious exploit will probably not appear on any list. Auditors who only pattern-match against known bugs miss it. Auditors who know what the contract is supposed to guarantee, and who understand the machine well enough to imagine how the guarantee might fail, have a chance of catching it. The second half of the argument concerns method. No single technique finds every class of flaw. Careful manual reading catches design errors that no tool understands. Static analysers catch known patterns cheaply and consistently. Fuzzers find the strange input sequences that humans never think to try. Formal verification can prove that a property holds for every possible input and every possible sequence of calls, but only for the property you wrote down, and only for the model of the system you gave it. Each technique is strongest exactly where the others are weakest, and the most mature security teams layer them deliberately, with the invariants serving as the common language that connects them. What this book covers and what it does not The first chapter explains why the Ethereum execution environment is unusually hostile to software: public state, public pending transactions, atomic execution, cheap flash-loaned capital and composability with code you did not write. The next four chapters take the major vulnerability families in turn. Reentrancy comes first, because it is the oldest and most instructive, and because its modern variants still catch experienced teams. Arithmetic follows, from the classic overflow bugs that Solidity 0.8 largely closed to the rounding and precision errors it did not. Access control and initialisation come next, covering the proxy and upgrade patterns that now dominate production code and the key-management failures behind several of the largest thefts. Then ordering: front-running, sandwich attacks, the economics of maximal extractable value, and the oracle manipulation that flash loans made cheap. The last three chapters turn from what goes wrong to how you find it. One describes the working method of an audit, from scoping and threat modelling through manual review to the writing of findings that developers can act on. One surveys the automated tools, static analysis, fuzzing and symbolic execution, and explains what each is actually good for. The final chapter addresses formal verification directly: what it means to prove a property of a contract, what such a proof does and does not guarantee, and how teams use it in practice without mistaking it for a certificate of safety. Historical incidents appear throughout, described at the level of mechanism. The point of studying them is to understand why an assumption failed, not to rehearse how to repeat the attack. Code examples are short, written in Solidity, and show vulnerable patterns alongside their fixes in the form a developer would recognise from a code review. Nothing here is a working exploit against a deployed system, and nothing needs to be: the defensive value lies in the reasoning, not the payload. Some things are deliberately left out. This is not an introduction to Solidity; it assumes you can read a contract and know what a modifier, a mapping and an external call are. It does not cover consensus-level attacks on blockchains themselves, wallet security for individual users, or the legal and regulatory consequences of a hack, all of which deserve treatment elsewhere. It concentrates on the Ethereum Virtual Machine because that is where most value and most audited code still live, though the reasoning carries over directly to other EVM-compatible chains and, with adjustments, to platforms such as Solana and the Move-based chains. A note on attitude Security writing has a tendency to swing between alarm and swagger. Neither helps. The useful attitude is closer to that of an engineer reviewing a bridge design: calm, sceptical, specific and slightly pessimistic. Assume the code is wrong until you can explain why it is right. Assume every external call is made to someone who wants your money. Assume that a value you did not validate will, eventually, be the worst possible value. And assume that the people who wrote the code were competent and careful, because they usually were, and the bug got through anyway. The DAO's authors were not fools. Neither were the developers of the Parity multisig library, the engineers at Euler Finance, or the maintainers of the Vyper compiler, all of whom appear in the pages that follow. What they shared was not incompetence but an unexamined assumption, a promise their code made that nobody had written down clearly enough to test. The craft of auditing is the craft of finding those promises and asking, with real persistence, whether they hold. Chapter 1: The Adversarial Machine Most programmers learn their trade in environments that quietly protect them. A web application runs on a server its owners control; its source code is private; its users reach it through interfaces that limit what they can send; and when a bug is found, a patch can be deployed before lunch. Fraudulent transactions can be reversed by a bank. Abuse can be rate-limited, logged and traced to an account. Even a serious vulnerability usually has to be found, weaponised and exploited before anyone notices, and the window for that is often short. A smart contract enjoys none of these protections. It is better understood as a vending machine bolted to a public pavement, stocked with cash, with its wiring diagram posted on the front, and with no owner on call. Anyone in the world can put any coin into any slot, in any order, as many times as they like within a single transaction. The machine cannot tell an honest customer from a professional thief, because on a blockchain there is no such distinction to draw: there are only addresses and calls. Everything in this book follows from taking that picture seriously. Five properties that change the threat model The first property is public state and public code. Every storage slot of every contract on Ethereum can be read by anyone running a node. Marking a variable private in Solidity only prevents other contracts from reading it through the language; it does nothing to hide the value from an observer who reads the chain directly. Contract bytecode is public by construction, and most serious projects publish verified source on block explorers so that users can trust what they are interacting with. The consequence is that an attacker has unlimited time to study the target, to deploy a copy on a private fork of mainnet, and to rehearse an attack against the real state until it works perfectly. Security through obscurity is not merely weak here; it is unavailable. The second property is immutability. Once deployed, contract bytecode cannot be changed. A protocol may build in upgradeability by routing calls through a proxy that delegates to a replaceable implementation, and many do, but that introduces its own risks, which Chapter 4 examines. Without such a mechanism, a flaw discovered after deployment can only be mitigated by pausing the contract, if a pause was designed in, or by persuading users to move their funds elsewhere before an attacker moves them first. The developer's normal feedback loop of ship, observe and fix is replaced by a single irreversible release. The third property is atomicity. An Ethereum transaction either completes entirely or reverts entirely. Within it, a caller can invoke any number of contracts, in any sequence, passing value and data between them, and if any step fails the whole thing unwinds as if it never happened. For users this is a feature: a swap that does not get the expected price can simply revert. For attackers it is a gift. They can construct elaborate multi-step operations, try them, and lose nothing but a little gas if they fail. They can also make their attack conditional on its own success, checking at the end that they profited and reverting otherwise. The fourth property is composability. Contracts call other contracts freely, and protocols are routinely built on top of one another. A lending market accepts a liquidity pool's share token as collateral, values it using a price feed derived from another pool, and lets users borrow a stablecoin issued by a fourth protocol. This is what makes decentralised finance powerful, and it is also why the security of any one contract depends on the behaviour of code its authors did not write, did not audit and cannot control. Every external call is a point at which control leaves your code and passes to someone else's, who may be hostile. The fifth property, which ties the others together, is cheap capital. In conventional finance, manipulating a market requires money, and money leaves a trail. On Ethereum, flash loans let anyone borrow enormous sums with no collateral, on the sole condition that the loan is repaid before the transaction ends. Because of atomicity, the lender bears no credit risk: if the borrower does not repay, the whole transaction, loan included, never happened. Flash loans became widely available in 2020, and within weeks they were used in a pair of attacks on the bZx lending protocol that combined borrowed funds, price manipulation on a thinly traded market and a leveraged position into a profit for the attacker. The lesson was immediate and lasting. Any protocol whose safety depended on an attacker being unable to command large amounts of capital for a few seconds was no longer safe. The anatomy of an external call Because so many vulnerabilities centre on the moment control leaves a contract, it is worth being precise about what happens at that moment. When a contract calls another, the Ethereum Virtual Machine pauses the caller's execution, creates a new execution frame for the callee and runs the callee's code. The callee can do anything a contract can do: read state, write its own storage, transfer value and, crucially, call back into any contract, including the one that called it. Only when the callee returns does the caller resume, with whatever return data the callee chose to provide. During that entire interval, the caller's storage reflects whatever the caller had written before making the call, and nothing it intended to write afterwards. Several mechanisms trigger external calls, and not all of them look like calls in the source. Sending ether to a contract address runs that contract's receive or fallback function. Transferring certain token standards, notably ERC-777 and the safe-transfer variants of ERC-721 and ERC-1155, invokes hooks on the recipient. Calling an interface method on an address supplied by a user runs whatever code lives at that address, which may bear no relation to the interface it claims to implement. Even a seemingly passive read through a view function executes foreign code, and that code can observe your contract's state at a moment you did not intend it to be observed. There is also delegatecall, which runs another contract's code in the caller's own storage context. It is the foundation of proxy-based upgradeability and of shared libraries, and it is as dangerous as it sounds: the delegated code can overwrite any storage slot of the caller, change its owner, or destroy it. Two of the most expensive incidents in the industry's history, the Parity multisig failures of 2017 and the Bybit theft of 2025, turned on what code was allowed to run through a delegatecall and in whose context. Gas is the last piece of the anatomy. Every operation costs gas, and each transaction carries a limit. Calls can forward all remaining gas or a fixed amount. For a period, developers relied on the fixed 2,300-gas stipend that Solidity's transfer and send forward as a defence against reentrancy, on the theory that the recipient could not do anything harmful with so little. That assumption broke when network upgrades repriced certain operations, and it is now considered unsafe to depend on specific gas costs for security. The general principle is worth stating: any security property that relies on the exact cost of an operation is relying on something the protocol's designers are free to change. Who the adversary is It is tempting to imagine the attacker as a lone genius. In practice, the adversaries a contract faces range across several distinct types, and a threat model should account for each. There are opportunistic attackers who run automated scanners across newly deployed contracts looking for known patterns: unprotected initialisers, public functions that should have been restricted, tokens with broken arithmetic. These scanners are fast and indiscriminate, and contracts with elementary flaws are sometimes drained within minutes of deployment. There are professional searchers and block builders, who are not attackers in the criminal sense but who extract value from transaction ordering as a business. They watch the public pool of pending transactions, simulate what each would do, and insert their own transactions around profitable ones. A contract that leaks value through ordering will be exploited by these actors as a matter of routine, legally and continuously. Chapter 5 deals with them. There are sophisticated, well-resourced teams who study a single high-value target for weeks. These are the attackers behind most of the largest incidents. Public investigations, including statements from the United States Federal Bureau of Investigation, have attributed several of the biggest thefts, among them the Ronin bridge compromise of 2022 and the Bybit theft of 2025, to groups linked to the North Korean state. Such adversaries combine smart contract analysis with social engineering, malware and supply-chain compromise. Defending against them requires attention to the people and infrastructure around a contract, not only to the contract itself. There are generalised front-running bots, which copy any pending transaction that would be profitable for them to send instead, substituting their own address. These bots mean that even a well-intentioned attempt to rescue funds from a vulnerable contract can be intercepted, a phenomenon several white-hat rescuers have described from bitter experience. Finally, there are the protocol's own privileged users: administrators, governance token holders, multisig signers and oracle operators. A threat model that treats them as fully trusted is making an assumption, and that assumption should be written down and examined, because their keys can be stolen, their votes can be bought, and their incentives can change. What changes about EOAs and contracts One assumption that a generation of contracts relied on deserves special mention because it has recently and formally ceased to hold. For most of Ethereum's history, there were two kinds of account: externally owned accounts, controlled by a private key and holding no code, and contract accounts, holding code and no key. Many contracts used this distinction for security, checking that msg.sender == tx.origin to ensure that the caller was a human-controlled account rather than a contract, or checking that the caller's code size was zero. Both checks were always fragile. A contract's code size is zero during its own constructor, so a contract under construction could pass the second check. The first check blocked legitimate smart contract wallets and was frowned upon for that reason. With the Pectra network upgrade in May 2025, EIP-7702 allowed externally owned accounts to delegate to contract code, so that an address controlled by a private key can now also execute arbitrary logic when called. A contract that assumes an EOA cannot reenter it, or that tx.origin guarantees a code-free caller, is now simply wrong. The broader lesson is that the execution environment is not fixed. Gas costs change, opcodes are added, account models evolve, and new standards introduce new callback hooks. An audit is a statement about code against a particular environment at a particular time. Assumptions about that environment belong in the threat model explicitly, so that when the environment changes, someone knows which assumptions to revisit. Why a small bug becomes a total loss In conventional software, the damage a bug can do is usually limited by everything around it. A flaw in a banking application might let one user see another's statement, but moving money still passes through settlement systems, fraud checks, daily limits and human review, any of which can stop or reverse a bad transfer. In a smart contract, the code is the settlement system. There is no second layer that looks at a withdrawal and asks whether it seems reasonable. If the contract's logic permits a transfer, the transfer happens and becomes final within seconds. Three features of the environment turn this into a pattern of total rather than partial loss. The first is that attacks are repeatable at no marginal cost. A flaw that lets an attacker take one unit of value can usually be exercised in a loop, or across many transactions in rapid succession, until the contract is empty. The second is that the funds at risk are typically pooled. A lending market or an exchange pool holds the deposits of thousands of users in a single contract, so a flaw in that contract exposes all of them at once. The third is speed. Attackers prepare in private, often on a local fork, and execute in a single transaction or a short burst of them. By the time monitoring systems raise an alarm, the money has usually left. Defenders have developed responses, and a serious threat model accounts for them. Many protocols build in a pause function, controlled by a guardian address or a multisig, which halts sensitive operations when something looks wrong. Pausing can limit damage if it is fast enough, but it has costs: a pause is itself a privileged power that could be abused, it may trap honest users' funds at a bad moment, and an attacker who knows the guardian's habits can time an attack for when they are least likely to respond. Rate limits on withdrawals, caps on how much can leave a bridge in a period, and delays on large transfers all trade convenience for a chance to intervene. Each of these is a design decision that has to be made before deployment, because there is no adding a pause to an immutable contract after the fact. When an incident does happen, the response has become more organised than it once was. Volunteer groups such as the Security Alliance coordinate emergency help for protocols under attack, and a number of protocols have adopted explicit safe-harbour terms under which white-hat hackers may move funds out of a vulnerable contract to protect them, returning them afterwards for an agreed bounty. These arrangements help, but they all operate after the flaw is live. The only response that reliably prevents a loss is finding the flaw before deployment. Not every EVM is the same EVM A final point about the environment concerns its variety. Much smart contract code is now deployed not only on Ethereum's main network but on many other chains and layer-two networks that run compatible virtual machines. Compatibility is not identity. Some networks lag behind Ethereum in adopting new opcodes, so a contract compiled for a recent target may include instructions that another chain does not support. Block timestamps and block numbers can behave differently on rollups, where blocks are produced by a sequencer at intervals that do not match Ethereum's. Some networks have a single sequencer that orders transactions, which changes the threat model for front-running described in Chapter 5, and which introduces new failure modes if the sequencer goes offline while prices move. Gas costs, precompiled contracts and the semantics of a few opcodes vary. Contracts that will be deployed across several networks need to be reviewed for each of them, not once for an idealised EVM. The audit should name the target chains, the compiler target version and any assumptions about timing, ordering or opcode availability. Several protocols have discovered, after deploying identical code on a new chain, that an assumption which held on Ethereum quietly failed elsewhere. Invariants as the unit of security If the environment is this hostile, what can a developer actually rely on? The answer, developed throughout the rest of this book, is the contract's own invariants. An invariant is a property of the contract's state that should be true whenever no transaction is mid-execution. Some are accounting identities: the sum of all balances in a token equals its total supply; the assets a vault holds are at least what it owes depositors. Some are about authority: only the governance contract can change the interest rate model. Some are about relationships between contracts: the price a lending market uses for an asset is within a tolerance of the price observed on a reference market. Some are temporal: a withdrawal request, once made, can be completed after the delay, and cannot be completed before it. Thinking in invariants changes how one reads code. Instead of asking whether a function looks correct, one asks which invariants it might temporarily break during execution, whether any external call happens while they are broken, and whether there is any path, any sequence of calls by any actor in any order, that ends with an invariant false. Each vulnerability class covered in the next four chapters can be understood as a particular way of answering that last question with yes. Reentrancy lets an attacker act while an invariant is broken. Arithmetic errors let a calculation produce a state no invariant allows. Access control failures let the wrong actor take a step that changes the rules. Ordering attacks exploit invariants that hold per transaction but fail across them, or that were never stated at all because the designers did not think of ordering as part of the contract's behaviour. The same framing is what makes testing and formal verification tractable. A fuzzer needs to know what to check after each random sequence of calls; the answer is the invariants. A formal verification tool needs a specification to prove; the specification is the invariants. An auditor's report is, at its best, a list of invariants that were believed to hold and a demonstration of the ones that do not. The machine is adversarial and indifferent. The invariants are the only part of the system that says what it is supposed to do. Chapter 2: Reentrancy, From The DAO to Read-Only Calls Reentrancy is the oldest named vulnerability in smart contract security and still one of the most instructive. It is also widely misunderstood. Many developers believe it is a solved problem: add a reentrancy guard, follow the checks-effects-interactions pattern, and move on. Those defences are sound and should be used. But the bug class they address is narrower than reentrancy itself, and the incidents of the last several years have mostly come from the parts of the class that the standard defences do not cover. To understand why, it helps to begin with the original case and then follow the idea outwards. The classic pattern Reentrancy occurs when a contract makes an external call while its own state is inconsistent, and the callee uses that window to call back into the contract and act on the inconsistent state. The canonical form is a withdrawal function that pays first and records the payment second. // Vulnerable: interaction before effect function withdraw() external { uint256 amount = balances[msg.sender]; require(amount > 0, "nothing to withdraw"); (bool ok, ) = msg.sender.call{value: amount}(""); require(ok, "transfer failed"); balances[msg.sender] = 0; } When the contract sends ether to msg.sender, and msg.sender is a contract, the recipient's receive function runs. At that moment balances[msg.sender] still holds the full amount, because the line that zeroes it has not yet executed. If the recipient calls withdraw again, the check passes a second time, a second payment goes out, and the process can repeat until the contract is empty or the call stack runs out of gas. Only when the innermost call returns does each frame proceed to set a balance that is already zero. The invariant at stake is plain: the contract's ether holdings should always cover the sum of recorded balances. The withdrawal function breaks that invariant temporarily, by sending ether before reducing the balance, and makes an external call during the break. That combination, a broken invariant and a transfer of control, is the whole of reentrancy. Everything else is variation. The DAO, in June 2016, fell to a version of this pattern in its splitDAO function, which allowed a member to withdraw their share of ether into a child organisation. The function transferred funds before updating the member's token balance, and the attacker's contract re-entered it repeatedly. Roughly 3.6 million ether moved into a child DAO controlled by the attacker. The Ethereum community's response was a contentious hard fork that restored the funds, and the minority who rejected it continued the original chain as Ethereum Classic. It remains the only time Ethereum's history was rewritten to reverse a contract exploit, and the political cost of doing so is one reason it will almost certainly not happen again. Developers today should assume that a drained contract stays drained. The two standard defences The first defence is ordering. The checks-effects-interactions pattern says a function should validate its inputs and preconditions first, then update its own state, and only then interact with other contracts. Applied to the example: // Fixed: effect before interaction function withdraw() external { uint256 amount = balances[msg.sender]; require(amount > 0, "nothing to withdraw"); balances[msg.sender] = 0; (bool ok, ) = msg.sender.call{value: amount}(""); require(ok, "transfer failed"); } Now a re-entrant call finds the balance already zeroed and fails the check. The invariant is restored before control leaves the contract, so there is no inconsistent state for the callee to exploit. This is the strongest defence because it removes the underlying condition rather than blocking one route to it. The second defence is a reentrancy guard, a mutex that marks the contract as busy when a protected function begins and clear when it ends, reverting any attempt to enter a protected function while the flag is set. OpenZeppelin's ReentrancyGuard and its nonReentrant modifier are the most widely used implementation. Since the Cancun upgrade in March 2024 introduced transient storage under EIP-1153, guards can store their flag in a slot that is automatically cleared at the end of each transaction, which makes them considerably cheaper; OpenZeppelin ships a transient variant for this purpose. Guards are valuable precisely because ordering is easy to get wrong in complex functions, where several state updates and several external calls interleave. But a guard only protects the functions it is applied to, and only within the contract that holds the lock. That limitation is where the more interesting variants live. A third practice reduces exposure rather than blocking reentry: pull over push. Instead of sending funds to recipients as part of a larger operation, record what each recipient is owed and let them withdraw it in a separate transaction. This isolates the external call into a small, easily reviewed function and stops a malicious recipient from interfering with operations that involve other users. Beyond the single function The variants that matter in modern code differ in where the re-entrant call lands and what it does there. Table 1 summarises them, and the sections that follow explain each. Table 1. Principal forms of reentrancy and their defences. Variant Re-entered target What the attacker exploits Primary defence Single-function Same function Stale balance or flag Checks-effects-interactions Cross-function Another function, same contract State shared between functions Contract-wide guard, ordering Cross-contract A different contract sharing state Guard held by only one contract Shared lock or consistent state before calls Read-only A view function Inconsistent value read by a third party Guard on views, check lock before reading Callback-hook Function reached through token hook Hidden external call in transfer Treat token transfers as external calls Cross-function reentrancy occurs when the attacker re-enters not the function that made the call but a different function that reads the same state. Suppose withdraw follows a sound order for its own logic but a separate transfer function lets a user move their internal balance to another address. If withdraw sends ether before zeroing the balance, the attacker's fallback can call transfer to move the still-recorded balance to a second address they control, then let withdraw finish. Placing nonReentrant on withdraw alone does not help; the guard must cover every function that touches the shared state, which in practice means applying it consistently across the contract's state-changing entry points. Cross-contract reentrancy extends the same idea across contract boundaries. Modern protocols are often split into several contracts that share state: a vault and a separate accounting module, a pool and a router, a lending market and its interest-rate controller. A guard in one contract does not lock the others. If the vault makes an external call while its accounting is inconsistent, and the attacker re-enters through the router or controller, the vault's lock is irrelevant. Defending against this requires either a lock shared across the system or, better, an architecture in which no contract makes an external call while any shared state is inconsistent. Callback-hook reentrancy is the variant that most often surprises developers, because the external call is not visible in the code they are reading. Token standards that notify recipients, including ERC-777's tokensReceived hook and the onERC721Received and onERC1155Received callbacks invoked by safe transfers, mean that a call to transfer or safeTransferFrom can execute arbitrary code at the recipient. In April 2020, the lending protocol Lendf.me lost around 25 million dollars when an attacker used the ERC-777 hooks of the imBTC token to re-enter its supply function during a deposit, inflating their recorded collateral. A related attack on Uniswap's first version drained imBTC liquidity the same week. In August 2021, Cream Finance lost a comparable sum through the AMP token, whose transfer logic also invoked recipient hooks. In each case, the protocol had reasoned about the token as if moving it were a passive bookkeeping operation. The defence is a change in habit: every token transfer is an external call unless you have verified otherwise, and should be placed and guarded accordingly. Read-only reentrancy The most subtle variant does not change the vulnerable contract's state at all. Instead, it exploits the fact that other protocols read that contract's state, and that during an external call those reads return values that should never be observable. Consider a liquidity pool that exposes a view function reporting the value of one pool share, computed from its reserves and its total supply of shares. When a user removes liquidity, the pool burns their shares and sends them the underlying assets. If, in the middle of that operation, the pool has reduced one quantity but not yet the other and makes an external call, perhaps because one of the assets is native ether and sending it runs the recipient's code, then anyone reading the share price at that instant sees a distorted value. The pool itself is not harmed: its own functions are guarded, and the inconsistency is resolved before the transaction ends. But a lending market that accepts pool shares as collateral, and values them by calling that view function, can be tricked into thinking the collateral is worth far more or far less than it is. This pattern affected integrations with several Curve Finance pools, whose virtual price function could be read during a removal of liquidity involving ether. Curve's own state was never at risk; the vulnerable parties were protocols that consumed the value. A number of lending and yield protocols were exploited this way, including dForce in February 2023. The vulnerability is a clear demonstration of the composability problem described in Chapter 1: the flaw lives in the gap between two systems, each of which was arguably correct on its own terms. There are two defences. A protocol that exposes values for others to consume can apply its reentrancy lock to view functions as well, reverting if they are called while the lock is held, or provide a separate function that does so. A protocol that consumes values from another can check, before reading, whether the source is in a locked state, typically by attempting a cheap guarded operation that would revert if a removal were in progress. Both approaches require the parties to understand each other's locking conventions, which is one more reason integration code deserves as much scrutiny as core logic. A worked review: minting with callbacks It is useful to see how these ideas apply to an ordinary-looking function, because reentrancy is rarely as visible in production code as it is in the textbook withdrawal example. Consider a simplified NFT sale contract that limits each address to a maximum number of mints and the collection to a fixed supply. // Vulnerable: safe mint calls the recipient before counters update function mint(uint256 quantity) external payable { require(minted[msg.sender] + quantity <= MAX_PER_WALLET, "limit"); require(totalMinted + quantity <= MAX_SUPPLY, "sold out"); require(msg.value == price * quantity, "wrong payment"); for (uint256 i = 0; i < quantity; i++) { _safeMint(msg.sender, totalMinted + i); } minted[msg.sender] += quantity; totalMinted += quantity; } Nothing in this function says "external call", and a reviewer scanning for call or transfer would pass over it. But _safeMint in the standard ERC-721 implementation checks whether the recipient is a contract and, if so, calls its onERC721Received function to confirm that it can handle the token. That call hands control to the recipient while minted[msg.sender] and totalMinted still hold their old values. A recipient contract could call mint again from inside its callback, pass both limit checks a second time because the counters have not moved, and acquire more tokens than the per-wallet limit allows. Depending on how token identifiers are calculated, the repeated calls might also try to mint identifiers that collide, causing a revert, or might succeed in ways that break the supply accounting. The invariant here is straightforward: the number of tokens held by any address through this sale never exceeds the per-wallet limit, and the number of tokens minted never exceeds the supply cap. The function breaks both invariants temporarily, by issuing tokens before updating the counters, and calls external code during the break. The fix follows the same principle as before. Update the counters before minting, so that a re-entrant call sees the new values, and add a reentrancy guard because the function moves value and interacts with external code. Alternatively, use the non-callback mint where the design does not require confirming that the recipient can hold the token. Each choice has trade-offs: mint can send tokens to contracts that will lock them forever, which is a usability issue rather than a security one, while _safeMint with correct ordering and a guard preserves the safety check at slightly higher cost. The general lesson is that reentrancy review must follow calls through library code and standard implementations. A developer who inherits from a well-audited library may reasonably trust the library's correctness, but must still understand which of its functions make external calls, because the library cannot know what state the inheriting contract has left inconsistent at the moment it calls out. Reentrancy and the evolving protocol There is a temporal dimension to reentrancy that audits sometimes miss. A contract may be entirely safe as deployed, because the only tokens it handles have no transfer hooks and the only addresses it pays are known, and then become vulnerable without a line of its own code changing. This can happen in several ways. Governance may vote to support a new collateral asset whose transfer function calls the recipient. An upgradeable token the protocol already supports may be upgraded by its own issuer to add hooks. A user who was once an externally owned account may, after EIP-7702, attach code to their address. An integrating protocol may begin to read a view function that was never designed to be consumed during an operation in progress. The defence against this kind of drift is to make reentrancy safety a property of the contract itself rather than of its current environment. A contract that orders its effects before its interactions and guards every state-changing entry point is safe regardless of which tokens or callers it encounters later. A contract that is safe only because its current counterparties happen to be passive has a latent vulnerability waiting for the environment to change. The asset-listing process of a lending protocol should include an explicit check of whether the new asset has any transfer hooks or callbacks, and if it does, whether every code path that handles it was designed with that in mind. When the guard itself fails In July 2023, several Curve pools written in Vyper were drained of roughly 70 million dollars in total, though a portion was later recovered through white-hat efforts and negotiation. The pools used Vyper's built-in @nonreentrant decorator, which should have prevented exactly the attack that occurred. The flaw was not in the pools but in the compiler. Specific versions of Vyper, 0.2.15, 0.2.16 and 0.3.0, contained a bug in how storage slots were allocated for reentrancy locks, so that functions intended to share one lock were each given separate slots. The guard appeared in the source, passed review and functioned within each individual function, but did not prevent cross-function reentry between functions that were supposed to be mutually exclusive. The incident carries several lessons. The first is that a security mechanism is only as good as its implementation, and the compiler is part of the implementation. Audits that review source code assume the compiler faithfully translates it; that assumption is usually justified, but not always, and high-value contracts benefit from at least some checking at the bytecode level. The second is that the bug had been fixed in later compiler versions without its security significance being recognised, so contracts compiled with the affected versions remained vulnerable long after the fix. Tracking which compiler version produced each deployed contract, and monitoring compiler security advisories, is part of operational security. The third is that the attack worked because the pools held native ether, whose transfer gives control to the recipient. Pools that held only standard tokens with no hooks were not exposed. How to audit for reentrancy An effective review for reentrancy proceeds systematically rather than by scanning for suspicious lines. Begin by listing every point at which control can leave the contract: low-level calls, ether transfers, calls to interfaces on addresses that are not hardcoded and trusted, token transfers for any token whose implementation is not fixed and known, and any callback that a standard might invoke. Then, for each such point, determine which state variables have been modified earlier in the same execution and which are still to be modified afterwards. If any invariant involving those variables is false at the moment of the call, there is a window. Next, enumerate every function, in this contract and in any contract sharing its state, that reads or writes the variables involved. Each one is a potential target for re-entry. Check whether a guard covers it, and whether that guard is the same lock as the one held at the call site. Then extend the search outward: which view functions expose values derived from the inconsistent state, and who consumes them? If your protocol consumes another protocol's views, ask the same questions about the other protocol. Finally, consider the environment. Since EIP-7702, an address that looks like an ordinary user account can run code. Tokens can be upgraded to add hooks. A function that is safe today because the only assets it handles have no callbacks may become unsafe when governance adds a new asset. A durable design does not rely on the current set of tokens or callers being benign; it ensures that state is consistent before any call, and guards every entry point that touches the state involved. Reentrancy is not an obscure bug. It is the most direct consequence of letting someone else's code run while your own promises are, for a moment, untrue. Chapter 3: Arithmetic That Lies Computers do not do arithmetic the way people do. They work with numbers of fixed width, and when a result does not fit, something has to give. In most programming contexts the consequences are bounded: a counter wraps, a graphic glitches, a test fails. In a smart contract, numbers are money, and an arithmetic error is a direct transfer of value from one party to another, usually from the protocol to whoever noticed first. The Ethereum Virtual Machine works natively with 256-bit unsigned integers. That is an enormous range, large enough that most quantities never come near its limits. Yet arithmetic flaws remain one of the most persistent sources of loss, for two reasons. The first is that attackers control inputs, and an attacker who can choose a value will choose the one that breaks the calculation. The second is that the most dangerous arithmetic errors today are not overflows at all but the quieter failures of precision and rounding, which no compiler check can catch because they are, from the machine's point of view, correct. Overflow and underflow An unsigned 256-bit integer can hold values from zero to two raised to the 256th power, minus one. In the EVM, arithmetic on these values wraps around modulo two to the 256th. Adding one to the maximum value yields zero. Subtracting one from zero yields the maximum value. Multiplying two large numbers yields the low 256 bits of the true product, which can be any value at all. Before December 2020, Solidity performed arithmetic with exactly these wrapping semantics and no warning. A contract that subtracted a withdrawal amount from a balance without first checking that the balance was large enough could underflow, leaving the user with an astronomically large balance. Developers defended against this with libraries, of which OpenZeppelin's SafeMath was the most widely used, that wrapped every operation in a check and reverted on overflow. The burden was on the developer to use the library consistently, and the record shows that they did not always do so. The best-known failure is the BeautyChain token, known by its ticker BEC, in April 2018. Its contract included a batchTransfer function that let a sender move the same amount to several recipients at once. It computed the total to be debited by multiplying the number of recipients by the per-recipient amount, then checked that the sender's balance covered the total. The multiplication used raw arithmetic. An attacker supplied two recipients and a per-recipient value of exactly half of two to the 256th power, so that the product wrapped around to zero. The balance check passed, since any balance covers zero, and each recipient was then credited with an amount far beyond the token's total supply. Several exchanges suspended deposits of ERC-20 tokens while they checked whether other tokens shared the flaw, and a number did. // Vulnerable under Solidity < 0.8: product can wrap to zero uint256 amount = uint256(receivers.length) * value; require(balances[msg.sender] >= amount); Solidity version 0.8.0 changed the default. Arithmetic operators now check for overflow and underflow and revert if either occurs. This single change eliminated the BEC class of bug for code compiled with modern versions and is one of the most consequential security improvements in the language's history. Vyper had checked arithmetic from early on. For new code, the simple rule is to use a current compiler and let it do this work. Where the checks stop Checked arithmetic is not universal, and auditors need to know exactly where it ends. The most obvious exception is the unchecked block, which developers use to save gas in loops and in calculations they believe cannot overflow. Every unchecked block is a claim that the developer has proved the absence of overflow by reasoning, and every such claim deserves to be verified by the reviewer. Loop counters bounded by an array length are usually safe. Unchecked subtraction of a user-supplied value from a balance is a red flag unless a preceding check guarantees the result. The second exception is explicit type conversion. Solidity 0.8 checks arithmetic operators but does not check narrowing casts. Converting a uint256 to a uint128 simply discards the high bits, so a value just above the 128-bit maximum becomes a very small number. Contracts that pack values into smaller types to save storage, which is common in automated market makers and lending markets, must verify that each value fits before casting it. OpenZeppelin's SafeCast library exists for this purpose. Conversions between signed and unsigned integers carry the same risk: a negative signed value converted to unsigned becomes enormous. The third exception is bit shifts. Shifting left discards bits that move beyond the type's width, with no revert. A calculation that uses shifts for multiplication by powers of two, again common in fixed-point mathematics, can silently lose its most significant bits. The fourth exception is inline assembly. Code written in Yul, Solidity's intermediate language, operates directly on EVM words with the machine's wrapping semantics. Highly optimised libraries for fixed-point arithmetic, square roots and logarithms are often written partly in assembly, and their correctness depends on careful manual reasoning about bounds. The Cetus decentralised exchange on the Sui blockchain illustrates what happens when hand-written overflow checks are wrong. Sui uses the Move language rather than the EVM, but the lesson transfers directly. In May 2025, an attacker drained roughly 223 million dollars from Cetus pools. Post-incident analyses by several security firms traced the root cause to a helper function in a shared mathematics library that was supposed to detect whether a left shift would overflow. The check compared the input against the wrong threshold, so certain values that would overflow passed it. The attacker used this to open a liquidity position that the protocol believed required only a tiny deposit while crediting them with an enormous amount of liquidity. Much of the stolen value was frozen by Sui's validators, a response that itself raised difficult questions about decentralisation. The arithmetic lesson is straightforward. A custom overflow check is a piece of security-critical code, and it needs the same scrutiny, the same tests at its boundaries and ideally the same formal reasoning, as anything else that guards funds. Precision, rounding and the order of operations Solidity has no native floating-point arithmetic. Fractional quantities are represented as integers scaled by a fixed factor, most commonly ten to the eighteenth power, matching the eighteen decimal places of ether and most tokens. Prices, interest rates, exchange rates and share values are all computed in this fixed-point form, and every division truncates towards zero. Truncation creates two classes of problem. The first is loss of precision through the order of operations. Integer division discards the remainder, so dividing before multiplying throws information away that cannot be recovered. Computing a fee as the amount divided by ten thousand, multiplied by the fee rate in basis points, gives zero for any amount below ten thousand units, whereas multiplying first and dividing second gives the correct result. In isolation, such losses are small. Repeated across many operations, or exploited deliberately with carefully chosen amounts, they accumulate into real money. The second, more subtle class concerns the direction of rounding. When a protocol calculates how many shares to give a depositor, or how many assets to give a withdrawer, the result is rarely an exact integer, and the protocol must choose whether to round up or down. The safe rule is that rounding should always favour the protocol over the user: round down the shares a depositor receives, round up the shares a withdrawer must burn, round down the assets paid out, round up the assets required. If any calculation rounds in the user's favour, an attacker can repeat the operation many times, each time extracting a fraction of a unit, and if the operation is cheap enough or the amounts small enough relative to the rounding error, the extraction becomes profitable. The Balancer V2 incident in November 2025, which cost roughly 128 million dollars across several chains, turned on exactly this kind of problem. Public analyses from security firms including Trail of Bits and OpenZeppelin described a rounding inconsistency in the scaling calculations of Balancer's composable stable pools. The attacker used batched swaps to push a pool into a state where the balances involved were tiny, the regime in which rounding errors are proportionally largest, and then exploited the inconsistency to distort the pool's internal invariant and the price of its shares. The code had been audited multiple times and had operated for years. What the audits had not established was that the pool's central mathematical invariant held under every combination of rounding directions at extreme values. The empty vault problem A widely studied instance of rounding risk is the first-depositor or inflation attack on tokenised vaults, particularly those following the ERC-4626 standard. Such a vault issues shares to depositors in proportion to the assets they contribute. The number of shares minted for a deposit is the deposit multiplied by the existing total shares, divided by the existing total assets. When the vault is empty or nearly so, this formula becomes fragile. An attacker who deposits first can acquire a tiny number of shares, then transfer a large amount of the underlying asset directly to the vault without going through the deposit function. This donation inflates the total assets without increasing the total shares, so each share is now worth a great deal. When an honest user then deposits, the share calculation rounds down, possibly to zero, and their assets are absorbed into the vault where the attacker's shares claim them. Several mitigations exist, and Table 2 compares them. The approach adopted in OpenZeppelin's ERC-4626 implementation adds virtual shares and virtual assets to the calculation, as though the vault always contained a small, fixed endowment that no one can withdraw. With a sufficiently large offset, the attack becomes unprofitable because the donation needed to inflate the share price is lost largely to the virtual holder. Other protocols mint a small number of dead shares to an unrecoverable address at deployment, or require a minimum initial deposit made by the deployer, or track assets internally rather than reading the contract's token balance, so that donations do not affect the share price at all. Table 2. Mitigations for the vault inflation attack. Mitigation How it works Cost or drawback Virtual shares and assets Offset added to share formula Slight rounding loss for all users Dead shares at deployment Initial shares burned to null address Deployer must fund and remember it Minimum first deposit Deployer seeds vault before opening Operational step that can be skipped Internal asset accounting Ignore direct token transfers Donated tokens are stranded Slippage check on deposit User specifies minimum shares Protects user, not the protocol The general principle reaches beyond vaults. Any formula that divides by a quantity an attacker can make very small, or that reads a balance an attacker can inflate by direct transfer, deserves particular attention. The edge cases are not exotic corners of the input space. They are exactly where an adversary will go. Tokens that do not add up A further class of arithmetic error arises not from the protocol's own calculations but from the assumption that the tokens it handles behave like idealised numbers. The ERC-20 standard specifies an interface, not a behaviour, and real tokens deviate from the simple model in ways that break accounting. Decimals are the first trap. Most tokens use eighteen decimal places, but the most widely used dollar stablecoins, USDC and USDT, use six, and some tokens use other values. A protocol that combines amounts of different tokens, as any lending market or multi-asset pool must, has to normalise them to a common scale before adding, comparing or pricing them. A contract that silently assumes eighteen decimals will treat one dollar of a six-decimal stablecoin as a trillionth of a dollar, or, if the error runs the other way, as a trillion dollars. Such errors are easy to make in code that was first written for one token and later generalised. Fee-on-transfer tokens are the second trap. Some tokens deduct a fee from every transfer, so that a transfer of one hundred units delivers ninety-eight to the recipient. A protocol that records a deposit of one hundred, on the strength of the amount it asked to receive, has credited the depositor with two units that do not exist. Over many deposits, the shortfall grows until the last users to withdraw find nothing left. The defence is to measure the contract's balance before and after each incoming transfer and credit only the difference, or to refuse to support such tokens at all and say so clearly. Rebasing tokens are the third. Their balances change without any transfer, because the token's contract periodically adjusts every holder's balance to reflect interest, rewards or a peg mechanism. A protocol that stores a fixed balance for each depositor will drift out of step with the token's own accounting, either leaving rewards stranded in the contract or, if the rebase is negative, owing more than it holds. Wrapping such tokens in a non-rebasing form before accepting them is the usual solution. Other deviations are less arithmetic but equally damaging: tokens that return false on failure instead of reverting, so that an unchecked transfer appears to succeed; tokens that do not return a value at all, breaking contracts that expect one; tokens with blocklists that can freeze a protocol's own address; tokens whose owners can mint without limit or pause transfers. Libraries such as OpenZeppelin's SafeERC20 handle the return-value inconsistencies. The rest require explicit decisions about which tokens a protocol will accept, documented as assumptions and enforced, where possible, at the point of listing. A rounding error, step by step Because rounding attacks are often hard to visualise, it is worth walking through a simplified case in words. Suppose a vault holds one thousand units of an asset and has issued one thousand shares, so each share is worth exactly one unit. Suppose its withdrawal function computes the shares to burn for a requested amount of assets by multiplying the amount by the total shares and dividing by the total assets, and that it rounds this result down. A user who asks to withdraw one unit burns exactly one share, and nothing is amiss. Now suppose, through accumulated interest, the vault holds one thousand and one units against its one thousand shares. A request to withdraw one unit now computes one thousand divided by one thousand and one, which is just under one, and rounds down to zero. The user receives one unit of asset and burns no shares at all. Repeated many times, each call extracts one unit of value from the other depositors at no cost but gas. On a network where gas is cheap, or in a vault whose units are worth a great deal, this is a profitable attack. The fix is to round the shares burned upwards, so that a request for one unit costs one share, and the user absorbs the fraction rather than the vault. The general rule, stated earlier, is that every rounding decision should favour the protocol. The worked case shows why the rule is not a matter of tidiness: a single rounding direction in a single line of a withdrawal function decides whether the vault is sound. Auditing arithmetic Reviewing arithmetic well requires a different posture from reviewing control flow. It is not enough to check that each operation is syntactically safe; the reviewer has to understand what each quantity represents, what its units and scale are, and what range of values it can take. A useful first step is to annotate every significant variable with its units and scaling: whether it is a raw token amount in that token's decimals, a value scaled by ten to the eighteenth, a price expressed as one asset per another, a rate per second. Many arithmetic bugs are unit errors, in which two quantities of different scale are added or compared, or a token with six decimals is treated as though it had eighteen. Annotation makes such errors visible. The second step is to identify every division and ask which way it rounds and whether that direction favours the protocol. Where the codebase provides explicit rounding helpers, such as mulDiv functions with a rounding mode, check that the mode chosen at each call site matches the intent. Where it relies on default truncation, check that truncation is the correct direction. The third step is to find the boundaries. What happens when a pool, vault or market is empty? When a balance is one unit? When a value is at the maximum of its type, or of the narrower type it will be cast to? When an exchange rate approaches zero or becomes very large? Testing at these boundaries is where fuzzing earns its keep, and where formal verification of the core mathematical properties, monotonicity, rounding direction and conservation of value, offers assurance that no amount of example-based testing can provide. Finally, the reviewer should treat any custom mathematical library, any unchecked block, any narrowing cast and any assembly as requiring a written argument for its safety. If the developer cannot state why the operation cannot overflow or lose precision in a harmful direction, the operation is not yet safe. Arithmetic is the place where a contract's promises meet the machine most directly, and the machine keeps only the promises that the code makes precisely. Chapter 4: Who May Act: Access Control, Initialisation and Upgrades Reentrancy and arithmetic errors are failures of mechanism: the code does something its authors did not foresee. Access control failures are often simpler and, measured in dollars lost, more costly. They occur when the code allows the wrong party to take an action, or when the right party is tricked or compromised into taking the wrong one. The bugs themselves can be embarrassingly small, a missing modifier, a function left public, an initialiser left uncalled. The consequences are routinely total. Every contract makes promises about authority. Only the owner can pause trading. Only the governance timelock can change parameters. Only a quorum of guardians can release funds from a bridge. Only the holder of a balance can move it, unless they have signed a permission. These promises are invariants in the same sense as accounting identities, and they can be falsified in three broad ways: the check is missing or wrong in the code; the check is correct but the contract was never properly set up, so the wrong party holds the role; or the check and the setup are both correct, but the party holding the role has lost control of it. This chapter takes each in turn, with particular attention to upgradeable proxies, which have made all three problems more consequential. Missing and mistaken checks The most elementary access control flaw is a function that should be restricted but is not. In Solidity, functions are public by default if no visibility is specified in older versions of the language, and even in modern code it is easy to mark an internal helper as external or public during development and forget to change it. Automated scanners look for exactly this, and contracts with unprotected functions that move funds, mint tokens or change ownership are sometimes drained within minutes of deployment. Subtler errors involve checks that exist but test the wrong thing. The classic example is the use of tx.origin for authorisation. tx.origin is the externally owned account that initiated the transaction, whereas msg.sender is the immediate caller. A contract that checks tx.origin == owner can be exploited if the owner is persuaded to interact with a malicious contract, perhaps by claiming an airdrop, which then calls the victim contract. From the victim's perspective, tx.origin is still the owner, so the check passes. Authorisation should always be based on msg.sender, and the use of tx.origin for any security purpose is a finding in itself. Other mistaken checks involve role systems that are more complicated than they need to be. Contracts using role-based access control, such as OpenZeppelin's AccessControl, can grant administrative rights over roles to other roles, and it is possible to build hierarchies in which some role can grant itself or others a power that was meant to be reserved. Reviewing access control means drawing out, in prose or on paper, the complete graph of who can grant what to whom, including at deployment, and checking that no path leads somewhere it should not. Authorisation by signature Many modern contracts authorise actions not by checking the caller but by verifying a signature. Token approvals via the permit function of EIP-2612, gasless meta-transactions, off-chain order books and bridge messages all work this way. Signatures move the access control question from the contract's code into cryptography and message design, and they introduce their own family of errors. A signature must commit to everything that should constrain the action it authorises. If it omits the chain identifier, it can be replayed on another chain where the same contract is deployed. If it omits the contract address, it can be replayed against a different contract that accepts the same message format. If it omits a nonce or deadline, it can be replayed indefinitely. The EIP-712 standard for typed structured data signing exists largely to make these commitments systematic, through a domain separator that binds the signature to a specific chain and contract. The ecrecover precompile, which recovers the signer's address from a signature, returns the zero address for invalid signatures rather than reverting. A contract that compares the recovered address to a stored signer, where the stored signer has never been set and is therefore also zero, will accept any malformed signature. ECDSA signatures are also malleable: for each valid signature there is a second, distinct one that recovers to the same address. A contract that uses the signature bytes themselves as a unique identifier, to prevent replay, can be bypassed with the malleable twin. OpenZeppelin's ECDSA library handles both issues and should be preferred over calling ecrecover directly. Cross-chain bridges are the most extreme case, since they authorise the release of funds on one chain based on attestations about events on another. The Wormhole bridge lost around 120,000 wrapped ether in February 2022 when an attacker bypassed signature verification on its Solana side by supplying a forged account in place of the system account that the verification logic was supposed to consult. The code checked that signatures had been verified but did not check that the verification it relied on was genuine. Although the flaw lived on Solana rather than an EVM chain, its shape is general: authorisation logic that trusts an input it should have validated. Initialisation and the proxy problem Upgradeable contracts are now the norm for serious protocols. The standard pattern places user-facing state in a proxy contract, which forwards all calls via delegatecall to a separate implementation contract that holds the logic. To upgrade, an authorised party points the proxy at a new implementation. Because delegatecall runs the implementation's code against the proxy's storage, the implementation never uses its own storage in normal operation. This architecture has a consequence that catches developers repeatedly. Constructors do not work. A constructor runs once, when a contract is deployed, and writes to that contract's storage, which for an implementation is storage nobody uses. Upgradeable contracts therefore replace constructors with an initialize function, which must be called on the proxy after deployment and must be protected so that it can only be called once. If it is not called, or not protected, anyone can call it and make themselves the owner. The Parity multisig wallet incidents of 2017 are the founding cases. Parity's wallet design placed most logic in a shared library contract, with each user's wallet a thin contract delegating to it. In July 2017, attackers discovered that the library's initialisation function could be reached through the wallet's fallback function, because the wallet delegated any unrecognised call to the library. Calling it on a wallet that was already in use reset the wallet's owners to the attacker. Around 150,000 ether was stolen from three wallets before a group of white-hat hackers used the same flaw to secure the funds in other vulnerable wallets. The fix for that bug introduced a second one. In November 2017, a user discovered that the new library contract, deployed as the shared logic for many wallets, had never itself been initialised. They called its initialisation function, became its owner, and then called its self-destruct function. The library was gone, and every wallet that depended on it was left delegating to empty code. Roughly 500,000 ether was frozen permanently, with no possibility of recovery short of a network change that was never adopted. The modern defences are well established. Initialisers should be guarded by a modifier that ensures they run once, as OpenZeppelin's Initializable provides. Implementation contracts should be locked at deployment by calling _disableInitializers in their constructor, so that no one can initialise the implementation directly. Deployment and initialisation of a proxy should happen in a single transaction, so that there is no window in which an attacker can front-run the initialisation. And upgrade logic in the UUPS pattern, where the upgrade function lives in the implementation rather than the proxy, must be protected in every implementation, since an implementation that omits it makes the proxy permanently non-upgradeable or, worse, upgradeable by anyone. OpenZeppelin disclosed a vulnerability of this last kind in its own UUPS implementation in September 2021, in which an uninitialised implementation could be taken over and destroyed, and coordinated fixes with affected projects before publication. Upgrades that break the rules Even when initialisation is correct, the act of upgrading introduces risk. Two forms deserve particular attention. The first is storage layout collision. Because the proxy's storage is interpreted by whichever implementation is current, a new implementation must lay out its state variables exactly as the old one did, appending new variables only at the end. If a developer inserts a variable in the middle, reorders variables, or changes a type, the new implementation will read existing data from the wrong slots. Proxies themselves need storage for the implementation address and administrator, and they must place it where no implementation variable can overwrite it. The EIP-1967 standard specifies pseudo-random slots for this purpose, and tooling such as the OpenZeppelin upgrades plugins checks layouts for compatibility. A reviewer of any upgrade should compare layouts explicitly. The second is upgrade logic that changes an invariant without anyone noticing. The Nomad bridge lost around 190 million dollars in August 2022 after a routine upgrade initialised a critical value, the trusted root against which incoming messages were verified, to zero. Because the default value of an unproven message was also zero, every message was treated as proven. Once the first attacker demonstrated the flaw, hundreds of others copied the transaction, replacing the recipient address with their own, in one of the most chaotic incidents on record. The code that failed had been audited. The upgrade that broke it was a configuration change whose interaction with an existing check no one had traced. These cases support a principle: every upgrade is a new deployment and deserves review as one. The questions to ask are the same as for the original code, plus two more. What invariants did the old version maintain, and does the new version still maintain them in combination with all existing state? And what values does the initialisation or migration logic write, and does any existing check depend on those values in an unexpected way? Governance as access control Many protocols hand ultimate authority not to a fixed set of keys but to governance: a system in which holders of a token vote on proposals, and approved proposals are executed on-chain, typically after a delay. Governance distributes control and is often presented as a security improvement over an administrator's key. It is better understood as a different access control mechanism, with its own failure modes. The most direct is the purchase of voting power. If a proposal can be created, voted on and executed within a short window, and if voting power is measured at the moment of the vote, an attacker can borrow or buy enough governance tokens to pass a proposal of their own design. The Beanstalk protocol lost roughly 180 million dollars in April 2022 in exactly this way. The attacker used a flash loan to acquire a supermajority of voting power, voted through a proposal that transferred the protocol's reserves to themselves, and repaid the loan, all within a single transaction. The protocol's emergency execution path allowed sufficiently large majorities to execute proposals immediately, which removed the delay that would otherwise have given the community time to react. The defences are now standard in mature governance designs. Voting power should be measured at a snapshot taken before the proposal was created, so that tokens acquired afterwards, including through flash loans, carry no weight. Approved proposals should pass through a timelock, a contract that holds them for a fixed period before execution, so that users who disagree can exit and monitors can raise the alarm. Proposal thresholds and quorum requirements should be set with the cost of acquiring voting power in mind, which for a token with thin liquidity may be much lower than its market capitalisation suggests. Subtler governance risks remain. A proposal's description may not match its code, and voters who read the description rather than the calldata may approve something they did not intend. A proposal may be technically correct but interact with existing state in ways nobody traced, as the Nomad upgrade did. A small group of large holders may effectively control outcomes, making governance a multisig by another name. Reviewing governance means reviewing not just the voting contracts but the process around them: who checks the calldata of each proposal, how long the timelock is, what powers the governance can exercise, and whether any emergency path bypasses the safeguards. The changing power of self-destruct One privileged operation deserves a note because the environment changed underneath it. Earlier versions of the EVM provided an opcode, SELFDESTRUCT, that deleted a contract's code and sent its ether to a chosen address. Several of the incidents in this chapter, including the November 2017 Parity freeze, depended on it. Proxy patterns had to guard against an attacker gaining control of an implementation and destroying it, which would leave every proxy pointing at empty code. The Cancun upgrade of March 2024, through EIP-6780, restricted the opcode so that it deletes code only when called in the same transaction that created the contract. In other cases it now only transfers the contract's ether. This closed off the most catastrophic consequence of an implementation takeover, but it did not make takeovers harmless: an attacker who controls an implementation can still, through delegatecall and storage manipulation, cause serious damage in many designs. As with other changes to the environment, the lesson is that defences designed around a particular opcode's behaviour need to be revisited when that behaviour changes, and that older contracts deployed before the change may still carry assumptions that no longer describe reality. When the keys are the vulnerability The largest losses in the industry's history have often involved no flaw in contract logic at all. Instead, the parties who held legitimate authority lost control of it. The Ronin bridge, which served the game Axie Infinity, required five of nine validator signatures to release funds. In March 2022, attackers obtained five of those keys, through a combination of compromising a company's infrastructure and exploiting an allowlist arrangement that had been set up temporarily and never revoked, and withdrew roughly 620 million dollars' worth of assets. The contract's access control functioned exactly as designed. The design simply concentrated too much authority in too few hands, several of which belonged to a single organisation. The Bybit theft of February 2025 took this pattern further. Bybit stored a large amount of ether in a Safe multisignature wallet, among the most widely used and heavily audited smart contract wallets in existence. According to the published forensic analyses, attackers compromised a developer machine associated with the Safe web interface and injected code into the interface that served Bybit's signers. When the signers approved what they believed was a routine transfer, they were in fact signing a transaction that used the wallet's delegatecall operation to run attacker-controlled code in the wallet's storage context. That code replaced the wallet's implementation pointer, giving the attacker complete control. Around 400,000 ether, worth roughly 1.5 billion dollars at the time, was drained. The FBI attributed the theft to North Korean actors. No line of the Safe contracts was wrong. The multisig did exactly what its signers authorised. The failure lay in the gap between what the signers saw and what they signed, and in the fact that a single transaction type, a delegate call to an arbitrary address, carried the power to rewrite the wallet entirely. The practical lessons are concrete. Signers of high-value transactions should verify transaction data independently of the interface that constructs it, ideally on a hardware device that displays the decoded call. Multisig configurations can and should restrict dangerous operations, such as delegate calls, through guard contracts or module restrictions. And large holdings should be divided so that no single transaction, however authorised, can move all of them. Reviewing authority Access control review is less about reading individual lines and more about building a complete picture of power in the system. The reviewer should produce a list of every privileged role, every function each role can call, and every mechanism by which roles are granted, revoked or transferred. For each role, the review should ask who holds it at deployment, who can hold it later, and what the worst possible use of it would be. If the answer to the last question is that a single compromised key could drain the protocol, that is a finding regardless of whether the code is correct. For upgradeable systems, the review should verify that every implementation is locked, every initialiser is protected and called atomically with deployment, every upgrade path is restricted, and the storage layout is consistent across versions. It should identify who can upgrade, through what delay, and whether users have time to exit before an upgrade takes effect. A timelock on upgrades turns a sudden compromise into a visible one and gives users and monitors a chance to react. Finally, the review should document trust assumptions plainly, in a form users can read. A protocol in which a three-of-five multisig can replace all logic at any moment is a protocol whose security is that of the multisig. That may be a reasonable choice for a young system, but it is a choice, and it should be stated rather than buried. Authority is the invariant that sits above all others: whoever can change the code can change every other promise the contract makes. Chapter 5: Order Matters: Front-Running, MEV and Oracle Manipulation Most programmers think of a function call as an event that happens on its own terms. Its inputs determine its output. On a blockchain this is no longer true in a sense that matters enormously. A transaction's outcome depends on the state it executes against, and that state depends on which other transactions ran before it, in which order, within the same block. That order is not decided by the user who sent the transaction. It is decided by whoever builds the block, and that party has every incentive to arrange transactions to their own advantage. This chapter covers the vulnerability classes that arise from transaction ordering, and the closely related problem of price manipulation. They are different from the bugs in earlier chapters in an important way. A reentrancy flaw or a missing access check is a mistake in the code. An ordering vulnerability is often a mistake in the design: a contract that behaves correctly for every transaction considered in isolation but leaks value when transactions are considered together. Many such contracts have passed audits whose reviewers examined each function carefully and never asked what would happen if someone else's transaction ran first. The public waiting room When a user submits a transaction on Ethereum, it has traditionally entered a public pool of pending transactions, known as the mempool, where every node can see it before it is included in a block. Anyone watching can see that a particular address is about to buy a particular token, liquidate a particular loan or claim a particular reward, and can see the exact parameters of the action. For most of Ethereum's early history, miners chose which pending transactions to include and in what order, generally prioritising those that paid the highest fees. This meant that anyone who saw a profitable pending transaction could send a similar transaction with a higher fee and have it executed first. That is front-running in its simplest form. A related technique, back-running, places a transaction immediately after a target, for instance to capture an arbitrage opportunity that the target's trade will create. In 2019, a team of researchers led by Philip Daian published a paper, later presented at the IEEE Symposium on Security and Privacy, titled "Flash Boys 2.0", which documented the existence of a competitive market of bots engaged in exactly this activity on decentralised exchanges. The paper introduced the term miner extractable value, now usually rendered as maximal extractable value or MEV, to describe the profit that whoever controls transaction ordering can extract by including, excluding or reordering transactions. It argued that this value was not a marginal curiosity but a systemic force that could distort the incentives of the consensus layer itself. Since Ethereum's move to proof of stake in September 2022, block production has largely separated into roles. Most validators outsource block construction through relay software, of which MEV-Boost from the Flashbots project is the dominant example, to specialised builders who compete to assemble the most profitable block. Searchers, who identify MEV opportunities, send bundles of transactions to builders through private channels. Users increasingly submit transactions through private order-flow services that do not broadcast them to the public mempool, in exchange for protection from front-running or a share of the value their transactions create. The machinery has changed, but the underlying fact has not: the order of transactions within a block is chosen by a party with a financial interest in it, and contracts must be designed accordingly. The forms of ordering attack Ordering exploits take several characteristic forms, which Table 3 summarises. Each is worth understanding in terms of the invariant it violates. Table 3. Common forms of ordering-dependent value extraction. Form What the extractor does Typical victim Main defences Front-running Copies or pre-empts a profitable action Claims, liquidations, initialisers Commit-reveal, access restriction Sandwich Trades before and after a victim swap Swaps with loose slippage Tight slippage limits, private order flow Back-running Acts on state a victim just created Usually benign arbitrage Often none needed Oracle manipulation Moves a price a protocol relies on Lending and derivatives Time-weighted or external oracles Just-in-time liquidity Adds and removes liquidity around a swap Passive liquidity providers Design of fee distribution Sandwich attacks are the most familiar form for users of decentralised exchanges. A user submits a swap on an automated market maker, specifying the minimum amount of output they are willing to accept. A searcher sees the pending swap, buys the same asset just before it, pushing the price up, lets the user's swap execute at the worse price, and then sells immediately after, capturing the difference. The user receives less than they would have, but still more than their minimum, so the transaction succeeds. The invariant violated here is one the contract never stated: that the user should receive something close to the price prevailing when they decided to trade. The only protection the contract offers is the slippage limit, and a loose limit is an invitation. For developers, the lesson is that any function performing a trade on behalf of a user must take a minimum output, or equivalently a maximum input, and a deadline after which the transaction will revert. Contracts that perform swaps internally, for example to rebalance a vault or harvest yield, must compute a sensible limit rather than passing zero. A surprisingly large number of protocol-level losses have come from internal swaps executed with no slippage protection at all, an open invitation to anyone watching. Generalised front-running threatens any transaction whose profitability does not depend on who sends it. A function that pays a reward to whoever first submits a correct answer, or that lets anyone claim a bounty by calling it, can be copied by a bot that simulates every pending transaction and resends the profitable ones from its own address. Initialisation functions on freshly deployed proxies are a classic target, as discussed in Chapter 4. So are transactions in which white-hat hackers attempt to rescue funds from a vulnerable contract: if the rescue transaction is visible in the mempool, a bot may front-run it and take the funds itself. Experienced rescuers now submit such transactions privately. The standard design defence is commit-reveal, in which the user first submits a hash of their intended action together with a secret, and only in a later block reveals the action itself. Because the commitment reveals nothing, it cannot be front-run, and by the time the reveal is visible, the commitment has already established priority. Commit-reveal adds latency and complexity and is not suitable for every use, but for auctions, games, and claims of any kind it is often the right tool. Other designs remove ordering sensitivity entirely: batch auctions, which settle all orders in a period at a single clearing price, are used by some exchange protocols precisely because they make the order of transactions within the batch irrelevant. Oracles and the price of a moment Every lending market, derivatives platform and stablecoin needs to know what assets are worth. It learns this from a price oracle. The design of that oracle is among the most security-critical decisions a protocol makes, and getting it wrong has been responsible for a long list of losses. The tempting design is to read the current price directly from an automated market maker, computed from the ratio of its reserves. This value is easy to obtain, entirely on-chain and always current. It is also trivially manipulable. A large trade moves the reserves and therefore the price, and with flash loans, the capital for a large trade is available to anyone for the duration of a single transaction. An attacker borrows heavily, trades to push the spot price of some asset up or down, interacts with the victim protocol while the price is distorted, perhaps borrowing against now-overvalued collateral or liquidating now-undervalued positions, reverses the trade, and repays the loan. All of this happens atomically, and if it does not produce a profit, it reverts. Harvest Finance lost roughly 24 million dollars in October 2020 to a variation of this pattern, in which an attacker repeatedly manipulated the price of stablecoins in a Curve pool that Harvest used to value its vault shares, depositing at an artificially low share price and withdrawing at a higher one. Inverse Finance lost a comparable amount in April 2022 when an attacker manipulated the pool that served as the source for its price feed. The Mango Markets incident of October 2022, on Solana, involved an attacker using two accounts to drive up the price of the protocol's own thinly traded token on the venues its oracle consulted, then borrowing against the inflated value of their position to withdraw well over 100 million dollars. That case was notable for its aftermath as well: the individual responsible publicly described his actions as a profitable trading strategy, and the matter became the subject of criminal and civil proceedings in the United States. The defences fall into two families. The first is to use a time-weighted average price rather than a spot price. Uniswap's second and third versions maintain cumulative price accumulators from which a contract can compute the average price over a window of minutes or hours. Manipulating such an average requires holding the price at a distorted level across many blocks, which is expensive and visible, and which flash loans, confined to a single transaction, cannot accomplish. Time-weighted averages are not invulnerable: since proof of stake, a validator who is selected to propose several consecutive blocks can in principle manipulate prices across them without arbitrage intervening, and a thin market can be pushed even over a long window. But they raise the cost of manipulation dramatically. The second family is to use an external oracle network, such as Chainlink, which aggregates prices from many off-chain and on-chain sources and publishes them on-chain through a network of independent node operators. This shifts trust to the oracle network and its data sources, and introduces risks of its own: stale prices if updates stop, incorrect prices if a source is compromised, and edge cases when a market's real price moves outside bounds the feed was configured to report. Contracts consuming such feeds must check that the reported data is recent, that it is within plausible bounds, and that the feed has not been deprecated. Many protocols combine an external feed with an on-chain check, refusing to act if the two diverge by more than a set tolerance. Beyond these technical defences, there is a design principle that matters more than any particular oracle. A protocol should never accept as collateral an asset whose market is so thin that its price can be moved by an amount of capital smaller than the value that could be borrowed against it. Mango Markets and several similar incidents were, at bottom, failures of this rule. The oracle reported the market price accurately. The market price was simply not a reliable measure of what the collateral was worth. Smaller ordering hazards Not every ordering problem involves large sums or sophisticated searchers. Several smaller hazards recur in audits often enough to deserve attention. The ERC-20 approve function has a well-known race. If an owner has approved a spender for one hundred tokens and submits a transaction to change the allowance to fifty, the spender can watch for that pending transaction, spend the original hundred before it lands, and then spend the new fifty after it, taking one hundred and fifty in total. The standard mitigations are to set allowances to zero before changing them, to use functions that increase or decrease allowances relative to their current value, or to use signed permits with deadlines that are consumed atomically. Contracts that manage approvals on behalf of users should avoid creating the race themselves. Deadlines are a second small hazard with large consequences. A swap submitted without a deadline can sit unexecuted while the market moves, and can then be included at a moment chosen by whoever orders the block, at a price that satisfies the user's original slippage limit but no longer reflects the market. Every user-facing function that depends on a price should accept a deadline after which it reverts, and contracts that call such functions internally should pass a meaningful deadline rather than the current timestamp, which provides no protection because it is always satisfied. Randomness is a third. Contracts that need unpredictable values, for lotteries, games or randomised assignments, have historically used values such as the block hash or block timestamp. These are known to whoever produces the block before anyone else, and in some cases can be influenced by them. Since proof of stake, the beacon chain's RANDAO value is available to contracts and is harder to manipulate, but a validator can still choose to skip proposing a block if the resulting value would be unfavourable, which gives them a limited form of influence. For outcomes of real value, verifiable random functions provided by oracle networks, or commit-reveal schemes involving the participants themselves, are the safer choice. Liquidations are a fourth. Lending protocols rely on third parties to liquidate unhealthy positions, and they pay those parties a bonus for doing so. Liquidation is therefore a competitive race, and the protocol's design determines who wins it and how much the borrower loses. A liquidation bonus that is too generous transfers excessive value from borrowers to searchers; one that is too small may leave positions unliquidated during volatile markets, threatening the protocol's solvency. Some designs use auctions that start with a small bonus and increase it over time, letting competition determine the price. Whatever the design, the reviewer should check that a liquidation always improves the health of the position being liquidated, and that it cannot be triggered by a price manipulated within the same transaction. A sandwich in numbers The mechanics of a sandwich attack become clearer with a simplified example worked through in prose. Consider a constant-product pool holding one hundred units of asset A and one hundred thousand units of asset B, so that the price of A is roughly one thousand B. Ignore fees for simplicity. A user submits a swap of ten thousand B for A, with a slippage limit allowing them to receive as little as eight units. Executed alone, the trade would move the pool to one hundred and ten thousand B, and to preserve the constant product the pool would hold about 90.9 units of A, so the user would receive about 9.1 units of A. A searcher sees the pending trade and places a purchase of ten thousand B ahead of it. That trade moves the pool to the state just described, and the searcher receives about 9.1 units of A. The user's trade then executes against the shifted pool: adding ten thousand B to make one hundred and twenty thousand, the pool must hold about 83.3 units of A, so the user receives only about 7.6 units. That is below the eight-unit limit, so in this example the user's transaction reverts and the searcher, having gained nothing, would also choose to revert their own trades. Had the user set a looser limit, of say seven units, their trade would have succeeded at 7.6 units, and the searcher would then sell their 9.1 units back into a pool now richer in B, recovering more B than they paid and keeping the difference. The example shows exactly how the slippage limit works as a defence. It bounds the amount a sandwich can extract, because any extraction beyond the limit causes the victim's trade, and therefore the attack, to fail. Setting it too loose hands the difference to whoever orders the block. Setting it too tight causes legitimate trades to fail when the market moves. Good interfaces compute a limit based on the expected price impact and current volatility, and contracts that trade on their own behalf must do the same rather than accepting any output at all. Designing for adversarial ordering Auditing for ordering vulnerabilities requires a shift in perspective. For each function, the reviewer should ask: what does this transaction reveal while pending, and who could profit from acting before or after it? What state does it read that another transaction in the same block could change? If an attacker could choose what happens immediately before this call, what would they choose? The answers point to design changes rather than line-level fixes. Functions that perform trades need explicit limits on price and time. Functions that award value to the first caller need commit-reveal or access restriction. Functions that depend on prices need oracles that cannot be moved within a single transaction and that are validated for freshness and plausibility. Functions that create arbitrage opportunities should consider whether that value could instead be captured by the protocol or returned to users, an area where newer designs such as order-flow auctions have begun to change the landscape. The deepest lesson of this chapter is that a contract's invariants must include properties of sequences, not just of states. It is not enough that every transaction leaves the books balanced. A lending market must also guarantee that no sequence of actions within one transaction can borrow more than the collateral is worth at a fair price. A vault must guarantee that no sequence of deposits and withdrawals can extract value from other depositors. Stating these properties is harder than stating accounting identities, and checking them requires tools that explore sequences of calls rather than single calls. The next three chapters describe how auditors do that work. Chapter 6: Reading Code Like an Auditor Knowing the vulnerability classes is necessary for auditing but nowhere near sufficient. Many reviewers who can explain reentrancy, overflow and oracle manipulation flawlessly still miss them in real code, because real code does not present its flaws one at a time with labels attached. Bugs hide in the interaction between a fee calculation and a liquidation threshold, in the assumption that a token has eighteen decimals, in a function that was safe until another function was added beside it. Finding them requires method: a disciplined way of building understanding, generating hypotheses and testing them, applied consistently enough that the reviewer's attention does not depend on luck. This chapter describes such a method. It is not the only one, and experienced auditors vary their approach to suit the code in front of them. But the elements described here appear, in some form, in the practice of most serious security teams, and they reflect the argument of this book: an audit is an attempt to find the contract's promises and falsify them. Before reading a line An audit begins with scope, and the scope must be precise. The reviewer should know exactly which commit of which repository is under review, which files are included, which compiler version and optimiser settings will be used for deployment, which external contracts the code will interact with, and on which chains it will be deployed. An audit of the wrong commit is worthless, and findings frequently arise from components that someone assumed were out of scope: a deployment script that misconfigures roles, a library with an unusual behaviour, a token the protocol plans to list next month. Next comes documentation. The developers should explain what the system is supposed to do, who its users are, what roles exist and what each can do, which external systems are trusted and to what degree, and which properties they believe the system guarantees. Many teams find this request harder than they expect. That difficulty is itself informative. If the developers cannot state clearly what the protocol promises, they have not tested whether it keeps those promises, and the auditor's first contribution is often to help articulate them. From the documentation and an initial reading, the reviewer builds a threat model. Who might attack this system, with what resources, and to gain what? Which assets are at risk, and how are they held? Which actors are trusted, and what would happen if each one became malicious or was compromised? Which external dependencies, from oracles to bridges to the tokens the protocol accepts, could behave unexpectedly? The threat model need not be long, but it should be written down, because it determines where the reviewer will spend their time. Writing down the invariants The centrepiece of an effective audit is a list of invariants, drawn from the documentation, from the code and from the reviewer's understanding of the domain. Some will be stated by the developers. Others the reviewer must infer: properties the system clearly relies on without anyone having said so. It helps to sort invariants into categories. Accounting invariants relate quantities that must stay consistent: total supply equals the sum of balances; the protocol's token holdings cover its liabilities; the shares outstanding correspond to the assets under management at the recorded exchange rate. Authority invariants describe who can do what: only the timelock can change risk parameters; only a position's owner can withdraw its collateral. Solvency invariants are specific to lending and derivatives: every position is either adequately collateralised or eligible for liquidation; liquidation always improves, never worsens, a position's health. Monotonicity invariants describe quantities that should only move in one direction: an interest index never decreases; a nonce only increments. Economic invariants concern sequences and incentives: no sequence of calls within a single transaction yields a profit without corresponding risk; depositing and immediately withdrawing never returns more than was deposited. The value of this exercise becomes clear in cases like the Euler Finance incident of March 2023. Euler was a lending protocol that had been audited multiple times by respected firms. An upgrade the previous year had added a function, donateToReserves, that allowed a user to give some of their deposited collateral to the protocol's reserves. The function correctly reduced the user's collateral, but unlike every other function that could reduce collateral, it did not check whether the user's position remained healthy afterwards. An attacker used a flash loan to build a heavily leveraged position, donated enough collateral to make it deeply insolvent, and then liquidated it from a second account under terms that, because of the size of the shortfall, were extraordinarily favourable to the liquidator. Roughly 197 million dollars was taken. In a rare outcome, the attacker later returned almost all of it. The missing check was a single line. The invariant it violated, that no user action may leave a position undercollateralised, was obvious once stated and was enforced everywhere else in the code. An auditor working from an explicit list of solvency invariants, checking each state-changing function against each one, would ask of donateToReserves exactly the question that revealed the flaw. A reviewer reading the function on its own, seeing that it correctly updated the balances it touched, could easily pass over it. Passes through the code With scope, threat model and invariants in hand, the reviewer reads the code. Most experienced auditors make several passes, each with a different focus, rather than trying to see everything at once. A first pass aims at comprehension. The reviewer traces the main user flows from entry point to completion, building a mental model of how state moves through the system. They note every external call, every privileged function, every place where value enters or leaves, and every assumption that seems to be made silently. At this stage, questions are more valuable than findings, and the best practice is to keep a running list of them. A second pass works through the invariants systematically. For each state-changing function, the reviewer asks which invariants it might affect, whether it preserves them on every path including error paths, and whether any external call occurs while an invariant is temporarily broken. This is where the reentrancy analysis described in Chapter 2 is applied, along with the access control review from Chapter 4. A third pass focuses on arithmetic and edge cases, applying the techniques of Chapter 3: annotating units and scale, checking the direction of every rounding, and examining behaviour at boundaries such as empty pools, single-unit balances and maximum values. It also considers the less glamorous vulnerability classes that do not fit neatly elsewhere. Denial of service is one: a function that loops over an array whose length users can grow without bound may eventually exceed the block gas limit, permanently locking whatever it guards. A function that sends funds to a list of recipients in sequence can be blocked by a single recipient whose fallback always reverts, an issue first illustrated by the King of the Ether game in 2016 and still seen in auction and distribution contracts. Unchecked return values from low-level calls or from tokens that return false instead of reverting can cause the contract to proceed as if a transfer succeeded when it did not. A fourth pass considers the system as an adversary would. Given everything the reviewer now knows, how would they attack it? What would they do with a flash loan of any size? What would they do if they controlled the order of transactions in a block? What tokens could they introduce, and what would those tokens do when transferred? What would they gain by front-running an administrator's transaction? This pass is where the reviewer's creativity matters most, and where knowledge of past incidents, including those in very different protocols, is most useful as a source of patterns. Throughout, the reviewer should write small tests to confirm or refute hypotheses. A suspicion that a function rounds the wrong way can be checked in minutes with a test that calls it with crafted values. A suspected reentrancy path can be demonstrated with a short mock contract in a local test environment. Proof-of-concept tests against a local fork turn suspicion into evidence, and they give the developers something concrete to reproduce and fix. A short walk through one function To make the method concrete, consider how an auditor might approach a single function in a hypothetical lending market: a borrow function that lets a user take out a loan against collateral they have deposited. The function checks that the market is not paused, calculates the user's borrowing capacity from their collateral and the current price, checks that the requested amount fits within that capacity, records the new debt, and transfers the borrowed tokens to the user. The comprehension pass establishes what each line does and notes the questions it raises. Where does the price come from, and can it be moved within a transaction? Which collateral assets count, and are their decimals normalised? Is interest accrued before the debt is recorded, or could a user borrow against a stale debt figure? Does the transfer happen before or after the debt is recorded, and could the borrowed token have hooks? The invariant pass checks the function against the list. The solvency invariant says that after any user action, every position is either healthy or eligible for liquidation; does borrow enforce that the position is healthy after the new debt is added, using the post-borrow figures rather than the pre-borrow ones? The accounting invariant says that total recorded debt equals the sum of individual debts; does the function update both? The authority invariant says only the position's owner can increase its debt; is there any path, perhaps through a delegated borrowing feature or a meta-transaction, where someone else can borrow on a user's behalf without consent? The arithmetic pass looks at the capacity calculation. It probably multiplies a collateral amount by a price and a collateral factor, then divides by some scale. In what order? Rounding in which direction? If the user borrows a tiny amount, could rounding let them borrow slightly more than their capacity, and could that be repeated? What happens if the price feed returns zero, or a stale value, or a value from a deprecated feed? The adversarial pass imagines the attacker. With a flash loan, could they inflate their collateral value briefly, borrow against it and leave the protocol with bad debt? Could they borrow, then use the borrowed tokens' transfer hook to re-enter and borrow again before their debt is recorded? Could they front-run a governance change to the collateral factor, borrowing heavily at the old, more generous rate? Could they deposit collateral that is itself a share token from another protocol, whose value can be manipulated through that protocol's read-only reentrancy window? None of these questions is exotic. Each is suggested directly by one of the vulnerability families in the previous chapters, applied to one function. The method's value lies in asking all of them, every time, for every function that matters, rather than trusting intuition to raise the right ones. Beyond the contracts in scope Some of the most expensive findings lie outside the files that were formally in scope. A disciplined audit extends its attention, at least briefly, to several adjacent areas. Deployment scripts and configuration are the first. A contract may be correct while the script that deploys it sets the wrong owner, forgets to call an initialiser, grants an administrative role to a developer's personal address, or configures an oracle with the wrong feed. Reviewing the deployment process, and verifying after deployment that the on-chain state matches what was intended, catches errors that no amount of source review will find. Integrations are the second. The tokens a protocol will accept, the oracles it will read, the external protocols it will call and the protocols that will call it all shape its security. The audit should list them, note their relevant behaviours, such as hooks, rebasing, upgradeability and pausability, and confirm that the code handles each. Off-chain components are the third. Keepers that trigger liquidations, relayers that submit meta-transactions, front-end interfaces that construct transactions for users and back-end services that sign messages all hold some form of authority. The Bybit incident showed that a compromised interface can turn a correct contract into a channel for theft. An audit focused on contract code cannot review every server, but it can identify which off-chain components hold which powers and flag those whose compromise would be catastrophic. Writing findings that get fixed An audit is only as valuable as the changes it produces. A finding that the developers do not understand, or do not believe, or cannot prioritise, will not be fixed. Writing findings well is therefore part of the security work, not an afterthought. Each finding should state the problem, the location, the impact and the recommended fix, in that order, in language a developer unfamiliar with the reviewer's reasoning can follow. It should include a proof of concept where possible, ideally as a test that fails before the fix and passes after. It should explain the conditions under which the issue is exploitable, because a flaw that requires the administrator to act maliciously warrants a different response from one that any user can trigger. Severity is usually assigned by combining impact, how bad the outcome would be, with likelihood, how easily and under what conditions it could happen. Table 4 shows a typical scheme. Conventions vary between firms and audit contest platforms, and what matters most is consistency and a clear explanation of the reasoning in each case. Table 4. A typical severity scheme for smart contract findings. Severity Typical impact Typical likelihood Example Critical Direct loss or permanent freezing of user funds Exploitable by any user now Unprotected initialiser on live proxy High Significant loss or protocol insolvency Requires specific but plausible conditions Oracle manipulable with flash loan Medium Limited loss, temporary denial of service Requires unusual state or privileged error Rounding favours user by small amounts Low Minor deviation from specification Unlikely or negligible effect Missing event on parameter change Informational No direct risk Not applicable Code clarity, gas improvements After the developers respond, a fix review checks that each change resolves its finding and introduces no new problem. Fixes are code, and code written under pressure to close an audit finding is particularly prone to new mistakes. A significant fraction of serious issues found in fix reviews were introduced by the fixes themselves. What an audit is and is not It is important to be honest about the limits of auditing. An audit is a time-boxed review by a small number of people. It can find many issues, and the best audits find the most serious ones with high probability. It cannot guarantee the absence of vulnerabilities, and protocols that describe themselves as audited, as though that settled the question of safety, mislead their users. Euler, Nomad and Balancer V2 had all been audited. Each audit was presumably done in good faith by competent reviewers. Each missed something, or reviewed code that was later changed. Mature security programmes therefore treat the audit as one layer among several. Multiple independent audits, ideally with different firms bringing different perspectives, find more than one. Competitive audit contests, in which many independent reviewers examine the same code for rewards based on what they find, add breadth. Bug bounty programmes, hosted on platforms such as Immunefi, give white-hat researchers a legitimate incentive to report flaws after deployment rather than exploit them. Automated tools, the subject of the next chapter, provide consistency that human attention cannot. Formal verification, the subject of the chapter after that, provides guarantees for the most critical properties. And monitoring after deployment, watching for unusual transactions, large movements of funds and governance actions, with the ability to pause the system quickly if something goes wrong, limits the damage when every other layer fails. Community resources help reviewers stay calibrated. The Smart Contract Weakness Classification registry, though no longer actively maintained, remains a useful catalogue of classic issues. The OWASP Smart Contract Top 10, updated periodically, ranks the most significant vulnerability categories based on recent incidents. Published audit reports and post-incident analyses from security firms are, collectively, the best education available, because they show how real flaws looked in real code before anyone knew they were flaws. An audit, done well, is a disciplined conversation between the people who built a system and the people trying to break it, conducted in the language of invariants. Its output is not a certificate. It is a sharper understanding of what the system promises, which of those promises it keeps, and where it is still exposed. Chapter 7: Machines That Read Code: Static Analysis, Fuzzing and Symbolic Execution Human reviewers are good at understanding intent. They can tell when a function does something its name does not suggest, when a design choice creates an incentive problem, or when two parts of a system make incompatible assumptions. They are poor at consistency. They tire, they skim, they see what they expect to see, and they cannot hold every path through a thousand-line contract in their heads at once. Automated tools have the opposite profile. They have no idea what a contract is for, but they never get bored, and they will check every instance of a pattern, every path or every random input with the same diligence. The practical consequence is that tools and human review are complements, not substitutes. Each tool family is good at a particular kind of question, and a security programme built around the invariants of the previous chapters uses each where it is strongest. This chapter surveys the three main families, static analysis, fuzzing and symbolic execution, and explains what each is good for, where each falls short, and how to make them earn their keep. Table 5, towards the end of the chapter, summarises the comparison. Static analysis A static analyser examines code without running it. It parses the source or bytecode into an internal representation, builds structures such as control-flow graphs and data-flow relationships, and runs a set of detectors that look for patterns associated with known vulnerabilities. The most widely used static analyser for Solidity is Slither, developed by Trail of Bits and released as open source. Slither converts Solidity into an intermediate representation called SlithIR and runs dozens of detectors over it, covering issues such as reentrancy, uninitialised storage, dangerous uses of delegatecall, unchecked return values, shadowed variables, incorrect use of tx.origin and many others. It also provides printers that summarise a codebase's structure: inheritance graphs, function summaries, lists of who can call what, and the storage layout of each contract. Experienced auditors often use these printers as much as the detectors, because they speed up the comprehension pass described in Chapter 6. Newer analysers, such as Aderyn from Cyfrin, follow a similar approach with different detector sets and implementation choices. Static analysis has two great virtues. It is fast, typically finishing in seconds or minutes, so it can run on every commit in a continuous integration pipeline and catch regressions the moment they are introduced. And it is consistent: if a detector exists for a pattern, every instance of that pattern will be flagged, regardless of how deeply it is buried. Its limitations are equally clear. A static analyser can only find what its detectors describe, and most detectors describe syntactic or structural patterns rather than semantic properties. Slither can tell you that a function makes an external call before writing to a state variable; it cannot tell you whether that state variable participates in an invariant that the call could exploit. As a result, static analysers produce false positives, flagging patterns that are harmless in context, and false negatives, missing flaws that do not match any pattern. A team that runs a static analyser and triages its output carefully gains real value. A team that treats a clean report as evidence of security gains nothing but false comfort. The most effective use of static analysis is as a floor rather than a ceiling. Run it early, run it often, investigate every high-severity finding, and suppress false positives explicitly with a written reason so that future readers know they were considered. Some teams also write custom detectors for patterns specific to their own codebase, such as a rule that every function modifying a particular mapping must carry a particular modifier, which turns a project-specific invariant into an automated check. Fuzzing and invariant testing A fuzzer runs the code. It generates large numbers of inputs, many of them random or semi-random, calls the contract with them and checks whether any property the developer specified has been violated. Where a unit test asks whether the code behaves correctly for a handful of examples the developer thought of, a fuzzer asks whether it behaves correctly for thousands or millions of examples the developer did not think of. Two forms matter for smart contracts. Stateless fuzzing calls a single function with random arguments and checks a property of the result. It is useful for pure functions such as mathematical libraries, where it can find inputs at which rounding goes the wrong way or an overflow check fails. Stateful fuzzing, often called invariant testing, generates random sequences of calls across many functions, possibly from many simulated users, and checks after each call that the contract's invariants still hold. This is the form most relevant to the vulnerability classes in this book, because the most serious bugs are rarely triggered by one call with an unusual argument; they are triggered by a particular sequence of calls that puts the contract into a state its designers did not anticipate. Echidna, also from Trail of Bits, was one of the first property-based fuzzers built specifically for Ethereum contracts and remains widely used. Medusa, a newer fuzzer from the same ecosystem, adds parallel execution and other performance improvements. The Foundry development framework, now the default toolchain for many Solidity teams, includes both stateless fuzz tests and stateful invariant tests directly in its testing framework, which has done a great deal to bring fuzzing into everyday development rather than leaving it to specialist auditors. An invariant test in Foundry typically consists of a handler contract, which wraps the system's functions and constrains their inputs to plausible ranges, and an invariant function, which asserts a property that should always hold. The fuzzer calls handler functions in random order with random arguments and evaluates the invariant after each call. // Invariant: the vault always holds at least what it owes. function invariant_solvency() public view { uint256 owed = vault.convertToAssets(vault.totalSupply()); assertGe(asset.balanceOf(address(vault)), owed); } Writing good invariant tests takes skill. If the handler constrains inputs too tightly, the fuzzer never reaches the interesting states. If it constrains them too loosely, most calls revert and the fuzzer wastes its effort. Developers often add ghost variables to handlers, tracking quantities such as the total deposited by all users, so that invariants can compare the contract's view of the world against an independent record. A fuzzer is also only as good as its properties: if no invariant describes the flaw, the fuzzer will execute the buggy sequence and report nothing. This is the point at which the invariant list developed during review pays off a second time. Each invariant the auditor wrote down in prose becomes a candidate for an invariant test. Some will be easy to express; others, particularly economic properties about profit across sequences, will require careful thought. The process of translating them into code frequently reveals ambiguities in the specification that are themselves findings. Fuzzing is often extended with fork testing, in which tests run against a local copy of mainnet state. This allows invariants to be checked against real tokens, real oracles and real liquidity, catching problems that arise only when the contract interacts with the actual systems it will be deployed alongside. Many of the integration flaws described in Chapter 2, where a protocol consumed a value from another protocol at an unsafe moment, can only be found this way. A related technique, mutation testing, evaluates the test suite itself. It deliberately introduces small bugs into the code, such as changing a comparison operator or deleting a line, and checks whether the tests catch them. If a mutation survives, the tests have a blind spot. Tools such as Gambit, developed by Certora, generate mutations for Solidity. Mutation testing is a useful way to find out whether a high test coverage figure reflects genuine scrutiny or merely execution. Symbolic execution Symbolic execution sits between testing and formal proof. Instead of running the code with concrete values, it runs it with symbolic values, variables that stand for any possible input. As execution proceeds, the tool accumulates a set of constraints describing the conditions under which each path is taken. When it reaches a point of interest, such as a failed assertion, it asks a constraint solver whether any assignment of inputs satisfies the constraints for that path. If the solver finds one, the tool has found a concrete input that triggers the failure. If the solver proves that none exists, that path is safe for all inputs. Mythril, originally developed by ConsenSys, applies symbolic execution to EVM bytecode to search for common vulnerability patterns. Halmos, developed by a16z, takes a different approach suited to modern workflows: it executes Foundry tests symbolically, so that a test written with arguments can be checked for all possible values of those arguments rather than for random samples. The hevm tool, which originated in the DappTools project, provides symbolic execution and equivalence checking at the EVM level and is used both for testing and for verifying that two implementations behave identically. The power of symbolic execution is that, for the paths it explores, it is exhaustive. A fuzzer might run a function a million times and never hit the one input that triggers an overflow; a symbolic executor will find it if it exists on an explored path. Its weakness is path explosion. Every branch doubles the number of paths, loops multiply them further, and complex arithmetic produces constraints that solvers struggle with. In practice, symbolic execution handles small functions and bounded sequences of calls well, and struggles with large systems and long sequences. Tools manage this with bounds on loop iterations and call depth, which means that a clean result is a guarantee only within those bounds. The most productive uses of symbolic execution are targeted. Mathematical libraries, where a small amount of code carries a great deal of risk and the properties are precise, are ideal candidates. So are individual functions with complex branching logic, and equivalence checks between an optimised implementation and a simple reference version. The Cetus incident described in Chapter 3, where a hand-written overflow check compared against the wrong threshold, is the kind of bug that a symbolic check of a short function against its intended property would expose quickly. Building an invariant suite Because stateful fuzzing is the tool most directly connected to the argument of this book, it is worth describing in more detail how a team might build an invariant suite for a typical system, such as a tokenised vault that accepts deposits of one asset and issues shares. The first step is to translate the prose invariants into checkable statements. "The vault is solvent" becomes an assertion that the vault's balance of the underlying asset is at least the value of all outstanding shares at the current exchange rate. "Shares and assets are consistent" becomes an assertion that the sum of every user's share balance equals the total supply. "No one profits from a round trip" becomes a check, performed inside a handler, that depositing an amount and immediately redeeming the resulting shares never returns more than was deposited. "Only the owner can change the fee" becomes an assertion that the fee has not changed unless the handler recorded a call from the owner. The second step is to write handlers. Each handler wraps one of the vault's functions, chooses an actor from a small set of simulated users, constrains the arguments to values the function could plausibly accept, and tracks any ghost variables the invariants need. A deposit handler might bound the amount to the actor's balance, make the deposit, and add the amount to a running total of all deposits. A withdrawal handler might bound the amount to what the actor can withdraw. A handler that simulates a direct transfer of assets to the vault, bypassing the deposit function, lets the fuzzer explore donation attacks. A handler that advances time lets it explore interest accrual and deadlines. The third step is to run the suite with enough depth and enough runs to reach interesting states, and to investigate every failure. A failing invariant comes with a sequence of calls that triggered it, which the fuzzer will usually shrink to a minimal form. Sometimes the failure reveals a real bug. Sometimes it reveals that the invariant was stated wrongly, for example that solvency should allow for a small rounding tolerance, or that the handler allowed an impossible state. Both outcomes are useful. Correcting a wrongly stated invariant clarifies the specification; correcting a handler tightens the model of how the system is used. The fourth step is to keep the suite alive. Invariant tests should run in continuous integration, with short campaigns on every change and long ones overnight or before releases. When the code changes, the handlers and invariants should change with it. When a bug is found by any means, including manual review or a bug bounty report, the team should ask why the suite did not find it and add the missing invariant or handler. Over time, a suite maintained this way becomes one of the most valuable assets a project has: an executable statement of what the system promises, continuously checked. A hypothetical illustration shows the kind of result this produces. Suppose the vault's withdrawal function rounds the shares to burn downwards, as in the worked example in Chapter 3. A unit test that withdraws round numbers from a vault at a one-to-one exchange rate would pass. An invariant suite with an interest-accrual handler and a withdrawal handler that tries small amounts would, after enough runs, reach a state in which the exchange rate is not exactly one and a tiny withdrawal burns zero shares. The round-trip invariant would fail, as would solvency over repeated calls, and the shrunk sequence, a deposit, some accrued interest and a one-unit withdrawal, would point directly at the faulty line. Choosing and combining tools Table 5 summarises the strengths and weaknesses of the three tool families, with examples of each. Table 5. Automated analysis approaches compared. Approach Example tools Best at Main limitation Static analysis Slither, Aderyn Known patterns, fast feedback in CI Syntactic; false positives and negatives Stateless fuzzing Foundry fuzz, Echidna Edge cases in single functions Random sampling may miss rare inputs Stateful fuzzing Echidna, Medusa, Foundry invariants Bugs reached by call sequences Only as good as handlers and properties Symbolic execution Halmos, hevm, Mythril Exhaustive checks on bounded paths Path explosion on large systems A sensible workflow layers them. Static analysis runs on every commit and catches regressions immediately. Stateless fuzz tests accompany unit tests for every function that performs arithmetic. A suite of invariant tests, derived from the specification and maintained alongside the code, runs nightly with long campaigns and before every release. Symbolic execution targets the handful of functions where exhaustive assurance is worth the effort. And the outputs of all of them feed back into human review: a fuzzer's counterexample is a starting point for understanding, not just a bug to patch. In the last few years, tools based on large language models have also entered the field, offering to review code, suggest invariants or triage the output of other tools. They can be useful aids for comprehension and for generating hypotheses, but their output must be treated like that of a capable but unreliable junior colleague: worth reading, never worth trusting without verification. A model that confidently describes a nonexistent vulnerability wastes time; one that confidently declares code safe is dangerous. The common thread is that every tool needs a statement of what correct behaviour looks like. Static analysers carry generic statements in their detectors. Fuzzers and symbolic executors need properties written by someone who understands the contract. The more precisely those properties capture the contract's real promises, the more each tool can find. Taken to its limit, that precision leads to formal verification, where the properties are stated completely enough that a machine can prove them for every possible input and every possible sequence of calls. Chapter 8: Proof, Not Persuasion: Formal Verification in Practice Every technique described so far gathers evidence. A manual review produces the considered judgement of a skilled reader. A fuzzer produces the absence of counterexamples across millions of random runs. A symbolic executor produces exhaustive results within bounds. All of this is valuable, and none of it settles the question absolutely. There is always one more sequence of calls, one more combination of inputs, one more path beyond the bound. Formal verification aims to close that gap. It uses mathematical reasoning, carried out or checked by a machine, to prove that a contract satisfies a stated property for every possible input, every possible state and every possible sequence of calls. When it succeeds, the result is not a strong suspicion but a theorem. That is a remarkable thing to be able to say about software that holds money, and for the most critical properties of the most valuable contracts it is increasingly expected. It is also easy to misunderstand. A formal proof is a proof of exactly what was stated, about exactly the model that was analysed, under exactly the assumptions that were made. The history of verified systems includes many cases where the proof was sound and the system still failed, because the statement was not the property that mattered, or the model left something out, or an assumption turned out to be false. This chapter explains how formal verification of smart contracts works, how it is used in practice, and how to read its results honestly. Specifications, models and provers Formal verification has three ingredients. The first is a specification: a precise, machine-readable statement of the properties the contract should satisfy. The second is a model of the contract and its environment: a mathematical representation of what the code does, derived either from the source code or from the compiled bytecode. The third is a prover: a tool that takes the specification and the model and attempts to establish that every behaviour of the model satisfies the specification, or else to produce a counterexample showing a behaviour that does not. Properties take several forms. An invariant states something that holds in every reachable state, such as the sum of balances equalling total supply. A rule describes the effect of an operation, such as "after a successful transfer of an amount from one address to another, the sender's balance decreases by that amount and the recipient's increases by it, and no other balance changes." A parametric rule quantifies over every function in the contract: "no function, called by anyone other than the owner, can change the fee." Parametric rules are particularly powerful because they automatically cover functions added later, which is exactly the protection that would have caught the Euler flaw described in Chapter 6, where a newly added function failed to enforce a solvency check present everywhere else. Provers rely on automated reasoning engines, chiefly satisfiability modulo theories solvers, which can decide whether complex logical formulas involving integers, arrays and bit vectors have solutions. The prover translates the question "is there any execution that violates this property?" into such a formula and asks the solver. If the solver reports that no solution exists, the property is proved. If it finds one, the solution describes a counterexample: a concrete starting state and sequence of inputs that lead to a violation. If the formula is too hard, the solver may time out, and the result is inconclusive. The tools in use Several approaches to formal verification are in active use for Ethereum contracts, ranging from tools built into the compiler to research-grade frameworks. The Solidity compiler itself includes a component called the SMTChecker, which can attempt to prove assertions in the code, detect potential overflows and identify unreachable code. It is enabled through compiler settings and requires no separate tooling, which makes it an accessible starting point. Its capabilities are limited for large contracts and interactions between contracts, but for self-contained logic it can provide genuine guarantees with little effort. The Certora Prover is the most widely deployed commercial tool for verifying DeFi protocols. Developers write specifications in the Certora Verification Language, which supports rules, invariants and parametric rules, along with ghost variables and hooks that track quantities across storage updates. The prover works on compiled bytecode, which means its results apply to what is actually deployed rather than to an idealised version of the source, an important property in light of the Vyper compiler bug described in Chapter 2. Major lending protocols, including Aave and Compound, have used it to verify core properties, and Certora made the prover available as open source in 2025. A rule in this style reads much like a precise statement of intent. The following is a simplified example, written in the spirit of such a specification, stating that withdrawing never increases a user's share balance: rule withdrawNeverIncreasesShares(address user, uint256 assets) { uint256 before = balanceOf(user); withdraw(assets, user, user); assert balanceOf(user) <= before; } The K framework, developed by researchers associated with the University of Illinois and the company Runtime Verification, takes a more foundational approach. KEVM is a complete formal semantics of the Ethereum Virtual Machine written in K, executable and tested against the official Ethereum test suite. Proofs in KEVM are proofs about the actual bytecode behaviour as defined by the formal semantics. Runtime Verification used this approach in verification work on early versions of the multi-collateral DAI system and on the deposit contract for Ethereum's proof-of-stake transition, and its Kontrol tool now brings KEVM-based verification to Foundry test suites. This kind of deep verification requires considerable expertise but offers among the strongest guarantees available. For a small number of critical components, teams go further still, using interactive theorem provers such as Coq, Isabelle or Lean to construct proofs by hand with machine checking at each step. This is labour-intensive and reserved for code whose correctness is foundational, such as cryptographic primitives, mathematical libraries or the core algorithms of systems on which much else depends. A specification exercise The difficulty and the value of formal verification both become clear when one tries to specify even a simple contract completely. Consider an ordinary fungible token with transfer, approval and transfer-from functions, plus a mint function restricted to an owner. The obvious properties come first. A successful transfer of an amount from one address to a different address decreases the sender's balance by that amount and increases the recipient's by the same amount. A transfer to oneself leaves the balance unchanged. A transfer of more than the sender holds reverts. These are rules about a single function, and they are easy to state. Next come properties about other state. A transfer changes no balances other than those of the sender and recipient, and changes no allowances at all. This is a frame condition, a statement about what an operation does not touch, and frame conditions are where many real bugs hide, because a function that does its intended job correctly may also do something unintended elsewhere. Tools that support quantification over all addresses make such properties expressible. Then come invariants. The sum of all balances equals total supply. This cannot be checked by reading storage directly, because a mapping has no enumerable list of keys, so specification languages provide ghost variables updated by hooks on every storage write: each time a balance changes, the ghost sum changes by the same amount, and the invariant compares the ghost sum with the recorded supply. Then come parametric properties. Total supply changes only when the owner calls mint. Any function other than transfer-from, approve and the permit function leaves every allowance unchanged. A user's balance can decrease only through a function they called or a function called by someone they approved. These rules quantify over every function in the contract, including any added later, and they are often the most valuable in the specification, because they encode the contract's promises about authority in a form that no future change can silently violate. Even this small exercise raises questions that the original developers may not have considered. Should a transfer of zero tokens succeed or revert? Should approving an address one already approved emit an event? What happens if the owner mints to the zero address? Is the owner allowed to mint at all once some cap is reached? A formal specification forces each of these to be decided and written down. In a token, the answers are mostly matters of convention. In a lending market or a vault, the equivalent questions concern solvency and the movement of user funds, and deciding them explicitly is a security improvement in its own right. Equivalence and optimisation One application of formal methods is particularly practical. Teams often write a straightforward version of a function first and then optimise it for gas, sometimes rewriting it in assembly. Optimised code is harder to read and more likely to contain subtle errors, especially in arithmetic and memory handling. Equivalence checking proves that the optimised version produces the same outputs and the same state changes as the reference version for every input. If the reference is simple enough to be reviewed with confidence, equivalence transfers that confidence to the optimised code. Symbolic tools such as hevm, and formal tools built on EVM semantics, support this kind of check, and it is one of the most cost-effective ways to apply formal methods to real code. What a proof guarantees The guarantee of a successful formal verification is strong but specific. It says that the model of the code satisfies the specification under the stated assumptions. Each part of that sentence deserves scrutiny. The specification may be incomplete. A proof that total supply equals the sum of balances says nothing about whether the right people hold those balances. A proof that no one but the owner can change the fee says nothing about whether the fee calculation is correct. The question an auditor must always ask of a verified contract is not "was it verified?" but "which properties were verified, and are they the properties that matter?" The answer often reveals large gaps. Economic properties, such as resistance to oracle manipulation or the absence of profitable sequences of calls, are particularly hard to specify formally, and many verified protocols have specifications that cover accounting and authority thoroughly while saying nothing about economic attack surfaces. The specification may be wrong. A property written incorrectly can be proved trivially and mean nothing. The most dangerous form of this is vacuity: a rule whose preconditions can never be satisfied together is proved for every execution because there are no executions to which it applies. Mature tools include sanity checks for vacuity, verifying that each rule's preconditions are reachable, and teams should always run them. A useful complementary practice is to mutate the code deliberately, as described in Chapter 7, and confirm that the specification catches the mutations. A specification that proves the same rules for the buggy and the correct code is not specifying what it claims to. The model may omit something. Verifying a single contract usually requires assumptions about the contracts it calls. A token might be modelled as a standard ERC-20 with no hooks, an oracle as returning any value within some range, an external protocol as behaving according to its documentation. If the real token has transfer hooks, or the real oracle can be manipulated, or the real protocol has a bug, the proof's assumptions are violated and its conclusions do not hold. The read-only reentrancy incidents of Chapter 2 are a case in point: a proof about a lending market that modelled the pool's price function as a pure, consistent read would have missed exactly the window the attackers used. The assumptions may be false. Proofs typically assume that the compiler is correct, that the EVM behaves as specified, that certain addresses are honest, and that loops run for bounded numbers of iterations. Each is usually reasonable and each has, on occasion, failed. Verifying bytecode rather than source removes dependence on the compiler; formal semantics of the EVM reduce dependence on informal descriptions of its behaviour. But some assumptions, such as the honesty of governance or the integrity of signers' devices, are outside the reach of any contract-level proof. The Bybit theft of 2025, which involved no contract flaw at all, is a reminder of how much of a system's security lies outside its code. Verification in a security programme Given these limits, how should a team use formal verification? Experience across the industry suggests several principles. Verification works best on a well-defined core. Rather than trying to verify everything, identify the small set of contracts and functions whose correctness is essential: the accounting of a vault, the solvency checks of a lending market, the mathematical heart of an automated market maker. Specify the properties of that core carefully and prove them. The Balancer V2 incident described in Chapter 3 shows why the mathematical core deserves particular attention. Its stable-pool invariant was central to the system's safety, and the flaw that broke it lived in the rounding behaviour of precisely that calculation at extreme values. Specification should start early. Writing a formal specification forces precision about what the system is supposed to do, and doing so during design, before the code exists, often reveals ambiguities and design flaws that would otherwise surface only in audit or in production. Teams that write specifications alongside code also find that the specification becomes living documentation, kept current because the proofs break when it is not. Verification complements, rather than replaces, the other layers. It should be preceded by manual review, which identifies which properties matter, and accompanied by fuzzing, which finds bugs quickly and cheaply in the parts of the system that are not formally specified. Counterexamples from the prover should be turned into regression tests. Properties that resist formal proof can often still be fuzzed. Results should be reported precisely. A security disclosure that says a protocol is "formally verified" without listing the verified properties is, at best, uninformative and, at worst, misleading. The honest statement lists the properties proved, the contracts and functions they cover, the assumptions made about external systems, and the properties deliberately left out of scope. Users and integrators can then judge what the proof actually protects them against. Formal verification is the strongest tool available for establishing that a contract keeps its promises. It is also the tool that most clearly demonstrates the argument of this book: the quality of the assurance depends entirely on the quality of the stated promises. A prover can establish an invariant beyond doubt. It cannot tell you that you wrote down the right one. Conclusion Ten years after The DAO, the vulnerability that drained it is still being found in new code, and the variants it spawned, cross-function, cross-contract, read-only, compiler-induced, continue to surprise competent teams. Arithmetic overflow has been largely tamed by checked arithmetic in the compiler, only for precision and rounding to take its place as the arithmetic failure that costs nine figures. Access control failures have migrated from missing modifiers to uninitialised proxies, mis-specified upgrades and compromised signers. Ordering attacks have grown from a curiosity into an industry with its own infrastructure. Through all of it, the same pattern recurs: a contract made a promise that nobody stated precisely enough to test, and someone found the sequence of events in which the promise failed. That pattern suggests what should change. The first change is in how developers and auditors think about vulnerability classes. Lists of known bugs are useful as reminders, but they are backward-looking by nature, and the most damaging incidents tend to be either new combinations of known ideas or failures in places the lists do not cover. The durable skill is the ability to state what a contract guarantees and to reason about every way the execution environment might allow that guarantee to be broken: by re-entry while state is inconsistent, by arithmetic that produces a value no rule allows, by an actor who should not have the power to act, or by an ordering of transactions the designers did not imagine. A developer who thinks this way will catch the next novel bug more often than one who has memorised every past one. The second change is in how invariants are treated. In most projects, the properties a contract must maintain live in the developers' heads, partially in documentation and almost never in a form that can be tested. Moving them into explicit, written specifications, first in prose and then in code as invariant tests and formal rules, is the single most valuable investment a team can make in its own security. It improves design, because ambiguities surface early. It improves review, because auditors can check functions against stated properties rather than guessing at intent. It makes fuzzing effective, because fuzzers need properties to check. And it makes formal verification possible, because provers need specifications to prove. The Euler incident is the clearest illustration in this book: a single unstated solvency check, enforced everywhere but in one new function, was the difference between a sound protocol and the loss of nearly two hundred million dollars. The third change is in how assurance is communicated. "Audited" has become a word that protocols use to reassure users and that users have learned, through painful experience, to discount. Nomad, Euler and Balancer V2 were all audited. The problem is not that audits are useless; they find a great many serious issues. The problem is that the word conveys no information about which properties were checked, by what methods, under which assumptions, and for which version of the code. A more honest practice would publish the specification alongside the code, list which properties were reviewed manually, which were fuzzed, which were formally proved, and which assumptions about external systems, privileged roles and the execution environment were made. Users and integrators could then make informed decisions, and the next team to build on the protocol would know exactly what had and had not been established. What remains unsettled Some problems in this field do not yet have good answers, and it would be misleading to suggest otherwise. Economic security is the largest. The tools for verifying accounting and authority properties are mature; the tools for establishing that no sequence of actions can extract value from a protocol, given arbitrary capital, arbitrary ordering and arbitrary interactions with other protocols, are not. Such properties are hard to specify, harder to prove, and depend on the behaviour of markets and other contracts that no single team controls. Oracle design, collateral selection and resistance to manipulation remain matters of judgement as much as engineering. Composability compounds the problem. Every protocol's security depends on the behaviour of the protocols it integrates with, and those dependencies change as other teams upgrade their code, governance adds new assets and new standards introduce new callbacks. There is no reliable way yet to verify a property of a system whose components are maintained by independent teams on independent schedules. The best current practice is defensive design, assuming that any external component may behave adversarially, combined with monitoring for the moments when it does. The human layer is perhaps the most stubborn. The largest theft in the industry's history involved no smart contract vulnerability at all. It involved signers approving a transaction they could not properly inspect, through an interface that had been compromised further upstream. Code-level verification cannot protect against that. Better signing tools, stricter limits on what any single transaction can do, separation of holdings and slower, more visible processes for high-value operations can reduce the risk, but they require organisational discipline that is often in tension with the speed the industry prizes. Finally, the environment keeps moving. EIP-7702 changed what an externally owned account can do. Transient storage changed how reentrancy guards are implemented. Changes to gas pricing, new precompiles, new account models and new layer-two systems with subtly different semantics will continue to arrive. Each change can invalidate assumptions that audited code relies on. A contract's security is not a fixed property established at deployment; it is a relationship between the code and an environment that will not stay still. What to do differently For developers, the practical implications are concrete. Write down your invariants before you write the code that maintains them. Follow checks-effects-interactions everywhere, and use reentrancy guards consistently across every function that touches shared state, treating every token transfer as an external call. Use a current compiler and understand exactly where its arithmetic checks stop. Lock implementations, initialise proxies atomically, protect every upgrade path, and put a delay on upgrades long enough for users to react. Give every trade a slippage limit and a deadline. Never read a price that can be moved within a single transaction. Turn your invariants into fuzz tests, run them continuously, and verify formally the small core whose failure would be catastrophic. For auditors, the implications are about posture. Begin with the promises, not the code. Treat every assumption as a candidate finding until it is justified. Spend as much attention on integrations, upgrades and deployment configuration as on core logic. Write findings that a developer can reproduce and fix. Be precise about what your review covered and what it did not, and resist the pressure to let a report be read as a guarantee. For both, the deepest lesson is a kind of intellectual honesty. Smart contract security is unforgiving because the machine does exactly what the code says, publicly, irreversibly and in front of people who are paid to find the gap between what the code says and what its authors meant. Closing that gap is not a matter of finding a better checklist or a cleverer tool. It is a matter of saying, precisely and in advance, what the contract is supposed to do, and then doing the patient work of finding out whether it does. Hashtags: #BlockchainSecurity #SmartContractSecurity #SmartContractAuditing #SmartContractVulnerabilities #ReentrancyAttacks #ReentrancyPrevention #SoliditySecurity #EthereumSecurity #DecentralizedFinance #DeFiSecurity #Web3Security #BlockchainAuditing #SecurityAudits #VulnerabilityDetection #FormalVerification #InvariantTesting #SmartContractTesting #FuzzTesting #StaticAnalysis #SymbolicExecution #AccessControlSecurity #IntegerOverflow #ArithmeticVulnerabilities #OracleManipulation #FlashLoanAttacks #FrontRunning #MEV #UpgradeableContracts #SecureSmartContracts #BlockchainCybersecurity

Download the Book (PDF): Introduction There is a particular kind of Saturday night that many university students know well. The pre-game in someone's room, the walk across campus in a loose, loud group, the house or basement where the music is too loud for anyone to hear a full sentence. Faces that were familiar at eleven are gone by midnight. You meet a dozen people and learn one fact about each of them, usually their major or their hometown. By Sunday afternoon you have a handful of new followers, a slightly hazy memory, and a feeling that is hard to name. Something happened. You were around people. And yet you are not sure you know anyone better than you did on Friday. This book is about that feeling, and about what to do with it. Over many years of sitting with students in counseling rooms, residence halls, advising offices and orientation sessions, I have heard the same quiet confession in dozens of different forms. It goes something like this: I go out, I have people to go out with, but I don't really have friends here. Sometimes it comes from a first-year student in October who expected to have found "their people" by now. Sometimes it comes from a third-year student who has been socially busy for two years and has just realised that none of the people in the group chat would notice if they went quiet for a week. The confession is almost always delivered with embarrassment, as if the speaker were admitting to a private defect. It is not a private defect. It is one of the most common experiences in university life, and it has understandable causes. The promise and the gap Universities are sold, in brochures and films and family stories, as the place where you will meet the friends of your life. For many people that turns out to be true. But the popular picture of how it happens is misleading. The picture says that friendship arrives through immersion in a busy social scene: go to the parties, say yes to everything, be seen, and connection will follow. The louder and fuller your social calendar, the more likely you are to be well connected. The research on how friendships actually form points somewhere else. Friendships grow from repeated, relaxed contact with the same people over time, ideally while doing something together that matters to everyone involved. They deepen through conversation that moves gradually from the surface toward what people care about. They survive through maintenance: small acts of attention, repeated for years. None of those conditions are well served by a crowded party. A party offers novelty, stimulation and a sense of belonging to something large. It rarely offers repetition with the same small group, conversation you can hear, or a shared purpose beyond the evening itself. That does not make parties bad, and this is not a book about giving them up. Plenty of students enjoy nights out, and some real friendships begin on a dance floor or a front porch at one in the morning. The argument here is narrower and, I think, more useful. Parties are a poor primary strategy for making friends, and many students who feel lonely are using them as exactly that. They are doing what they were told would work, working hard at it, and wondering what is wrong with them when it does not. What this book argues The controlling idea of this book is simple to state and takes the rest of the pages to unpack. Meaningful friendship on campus is less a matter of personality, luck or charisma than of structure: of choosing to put yourself, again and again, into small recurring settings organised around something you value, and then taking deliberate, modest steps to turn familiar faces into friends. Niche clubs and volunteer work are two of the most reliable structures a campus offers, because they combine the three ingredients that friendship needs: repeated contact, shared activity, and shared values. The word niche matters. A general interest society with four hundred members on its mailing list may be a fine thing to join, but it behaves more like a party than like a friendship engine. The small club that meets every Tuesday to restore old bicycles, play Go, translate poetry, or run a campus radio show is different. It filters for people who care about the same odd thing you do, and it gives you a reason to see them every week whether or not you feel socially energetic that day. Volunteering does something similar with values rather than hobbies. The people who show up at seven on a Saturday morning to sort food bank donations, tutor children, or clear invasive plants from a river bank are telling you something about what they care about before they have said a word. How the book is organised The first chapter asks what friendship actually is, drawing on philosophy and the psychology of close relationships, and explains why a party-centred social life so often produces many acquaintances and few friends. The second chapter deals with a set of misperceptions that keep students stuck, including the widespread belief that everyone else is already settled and happy, and the systematic way people underestimate how much others like them. The third chapter describes the conditions under which friendships reliably form, and shows why structures beat events. The middle of the book is practical. Chapter 4 is about finding and committing to niche clubs, including how to choose, how to survive the awkward first weeks, and how to move from the edge of a group toward its centre. Chapter 5 turns to volunteering as a path to friendship through shared values, with honest attention to its pitfalls. Chapter 6 describes the craft of deepening a connection: how to move from someone you see every week to someone you would call at a hard moment, with scripts and exercises. The last two chapters deal with what gets in the way and what comes after. Chapter 7 addresses social anxiety, identity, money, time, commuting, being an international or first-generation student, and the particular situation of students who do not drink. Chapter 8 is about keeping friends once you have them: maintenance, conflict, drift and change. A short conclusion draws out what follows from all this, both for individual students and for the institutions that shape their social lives. A few words before you begin Throughout the book you will meet students with names like Maya, Daniel and Priya. They are composites, drawn from patterns I have seen many times, and they are there to make ideas concrete. They are not real individuals, and their words are not quotations. Where I cite research, I have tried to cite work that is well established, and to describe it without inflating what it shows. Social science about friendship is a young and imperfect field, and I will say so where it matters. I also want to say something plainly at the outset. Loneliness is painful, and for some people it becomes part of a larger struggle with depression, anxiety or thoughts of self-harm. This book offers practical help with building connection, but it is not a substitute for professional support. If you are in crisis, please reach out now rather than later: in the United States you can call or text 988 to reach the 988 Suicide & Crisis Lifeline, and most campuses have counseling services with same-day or urgent appointments. Services differ by country and by institution, so it is worth finding out today what yours offers. If you or someone else is in immediate danger, contact local emergency services. Finally, a note on expectations. Nothing here will make you a different person, and nothing here needs to. You do not have to become more extroverted, more confident or more interesting to have good friends at university. You need a few places to be, a willingness to keep showing up, and some practical knowledge about how acquaintance becomes friendship. The rest of the book is an attempt to supply the knowledge. The showing up is yours. Chapter 1: What Friendship Actually Is Before we can talk about how to build meaningful friendships, it helps to be clear about what we are trying to build. The word friend is stretched thin in modern life. It covers the person you share a flat with and have never had a real conversation with, the three hundred people who can see your posts, the teammate you have known for a decade, and the one person you would call from a hospital waiting room. When students tell me they have "lots of friends but no real ones", they are pointing to this stretching. They have plenty of the lighter kind and too few of the heavier kind, and the language gives them no easy way to say so. Three kinds of friendship The oldest useful account of the problem is more than two thousand years old. In Books VIII and IX of the Nicomachean Ethics, Aristotle distinguished three kinds of friendship according to what holds them together. In friendships of utility, people are bound by what each is useful for: the classmate you study with before exams, the neighbour who lends you a charger. In friendships of pleasure, people are bound by the enjoyment they share: the person who is fun to go out with, the one who makes every party better. In the third and rarest kind, which Aristotle associated with virtue or character, people are bound by appreciation of who the other person is. Each wishes good things for the other for the other's own sake. Aristotle did not dismiss the first two kinds. He thought them real and valuable. His point was that they are fragile, because they depend on something outside the people themselves. When the exam is over or the partying stops, friendships of utility and pleasure tend to dissolve, not because anyone has behaved badly, but because the thing that held them together is gone. Friendships of character last longer because they are built on something that persists: who each person is and what each cares about. The distinction maps onto campus life with uncomfortable precision. Table 1 sets it out. Table 1. Aristotle's three kinds of friendship on a modern campus. Kind What holds it together Typical campus setting What happens when the setting ends Utility Mutual usefulness Study groups, lab partners, flatmates sharing chores Usually fades quickly Pleasure Shared enjoyment Nights out, parties, gaming sessions Often fades once the fun changes Character Appreciation of who the other person is Emerges over time from repeated shared commitment Tends to survive moves, graduation and change Two things are worth noticing about the table. The first is that the third kind of friendship does not have its own setting. There is no event called "character friendship night". Deep friendships almost always begin as one of the first two kinds and grow into something more. The lab partner becomes a confidant; the person you went out with every Friday becomes the person you call when your parent is ill. The second thing is that some settings make that growth far more likely than others. A setting built around shared values and repeated contact gives a friendship of utility or pleasure room to become something more. A setting built around novelty and noise tends to keep things at the level where they began. It is easy to treat friendship as a pleasant extra, something to sort out once grades and career plans are in order. The evidence suggests that is a mistake. In 1995 the psychologists Roy Baumeister and Mark Leary published a long review in Psychological Bulletin arguing that the need to belong is a fundamental human motivation: that people need frequent, pleasant interactions with a small number of others, set within a framework of lasting mutual concern. Both halves of that definition matter. Frequent pleasant interactions with ever-changing strangers do not satisfy the need, and neither does a stable relationship in which you rarely interact. People need both contact and continuity. A party-centred social life tends to supply the first without the second. The consequences of an unmet need to belong reach well beyond mood. In 2010, Julianne Holt-Lunstad, Timothy Smith and Bradley Layton published a meta-analysis in PLOS Medicine drawing on 148 studies. They found that people with stronger social relationships had a markedly higher likelihood of survival over the follow-up periods studied than people with weaker ones, an effect comparable in size to well-known health risks such as smoking. In 2023, the United States Surgeon General, Vivek Murthy, issued an advisory titled Our Epidemic of Loneliness and Isolation, which drew on this and other research to argue that social disconnection is a public health problem, not merely a private sorrow. For students the stakes are also more immediate. Friends are the people who notice when you stop coming to class, who lend you notes when you are ill, who tell you honestly that the relationship you are in seems to be making you miserable. They are the difference between a setback that is survivable and one that feels like the end of the world. Researchers who study why students leave university, most famously Vincent Tinto, have long argued that social integration, feeling connected to people and to the life of the institution, matters alongside academic performance in whether students persist. A student with two or three solid friends is in a very different position from a student with fifty acquaintances. The arithmetic of closeness If friendship matters that much, why is it so hard to come by? Part of the answer is simple arithmetic. In 2019 the communication researcher Jeffrey Hall published a study in the Journal of Social and Personal Relationships asking a question that sounds almost playful: how many hours does it take to make a friend? Drawing on surveys of adults who had recently moved to a new place and of first-year university students, he estimated that moving from acquaintance to casual friend took somewhere in the region of fifty hours of time together, becoming a friend took roughly ninety hours, and becoming a close friend took more than two hundred. These figures are estimates from self-reports, not precise laws, and the real number varies enormously between people. But the order of magnitude is the point. Close friendship is built from hundreds of hours, not from a few intense evenings. Hall also found that the kind of time mattered. Hours spent together in obligatory settings, such as working side by side, counted for less than hours spent hanging out by choice, joking, catching up and talking about things that mattered. That finding is important for what follows in this book. Proximity alone is not enough. You need proximity that leads to voluntary, relaxed time together. Now consider how a typical party-centred week distributes your hours. Suppose you go out twice a week for four hours each time. You might spend those eight hours with thirty different people, in fragments of five or ten minutes, much of it shouting over music. Even if you are with a stable group, a good share of the time is spent in the logistics of the night: getting ready, finding the place, finding each other, getting home. It is entirely possible to spend a whole semester socially busy and accumulate barely a handful of meaningful hours with any single person. Compare that with a student who spends the same eight hours a week in two recurring settings: a small club that meets for two hours each Tuesday and usually ends with the same six people getting food afterward, and a volunteer shift on Saturday mornings followed by coffee with the two others who worked the same shift. Over a twelve-week term, that student has spent perhaps forty or fifty hours in the company of a small, stable set of people, much of it in conversation. The arithmetic alone is not destiny, but it is not neutral either. There is a second way to think about the arithmetic. The anthropologist and evolutionary psychologist Robin Dunbar, best known for the proposal that humans can maintain stable relationships with roughly 150 people, has argued that these relationships are arranged in layers of increasing size and decreasing closeness. In his book Friends (2021) he describes an innermost layer of around five people, the ones you would turn to in a crisis; a layer of around fifteen that includes those five and adds the people you spend most social time with; then around fifty good friends, and around 150 meaningful relationships. The exact numbers are debated and vary from person to person, but the shape is widely accepted: a small core, and progressively larger and looser circles around it. Dunbar's work adds two points that matter for students. First, the inner layers are expensive. Dunbar has argued that people devote a large share of their social time and effort to the handful of people in their innermost circles. You cannot have fifty best friends, because you do not have the hours. Second, relationships in these layers decay without contact. A friendship that is not fed with time drifts outward, from the inner layer toward the edges, and eventually out of the circles altogether. For a new student this has a liberating implication. You do not need dozens of close friends. You need a handful of people in the inner layers, surrounded by a larger circle of friendly acquaintances. The goal of this book is not to make you popular. Popularity is a property of the outer layers. The goal is to help you fill the inner ones. Why the party so often fails We can now say more precisely why a party-centred social life so often produces the "lots of people, no real friends" experience. Parties are excellent at generating first contacts. They put you in a room with many people you have not met, lower inhibitions, and create a shared sense of occasion. For a student who knows no one, that can be genuinely valuable, and I would not advise anyone to avoid parties on principle. But parties are weak on almost every other condition that friendship requires. The people are different each time, so contact does not repeat. The noise and movement make sustained conversation difficult. The shared activity, being at the party, does not reveal much about what anyone values. Alcohol can make conversation feel intimate while making it hard to remember. And the social currency of parties tends to reward the qualities that make someone fun in a crowd, such as energy, confidence and humour, rather than the qualities that make someone a good friend, such as reliability, attention and honesty. Consider a composite student I will call Daniel. In his first term he went out three or four nights a week with a group formed in the first days of orientation. He was, by any outward measure, socially successful: invited everywhere, recognised across campus. In November his grandmother died. He flew home for the funeral and returned a week later. Two people from the group sent messages; nobody asked him how he was when he came back, because the group's conversations were about plans, not about lives. Daniel was not surrounded by bad people. He was surrounded by friendships of pleasure, which were doing exactly what friendships of pleasure do. What he lacked was anyone for whom the friendship had become about him. What changed things for Daniel, in the second term, was not abandoning his group but adding something else: a weekly shift at a campus tutoring programme for local secondary school students. There he met two other tutors who stayed after each session to plan the next week's work. Planning turned into dinner, dinner turned into long conversations about why each of them had chosen to tutor, and by spring he had people who knew about his grandmother, his worries about his degree and his family's expectations. The party group remained, and he enjoyed it. It simply stopped being his only source of connection. What a good friendship looks like from the inside Definitions are useful, but most students recognise a real friendship less by theory than by how it feels. Across many conversations, the same markers come up again and again, and they are worth naming because they give you something to aim at. The first is ease. With a close friend you do not have to perform. You can be tired, dull, badly dressed, or in a bad mood, and the friendship does not wobble. Many students describe their first real university friendship in exactly these terms: the moment they realised they could sit in someone's room in silence, each doing their own work, and feel entirely comfortable. Ease is not the starting point of friendship; it is one of its products. It grows from enough shared time that neither person is still auditioning. The second is reliability. A friend is someone whose behaviour you can predict in the ways that matter. They turn up when they say they will. They keep what you tell them to themselves. They do not disappear when you are less fun to be around. Reliability is built through small promises kept, which is one reason recurring settings are so powerful: they give people dozens of small opportunities to show that they are dependable. The third is mutual knowledge. A close friend knows your history, your worries, the names of the people who matter to you, and the things you are secretly proud of. You know theirs. This knowledge accumulates slowly, through conversations that move beyond the surface, and Chapter 6 is devoted to how that movement happens. The fourth is honesty with care. Good friends can tell each other difficult things: that a plan seems unwise, that a comment was hurtful, that someone seems to be drinking more than is good for them. They can do this because the relationship is secure enough to absorb it. Friendships of pleasure rarely survive honesty of this kind, because the relationship's purpose is to keep things pleasant. It is also worth saying what a good friendship is not. It is not a relationship in which one person does all the reaching out, all the listening, or all the accommodating. It is not one in which you feel anxious about being dropped, or in which affection is given and withdrawn as a form of control. Some students, eager for connection, stay in friendships that leave them diminished because any friendship seems better than none. Part of building meaningful friendships is learning to notice when a relationship is taking more than it gives, and having enough other connections that you are free to step back from it. Taking this chapter as a whole, it helps to hold three ideas together as you read on. First, friendship comes in kinds and in layers, and the kind most students are missing is the close, character-based friendship of the inner layers. Second, that kind of friendship requires a great deal of time, much of it relaxed and voluntary, spent with the same people. Third, the settings you choose shape whether that time accumulates. The rest of the book is about choosing and using those settings well. A small exercise is worth doing now, before the next chapter. Take a sheet of paper and draw three concentric circles. In the innermost, write the names of anyone, anywhere, you would call in a genuine crisis. In the second, write the people you see or talk with regularly and enjoy. In the outer circle, write people you know by name on campus and would stop to chat with. Do not judge what you find. Many students discover that the outer circle is full and the inner one is sparse or filled only with people from home. That is not a verdict on you. It is a map of where the work needs to happen, and a baseline you can return to at the end of the term. Chapter 2: The Myth of Everyone Else Loneliness at university has a peculiar feature: it is intensified by the belief that you are the only one experiencing it. Walk through a dining hall at lunchtime and you see tables of students laughing together. Scroll through your phone on a Friday night and you see photographs of groups who all appear to have found each other. Sit in a lecture hall in the third week of term and it can seem that everyone around you arrived with a ready-made circle. The conclusion many students draw is that something went wrong for them specifically, that they missed a window everyone else caught. This chapter is about why that conclusion is usually false, and why its falseness matters in practice. The misperceptions described here are not just sources of unhappiness. They change behaviour. A student who believes everyone else is settled is less likely to reach out, less likely to join something new in the second term, and more likely to interpret a lukewarm response as rejection. Correcting the misperceptions is therefore not a matter of feeling better. It is a precondition for acting well. Pluralistic ignorance and the party norm In the early 1990s, the psychologists Deborah Prentice and Dale Miller studied attitudes toward drinking among undergraduates at Princeton University. Their findings, published in 1993 in the Journal of Personality and Social Psychology, have become a classic illustration of what social psychologists call pluralistic ignorance. Students on average reported being less comfortable with the campus drinking culture than they believed the typical student to be. Each individual privately had reservations, but because everyone publicly went along with the norm, each assumed the others genuinely endorsed it. The result was a norm that was widely followed and much less widely liked than it appeared. The broader lesson of pluralistic ignorance is that public behaviour is a poor guide to private experience, especially when the behaviour is expected. If the visible norm on your campus is that social life means going out, then many students will go out whether or not they enjoy it, and each will look at the others and conclude that they, at least, are having a wonderful time. The quiet dissatisfaction that brought you to this book is probably shared by a substantial number of the people standing next to you at the last party you attended. They, too, are looking at you and assuming you are fine. Social media sharpens the problem considerably. A feed is a highly selected record of other people's best moments: the group photo from the night that went well, not the evening spent alone watching videos in bed. Everyone knows this in the abstract, and almost no one is protected by knowing it. When you compare your own ordinary Tuesday with a composite of everyone else's highlights, the comparison is rigged. Consider a composite student, Tomas, who in his first term would check his phone every Saturday night and count the stories from parties he had not been invited to. Each one felt like evidence. When he later got to know several of the people in those photographs, he learned that more than one of them had spent those same nights feeling on the edge of the group they were photographed with. The pictures were accurate records of where people were. They were poor records of how people felt. A practical response is not to abandon social media, which for many students is also a genuine tool for keeping in touch, but to notice when you are using it as a measuring instrument. If checking your feed on a weekend evening reliably leaves you feeling worse, treat that as information about the instrument, not about your life. This has an encouraging practical implication. If you start something quieter, such as a weekly board game evening, a Sunday walk, or a standing invitation to cook dinner together, you will probably find more takers than you expect. The demand for connection that is not organised around a party is real but largely invisible, because the people who want it have each concluded that they are unusual. The social norms approach to campus health, associated with the sociologists H. Wesley Perkins and Alan Berkowitz, grew partly out of this insight. It holds that when students learn how their peers actually behave and feel, rather than how they imagine their peers behave, their own behaviour tends to shift. You can apply the same approach to yourself. Whenever you catch yourself thinking "everyone else is out tonight" or "everyone else has their group", treat it as a hypothesis rather than a fact, and notice how little evidence you have for it. The liking gap and the value of strangers A second misperception concerns what happens after you meet someone. Most people leave a conversation with a new acquaintance and immediately begin to audit it. Did I talk too much? Was that joke strange? They seemed a bit distracted at the end; perhaps they were bored. In 2018, Erica Boothby, Gus Cooney, Gillian Sandstrom and Margaret Clark published a set of studies in Psychological Science documenting what they called the liking gap. After conversations with strangers, people consistently underestimated how much their conversation partners liked them and enjoyed their company. The pattern appeared in brief laboratory conversations, in workshops, and among first-year university students getting to know the people they lived with, where it persisted for months. In other words, the tendency to assume others are less keen on us than they are is not just a first-meeting jitter. It can linger well into a relationship. The liking gap matters because friendship requires someone to take the second step. After a pleasant first conversation, one person has to suggest meeting again, send a message, or sit next to the other the following week. If both people leave the conversation privately convinced that the other was lukewarm, neither takes the step, and a friendship that could have begun does not. Many of the missed friendships of university life die precisely here, in the gap between how much someone liked you and how much you believed they did. A useful habit is to correct for the gap deliberately. When you leave a conversation feeling that it went "fine but probably not great", assume it went somewhat better than that, because the evidence suggests your estimate is biased downward. Then act on the corrected estimate. Send the message. Suggest the coffee. The worst realistic outcome is a polite decline, which costs you very little. The more common outcome is a pleased yes from someone who was also wondering whether you had enjoyed talking to them. A related set of findings concerns our reluctance to talk to people we do not know at all. In 2014, Nicholas Epley and Juliana Schroeder published research in the Journal of Experimental Psychology: General in which commuters on trains and buses were asked either to strike up a conversation with a stranger, to keep to themselves, or to commute as usual. Those who talked to a stranger reported a more pleasant journey, yet when people were asked to predict how such a conversation would go, they expected the opposite. Participants also tended to believe that strangers would not want to talk to them. The researchers titled their paper "Mistakenly seeking solitude", and the phrase captures something about university life too: people avoid small moments of connection because they misjudge how those moments will feel, and how they will be received. Gillian Sandstrom and Elizabeth Dunn, writing in Personality and Social Psychology Bulletin in 2014, looked at the value of what sociologists call weak ties: the acquaintances you see often but do not know well. In one of their studies, students reported more positive feelings, and a greater sense of belonging, on days when they had more interactions than usual with classmates, not only with close friends. The sociologist Mark Granovetter had famously argued in 1973 that weak ties are valuable because they connect us to information and opportunities beyond our immediate circle. Sandstrom and Dunn's work suggests they also matter for how we feel day to day. This is important for two reasons. First, it means that small interactions are not trivial. The brief chat with the person who always sits near you in the library, the joke with the barista, the nod to someone from your club on the path: these contribute to wellbeing in their own right. Second, weak ties are the raw material of close friendship. Almost every close friend you will ever have was once a weak tie. The sociologist Mario Small, in his book Someone to Talk To (2017), found that the graduate students he studied often confided important personal matters in people they did not consider close, because those people happened to be present and seemed understanding at the right moment. Weak ties are where the future inner circle is drawn from. The student who believes she has "no friends" often has a surprising number of weak ties that she has never thought of as connections, because she has been measuring her social life only against the image of the tight friendship group. Recognising those ties is the first step toward developing some of them. There is one more reason these misperceptions are so sticky, and it has to do with loneliness itself. The social neuroscientist John Cacioppo, who spent much of his career studying loneliness and wrote about it for general readers with William Patrick in Loneliness (2008), argued that feeling isolated tends to make people more alert to social threat. A lonely person is more likely to notice the ambiguous glance, to read a delayed reply as a slight, and to approach new encounters defensively. From an evolutionary point of view this vigilance may once have been protective. On a modern campus it tends to be self-defeating, because the guardedness it produces can make the lonely person seem less approachable, which in turn confirms their fears. Knowing that loneliness distorts perception in this way does not switch the distortion off, but it gives you a reason to distrust your first reading of social situations when you are feeling most alone, and to test it against evidence before acting on it. Transition, timing and the second term A final misperception concerns timing. Many students arrive believing there is a narrow window at the start of university, perhaps the first few weeks, during which friendship groups form and then lock. If you have not found your people by the end of orientation, the story goes, you have missed your chance. There is a grain of truth here. The first weeks of university are unusual. Almost everyone is new, almost everyone is looking, and social openness is higher than it will be later. Groups do form quickly, often around proximity: the same floor, the same orientation group, the same seminar. But those early groups are also notoriously unstable. They form on the basis of who happened to be nearby, not on shared values or interests, and many of them thin out or dissolve over the first year as people discover what they actually care about. The friendships that last are often not the first ones. Consider a composite student, Maya, who spent her first term attached to the group from her residence floor. By December she felt increasingly out of place: the group's social life revolved around going out several nights a week, and she found herself pretending to enjoy it. In January she assumed it was too late to start again. Everyone, she thought, had their people now. With some encouragement she joined a small student choir that took new members at the start of each term. She discovered that roughly a third of the other new members were in the same position she was: second-term students who had realised their first groups were not the right fit. The second term, far from being too late, turned out to be a second window, quieter than the first but populated by people who were now choosing more deliberately. This pattern repeats at later points too. The start of each academic year, the move into a new residence, the beginning of a new module or a new volunteer placement, returning from a period of study abroad: all are moments when many people are simultaneously open to new connections. It is worth knowing when those moments are on your own campus, and planning to use them. Research on the transition to university also offers some reassurance. Robin Dunbar, working with Sam Roberts, followed young people through the move from school to university and found that friendships with people back home tended to weaken in emotional closeness unless actively maintained, while new friendships formed over time at university. The transition, in other words, is a process that unfolds across months, not a single moment. Feeling adrift in the first term is a normal stage of that process, not evidence that it has failed. Holding the misperceptions at arm's length It is one thing to know about these biases and another to stop being governed by them. Knowledge helps, but it helps most when it is turned into habits. Four are particularly useful. First, name the story. When you notice a thought like "everyone else has their people", say to yourself, silently or aloud, "that is the everyone-else story". Naming a pattern of thought creates a small distance from it, which is a technique common to several evidence-based therapies. Second, look for disconfirming evidence. Who, among the people you know, might also be looking for more connection? Who has mentioned being homesick, or not loving their flatmates, or wishing there were more to do on weekends? You will usually find more than you expect. Third, lower the stakes of reaching out. Much of the fear behind the liking gap comes from treating every invitation as a test of whether the other person likes you. Try instead treating invitations as offers: small, low-cost, and easy to decline. "A few of us are getting food after the meeting if you want to come" is an offer. It does not demand a verdict. Fourth, count your weak ties. For one week, keep a simple note of every person on campus with whom you exchange more than a greeting. At the end of the week, look at the list and circle two or three people you would like to know better. You now have a starting point for the work described in the chapters that follow. Fifth, run a small experiment rather than waiting to feel confident. Choose one interaction this week that you would normally avoid: asking the person next to you in a seminar what they thought of the reading, staying five minutes after a club meeting instead of leaving straight away, or replying to a group message with a suggestion rather than a reaction. Before you do it, write down what you predict will happen and how you expect to feel. Afterward, write down what actually happened. Most students who try this find that their predictions were gloomier than the outcome, often considerably so. A handful of these small experiments teaches the lesson of the research more convincingly than any amount of reading about it, because the evidence is your own. None of this will make the feeling of being left out disappear overnight. But it will loosen its grip enough for you to act, and action, as the next chapter argues, is what builds friendship. The conditions under which friendships form are well understood, and most of them are within your control. Chapter 3: The Conditions That Make Friends If you ask people how they met their closest friends, the stories often sound like accidents. We were put in the same seminar group. We lived across the hall. We both got stuck with the worst shift at the summer job. There is an element of chance in every friendship story, but the chance is not random. Look closely and the accidents share a common structure. The people involved kept running into each other, they were doing something together, and at some point the circumstances let them relax enough to talk about more than the task at hand. This chapter is about that structure. Understanding it is the key to the whole book, because once you see what conditions produce friendship, you can deliberately seek them out rather than hoping to stumble into them. The central claim is that friendship grows best in structures, meaning recurring settings with the same people, rather than in events, meaning one-off occasions with a changing cast. Proximity and repetition Some of the earliest research on friendship formation was conducted in student housing. In the late 1940s, Leon Festinger, Stanley Schachter and Kurt Back studied a housing complex for married students at the Massachusetts Institute of Technology, known as Westgate. Residents had been assigned to their apartments rather than choosing them, which made the complex something like a natural experiment. The researchers found, in their 1950 book Social Pressures in Informal Groups, that friendships followed the physical layout of the buildings to a striking degree. People were far more likely to become friends with those who lived next door than with those a few doors away, and residents whose apartments were near staircases or mailboxes, where others passed by frequently, tended to have more friends than those in more isolated positions. The Westgate findings have been echoed many times since. The general principle is that physical and functional proximity, meaning how often your paths actually cross, is one of the strongest predictors of who becomes friends with whom. It is not glamorous. It suggests that the best predictor of your future friends may be the route you walk to breakfast. But it is also empowering, because proximity is something you can arrange. Part of the reason proximity works is what psychologists call the mere exposure effect. In a set of studies published in 1968, Robert Zajonc showed that people tend to develop a more positive attitude toward things, including faces, simply through repeated exposure to them. Familiarity, in modest doses, breeds liking rather than contempt. The person you see every Tuesday becomes, over a few weeks, someone you feel faintly warm toward, even if you have barely spoken. That warmth lowers the threshold for conversation, and conversation, repeated, begins to build something more. Repetition does a second kind of work as well. It removes the need for anyone to take a risky initiative. In a one-off setting, if you want to see someone again you have to ask, and asking exposes you to the possibility of a no. In a recurring setting you will see them again anyway. The friendship can grow gradually, a few minutes more each week, without anyone having to make a declaration. For people who find initiating contact difficult, which is most people, this is an enormous advantage. Sociologists who study friendship, such as Rebecca Adams, have often summarised the conditions for making close friends in three parts: proximity, repeated and unplanned interactions, and a setting that encourages people to let their guard down and confide in each other. The phrase "unplanned" matters. Friendships grow most easily where contact happens without anyone having to organise it, because organising contact is effortful and slightly exposing. A shared corridor, a weekly club, a regular volunteer shift all provide contact that simply happens. The coffee afterward, the walk back to residence, the conversation while packing up: these are where people start to let their guard down. Shared activity and shared purpose Proximity and repetition bring people together. Shared activity gives them something to do while they are together, and shared purpose gives them a reason to care about each other's contribution. The importance of shared activity is easy to underestimate, especially for people who assume that friendship consists mainly of talking. Many people, and perhaps especially many men, report that their friendships are built as much side by side as face to face: playing a sport, fixing something, working on a project. Activity takes the pressure off conversation. You do not have to fill every silence, because the task fills it. Conversation arises naturally from the activity itself, and it can drift toward personal matters gradually, at whatever pace feels comfortable. A student who would find a one-to-one coffee with a near-stranger excruciating may find it easy to talk for an hour while painting scenery for a student theatre production. Shared purpose adds something more. When people work toward a common goal, they have a reason to cooperate, to rely on each other and to notice each other's efforts. The social psychologist Muzafer Sherif's famous Robbers Cave study in the 1950s, in which groups of boys at a summer camp were first set against each other and then brought together to solve problems that required cooperation, is usually cited for what it showed about intergroup conflict. But it is also a vivid illustration of the power of what Sherif called superordinate goals, meaning goals that neither group could achieve alone. Hostility eased not through contact alone but through working together on something that mattered to everyone. The study had real methodological problems and should not be taken as a precise model of anything, but the broader idea, that shared goals draw people together, is well supported by everyday experience. On campus, shared purpose shows up whenever a group is trying to accomplish something: putting on a play, winning a debate tournament, running a community garden, producing a magazine, raising money for a cause, tutoring children who need help. In each case members see each other at their best and at their most stressed, they depend on each other, and they share the satisfaction when something goes well. Those experiences are exactly the raw material that friendships of character are built from. They reveal what people value, how they behave under pressure, and whether they can be relied upon. The educational researcher Alexander Astin, in his influential theory of student involvement published in 1984, argued that students learn and develop in proportion to the physical and psychological energy they invest in their university experience. Involvement, in his account, was not simply a matter of joining things but of investing in them. The same principle applies to friendship. It is not membership of a group that builds friendships, but participation in its shared work. Structures versus events We can now compare the settings available to a typical student. Table 2 sets out how several common campus settings perform against the conditions described so far. The judgements in it are general tendencies, not rules; a particular party can be intimate and a particular club can be cold. But the tendencies are strong enough to guide choices. Table 2. How common campus settings support the conditions for friendship. Setting Repeated contact with same people Room for real conversation Shared activity or purpose Signals shared values Large party Low Low Low Low Large lecture Moderate Low Low Low Residence floor High Moderate Low Low Small seminar or lab group High Moderate Moderate Low Niche club High High High Moderate to high Regular volunteering High High High High The pattern in the table is the practical heart of this book. Settings that score well across all four columns are the ones most likely to produce close friendships, and the two that stand out are niche clubs and regular volunteering. Both combine repetition, conversation, activity and some signal of what members care about. The residence floor is excellent for repetition but provides little shared purpose, which is why floor friendships are common in the first term and often thin out later. The large party provides very little of anything except initial contact. The distinction between structures and events deserves emphasis because campus social life is heavily tilted toward events. Welcome week is a succession of events. Most of the social media accounts you follow advertise events. The unspoken assumption is that you make friends by attending as many events as possible. But events are best understood as entry points. Their job is to put you in contact with a structure: the club fair that leads you to a club, the volunteer information session that leads you to a regular placement, the one-off open rehearsal that leads you to the choir. If you attend events and never commit to a structure, you will meet many people once and few people often. A composite student, Priya, illustrates the difference. In her first term she attended around fifteen different events: welcome parties, society taster sessions, a hiking trip, a quiz night. She enjoyed most of them and met dozens of people. By the end of term she had exchanged numbers with many and was close to none. In her second term she made a different choice. She picked two of the settings she had sampled, a small astronomy society that met every Thursday and a Saturday morning shift at a community kitchen, and committed to attending every week regardless of how she felt. By the end of that term she had a group of five or six people she saw regularly and two she considered close friends. Her personality had not changed, and neither had her social skills. What changed was the structure of her week. Designing your own week If friendship grows best in structures, then the practical question becomes: what structures does your week contain, and what structures could it contain? This is a design problem, and it is worth approaching as one. Start by mapping your current week. Take a typical week and list every setting in which you regularly encounter the same people: classes, labs, your residence, your job, a sports team, a religious community, a club. For each, note how many hours you spend there, whether there is room for conversation, and whether there is any shared purpose beyond being present. Many students find that their week contains several high-repetition settings, such as classes and residence, that offer little conversation or purpose, and few or none that combine all the ingredients. A typical map might show fifteen hours of lectures, ten hours in a residence where people keep their doors closed, a part-time job with rotating shifts, and a vague intention to "go out more". There is nothing wrong with any of those, but none of them is doing the work of friendship. Seeing that clearly on paper is often the moment a student stops blaming themselves and starts redesigning. Next, identify where you might add one or two new structures. Two is usually the right number for a first step. One is fragile, because if that single setting turns out to be a poor fit you are back where you started. Three or more tends to spread your time too thinly for any one setting to build real relationships, and can crowd out academic work and rest. Two structures, chosen carefully and attended consistently, can transform a student's social life within a term. When choosing, favour settings with these features: • They meet at a regular time, ideally weekly, so that attendance becomes a habit rather than a decision. • They are small enough that you will see the same faces each time, usually somewhere between six and thirty regular members. • They involve doing something, not just being present. • They connect to something you genuinely care about or are curious about, so that you would want to go even if you made no friends there. • They have a natural moment for informal time before or after the main activity, such as a meal, a walk or a drink. That last feature is easy to overlook and surprisingly important. Many groups have an informal tradition, such as the handful of members who always go for food after the meeting, and that tradition is often where friendships actually form. When you are assessing a new group, notice whether such a tradition exists. If it does, make a point of joining it. If it does not, you may be able to start one. Some students feel uneasy about all this. Designing your week in order to make friends can sound calculating, as if you were treating other people as a project. The worry deserves a direct answer. Choosing where to spend your time is not manipulation; it is the ordinary condition of adult life. Every friendship you have ever had began because two people happened to be in the same place, and until now that place was chosen for you by your school, your neighbourhood or your parents. At university, for the first time, many of those choices are yours. Making them deliberately does not make the resulting friendships less genuine. What would make them less genuine is joining a group you do not care about purely to collect contacts, which is one reason the guidance above emphasises choosing settings you would value even if you made no friends there. The intention to connect is not the problem. Pretending to an interest you do not have is. A related question concerns online communities. Many students find real connection in group chats, gaming servers and online forums organised around shared interests, and for some, particularly those who are isolated by distance, disability or social anxiety, these spaces are a lifeline. They can supply repetition and shared interest very well. What they often supply less well is the relaxed, embodied time together that seems to turn familiarity into closeness, and the small unplanned encounters on the way in and out. The most useful approach is usually to treat online communities as a bridge rather than a destination: a place to find people who share your interests, followed wherever possible by some in-person contact. The campus Discord server for a particular game, for example, may lead you to a weekly in-person session in a common room. The combination tends to be stronger than either alone. Finally, protect the structures once you have chosen them. The biggest threat to a new structure is the week you do not feel like going. You are tired, there is an assignment due, the weather is terrible, you are not sure anyone will notice if you skip. Those weeks will come, and what you do in them determines whether the structure works. Friendships in recurring settings are built by the people who keep showing up, and people notice who keeps showing up. A useful rule is to commit in advance to attending a new setting at least six times before deciding whether it is right for you. That is long enough for the mere exposure effect to begin working, for faces to become familiar, and for the first small conversations to happen. Judging a group on the basis of a single meeting, when you know no one and everyone else seems to know each other, is judging it at the worst possible moment. If it helps, tell one person in the group early on that you are new and planning to keep coming; members often take particular care to welcome someone who has said so. The next two chapters look closely at the two structures that tend to work best: niche clubs, organised around shared interests, and volunteering, organised around shared values. Each has its own opportunities and its own pitfalls, and each rewards a slightly different approach. Chapter 4: Finding Your Niche Every autumn, on campuses around the world, a large hall or lawn fills with tables. Behind each table stand a few enthusiastic students with a banner, a sign-up sheet and a bowl of sweets. Some tables represent enormous organisations with hundreds of members; others represent a dozen people who share an unusual passion. New students drift from table to table, writing their email addresses on a dozen lists, collecting leaflets and free pens. Within a month, most of those emails will go unread and most of those clubs will never see the new students again. The club fair is one of the great missed opportunities of university life. It is a catalogue of structures, each capable of producing exactly the kind of recurring, purposeful contact that friendship needs. Yet most students treat it as an event rather than as a menu, sampling everything and committing to nothing. This chapter is about using clubs well: choosing the right ones, getting through the awkward early weeks, and moving from the edge of a group toward its centre. Why niche beats general The word "niche" in this book's argument is deliberate. A small, specific club tends to be a better engine of friendship than a large, general one, for three reasons. The first is size. In a society with four hundred members, the people you meet at one meeting may not be there at the next. The group behaves more like a series of events than a structure. In a club of fifteen or twenty-five regular members, you see the same faces every week, and repetition does its work. Small groups also make it harder to hide, which is uncomfortable at first but ultimately helpful. People notice when you come and when you do not. They learn your name because there are only so many names to learn. The second is filtering. A specific interest acts as a filter that pre-selects people with something in common with you. If you join a general social society, the only thing you are guaranteed to share with other members is a desire to socialise. If you join a club devoted to film photography, medieval re-enactment, competitive programming, urban beekeeping or Portuguese conversation, you share at least one real enthusiasm with everyone in the room, and often more than one. Unusual interests tend to cluster with other characteristics: a certain kind of curiosity, patience, sense of humour or set of values. The niche club does part of the work of finding compatible people before you arrive. The third is conversation. Shared niche interests give you something to talk about that is neither small talk nor intrusive. You do not have to ask a near-stranger about their family or their feelings; you can ask what they thought of the new release, how they got into the hobby, or whether they have tried the technique someone mentioned last week. These conversations are low-risk and genuinely interesting, and they reveal a surprising amount about the people having them. From "how did you get into this?" it is a short step to stories about childhood, family, turning points and hopes, the material from which closer friendships are made. None of this means large organisations are useless. Many large societies contain smaller working groups, committees, teams or regular sub-events, and joining one of those can give you the benefits of a niche club within a larger body. A big student newspaper, for example, may be run day to day by a small section team who work together every week. The principle is to look for the small recurring group within whatever organisation you join. Choosing where to belong How do you choose among dozens or hundreds of possible clubs? The answer depends partly on whether you already have interests you want to pursue or are hoping to discover new ones. Both are good reasons to join clubs, and many students do some of each. If you have an existing interest, the choice is straightforward in principle. Find the club that matches it, and if there are several, try each at least twice. The main danger is being too narrow in how you define the interest. A student who loved playing in a school orchestra may find the university orchestra highly competitive and alienating, but may find a relaxed chamber music group, a folk session or a musical theatre pit band much more welcoming. Think about what you actually enjoyed about your old activities. Was it the music itself, the sense of being part of an ensemble, the performances, the rehearsal-room jokes? Different clubs will offer different parts of the experience. If you are hoping to discover something new, university is an exceptional place to do it. Clubs exist for activities you may never have heard of, and most welcome beginners enthusiastically, because small groups need new members to survive. A useful approach is to choose one familiar interest and one genuinely new one. The familiar interest gives you a setting where you feel competent from the start; the new one gives you the pleasure of learning and the particular bond that forms among beginners struggling together. Whatever your starting point, a few questions can help you judge a club once you have attended a meeting or two: • Does it meet regularly, ideally weekly, at a time you can reliably attend? • Is the core group small enough that you would get to know people? • Do members actually do something together, or mainly sit and listen? • Is there an informal tradition, such as food or a drink afterward, that newcomers are welcome to join? • Did anyone speak to you, learn your name or invite you back? • Would you go even if you made no friends there, simply for the activity itself? That last question is the most important. A club you join purely as a friendship strategy, with no real interest in its activity, is unlikely to work. You will find reasons to skip it, and the people there will sense your lack of engagement. A club you would enjoy on its own terms is much more likely to hold you through the awkward early weeks, and the enjoyment itself will make you better company. It is also worth thinking about values, not just interests. Some clubs are organised around interests that carry values with them: an environmental action group, a faith society, a mutual aid network, a debating society that prizes rigorous argument, an outdoor club with an ethic of care for wild places. Joining a group like this lets you meet people who share not only what you enjoy but what you think matters. The next chapter pursues this idea through volunteering, but it applies to clubs as well. A caution about timing. Many students join clubs at the start of the year and then drop them within weeks as academic work intensifies. If you know that the middle of term is always your busiest period, choose clubs that meet at times you can protect, and be realistic about how many you can sustain. Two clubs you attend faithfully are worth far more than six you attend occasionally. From the edge to the centre The first few meetings of any club are hard. You arrive knowing no one; everyone else seems to know each other; the in-jokes are incomprehensible; the conversation at the pub afterward is about people and events you have never heard of. Many students attend once or twice, feel like outsiders, and quietly stop going. This is the single most common way the club route to friendship fails, and it is almost entirely avoidable. The first thing to understand is that the feeling of being an outsider in a new group is normal, temporary and shared. The members who seem so comfortable were newcomers themselves not long ago. Some of the people who appear to know everyone may have joined only a few weeks before you. The feeling of exclusion is usually not the result of anyone excluding you; it is the natural state of being new. The second thing to understand is that moving from the edge of a group to its centre follows a fairly predictable path, and there are ways to speed it up. Consider the following approach, which has helped many students I have worked with. Arrive a few minutes early. The period before a meeting starts, when a few people are setting up chairs or waiting for others, is the easiest time to talk. The group is small, nobody is yet absorbed in the activity, and there is a natural task to help with. "Can I help set up?" is one of the most useful sentences in any new setting. Learn names and use them. In a small group, learning people's names is a clear signal that you intend to stay. If you are poor at remembering names, write them down after each meeting with a detail or two: "Sam, second-year chemist, into analogue cameras; Leila, runs the social media, from Manchester." Using someone's name the following week, and remembering something they told you, has a disproportionate effect. Ask about the group itself. Existing members usually enjoy explaining their club: how long they have been involved, how it started, what the best event was, what they would change. These questions show interest, give you useful information, and position you as someone who is engaging with the group rather than just consuming it. Stay for the informal part. If there is a tradition of food or drinks afterward, go, even when it feels awkward. Much of a club's social life happens here, and it is where the transition from fellow member to friend begins. If you do not drink alcohol, go anyway and order something else; Chapter 7 has more on this. Take a small job. This is perhaps the most powerful step of all. Nearly every club has small tasks that need doing: bringing snacks, booking a room, managing the email list, taking photographs, welcoming newcomers, keeping the equipment in order. Offering to take one of these on, even a modest one, changes your position in the group. You become part of how the club runs rather than someone it runs for. You have reasons to talk to the organisers, you are relied upon, and people start to associate you with the group. A composite student I will call Kofi joined a small campus film society in his first year and felt invisible for the first month. When the organisers mentioned that they needed someone to write the short introduction read aloud before each screening, he volunteered. Within weeks he was part of the planning conversations, knew everyone's name, and had been invited to the organisers' informal gatherings. Nothing about him had changed except his role. Stick to the six-meeting rule described in the previous chapter. Do not decide whether a club is right for you until you have attended at least six times. By then you will know several people by name, have had a few real conversations, and have a much better sense of whether the group suits you. As weeks turn into months, the next step is to take on more responsibility, if the club is one you want to invest in. Many clubs hold elections for committee roles in the spring. Serving on a committee is a significant time commitment, and you should weigh it carefully against academic demands, but it is also one of the most reliable ways to form close friendships at university. Committee members work closely together on shared goals, share the stress of organising events, and spend many hours in each other's company. Those are precisely the conditions that produce friendships of character. One more thing belongs in this section, because moving toward the centre of a group only makes sense if the group is worth belonging to. Not every club is healthy. Some are cliquish in a way that goes beyond ordinary newcomer awkwardness: the same inner circle controls everything, newcomers are tolerated but never included, and the informal culture seems designed to keep people out. Some place heavy pressure on drinking, or expect new members to prove themselves through humiliating or risky rituals. Hazing of that kind is prohibited by most universities and by law in many places, and you should never feel obliged to go through it to belong. Campuses usually have a student activities or student conduct office where concerns can be raised, often confidentially, though it is worth asking exactly how a report will be handled before you make one. More commonly, a club is simply not a good fit: pleasant enough, but not a place where you are ever going to feel at home. Knowing when to leave is part of the skill. If, after six or more meetings, you still feel consistently unwelcome or uncomfortable, it is reasonable to conclude that the problem is the fit, not you, and to invest your time elsewhere. When nothing fits: starting something Sometimes a student looks through every club on offer and finds nothing that fits. Perhaps their interest is genuinely unusual, perhaps the relevant club has collapsed, or perhaps the existing clubs have cultures that feel alienating. In these cases, starting a small group of your own can be a remarkably effective strategy. Starting a group does not have to mean founding a formal society with a constitution and a budget, though most universities make that possible. It can be as simple as a standing weekly event: a Sunday afternoon walk, a Wednesday evening where people bring a dish to share, a reading group that meets in a café every other Thursday, a pick-up game in the park. The key features are regularity and an open invitation. You choose the time and place, announce it in whatever channels reach the people you hope to attract, and then turn up every week, whether two people come or twelve. The consistency matters more than the numbers. In the early weeks, attendance may be small or erratic. That is normal. What makes a standing event work is the knowledge that it is reliably there. People who come once and enjoy it will return, and they will bring others. Over a term, a small, well-run standing event can become a genuine community. And the person who started it has a natural role at its centre, which makes forming friendships within it much easier. Consider a composite student, Hana, a third-year who loved hiking but found the university's outdoor club focused on ambitious expeditions she could not afford. She posted a notice in her department and on a campus forum offering an easy, free walk every Saturday morning, meeting at the main gate at nine. The first week, two people came. The third week, six. By the end of term, a core group of eight or nine walked together most weekends, and several had become close friends. Hana had not needed to find a niche. She had built one. If you do start something, a few practical points help. Choose a time and place that are easy to find and remember. Keep the cost at or near zero, so that money is not a barrier. Write a short, warm description that makes clear beginners are welcome. Learn the names of everyone who comes, and message them before the next session. And do not be discouraged by small numbers early on. Every thriving group was once three people in a room, wondering whether anyone else would come. Whether you join existing clubs or build your own, here is a practical plan that draws together the advice in this chapter. In the first two weeks, sample widely. Attend taster sessions for five or six clubs, including at least one outside your comfort zone. Use the questions above to assess each. By week three, choose two. Pick the two that best combine regular meetings, a small core group, shared activity and genuine interest. Commit to attending each at least six times. In weeks three to eight, move from the edge toward the centre. Arrive early, learn names, stay for the informal part, and look for a small job to take on. Around week eight, review. Ask yourself whether you know people by name, whether anyone is starting to feel like more than an acquaintance, and whether you look forward to going. If a club is not working, replace it with another rather than giving up on clubs altogether. In the second term and beyond, deepen. Consider taking on more responsibility in the clubs that suit you, and start to extend friendships outside the club setting, a step described in detail in Chapter 6. Clubs organise people around shared interests. The next chapter turns to a structure that organises people around shared values, and that brings a set of benefits clubs cannot always offer: volunteering. Chapter 5: Volunteering and the Friendship of Shared Values A club gathers people around what they enjoy. Volunteering gathers them around what they believe. That difference is the reason volunteering deserves a chapter of its own. When you spend a Saturday morning sorting donations at a food bank, reading with children at a local primary school, or answering calls for a student-run listening service, you are among people who have chosen to give their time to something beyond themselves. You may not share their hobbies, their politics or their taste in music. But you know something important about them before you have exchanged a word: they care about the same thing enough to get up and do something about it. Aristotle's highest form of friendship, you will recall from Chapter 1, was built on appreciation of the other person's character. Volunteering is one of the few settings on campus that puts character on display from the first day. You see how people treat those they are serving, how they respond when a shift goes badly, whether they do the unglamorous jobs without being asked. Those observations are the foundations of trust, and trust is the foundation of the kind of friendship this book is about. What volunteering offers There is good evidence that volunteering is associated with wellbeing. In a widely cited 2001 study in the Journal of Health and Social Behavior, the sociologists Peggy Thoits and Lyndi Hewitt examined data from a national survey of American adults and found that volunteer work was associated with gains in happiness, life satisfaction, self-esteem, sense of control over life, and physical health, and with lower levels of depression. They also found that people who were already doing well were more likely to volunteer, so the relationship runs in both directions. Volunteering is not a cure for unhappiness, and people who are struggling seriously should not be told that helping others will fix things. But the association between giving time and feeling better is consistent enough to take seriously, and much of it plausibly runs through the social connections that volunteering creates. For students, volunteering offers several specific advantages as a route to friendship. It provides a structure with built-in repetition. Most volunteer roles involve a regular commitment: a weekly shift, a fortnightly session, a term-long placement. That repetition, as we saw in Chapter 3, is the soil in which friendships grow. It provides a shared task that makes conversation easy. Working side by side on something practical, whether stacking shelves, preparing meals, clearing a footpath or planning a tutoring session, removes the pressure of face-to-face small talk. Conversation grows naturally out of the work. It provides meaning. Many students describe a hollowness in party-centred social life that goes beyond loneliness. They are around people, but the time does not seem to add up to anything. Volunteering supplies purpose, and people who share a sense of purpose tend to feel bonded by it. The conversations that happen during and after a volunteering shift often turn quickly to why people are there, which leads naturally to what they value, where they come from and what they hope for. It mixes people who might not otherwise meet. A volunteering programme often brings together students from different courses, years and backgrounds, and sometimes people from the wider community as well: retired people, local residents, staff. This mixing broadens your social world beyond the narrow band of people who share your course or residence, and it can be a relief to spend time with people for whom university is not the whole of life. And it gets you off campus, at least sometimes. Many volunteer placements take place in the surrounding town or city. For students who feel that campus life is an intense, self-contained bubble, stepping outside it regularly can restore a sense of proportion, and the friendships formed there often have a different, steadier quality. Finally, volunteering suits people who find conventional socialising draining. Quieter students often say that they are at their best when there is a clear role to play and a job to get on with, and at their worst when the only task is to be sociable. A volunteer shift gives everyone a role. Nobody has to wonder what to say or where to stand, and the quieter person who works steadily and notices what needs doing is often more valued than the one who talks the most. For students who have spent months feeling out of step with the loudness of campus life, this can be the first setting where they feel socially competent. Choosing a volunteer role Not all volunteering is equally good at building friendships. Table 3 compares some of the common forms students encounter. As with the earlier comparison of campus settings, these are tendencies rather than rules. Table 3. Common forms of student volunteering and their friendship potential. Form Typical commitment Contact with same people Friendship potential One-off service day A few hours, once Low Low, but a useful entry point Regular community placement Weekly shift for a term or more High High Student-run service programme Weekly sessions plus planning meetings High Very high Advocacy or campaign group Irregular meetings and actions Moderate Moderate to high Skilled or remote volunteering Flexible, often solitary Low Low The pattern is similar to the one in Chapter 3. One-off events are valuable as entry points but rarely produce friendships on their own. The strongest options are regular placements and, especially, programmes run by students themselves, such as tutoring schemes, peer mentoring, student-run food pantries, campus listening lines and environmental projects. These combine regular service with the shared work of running the programme, which means meetings, planning and problem solving with the same small group. That combination is powerful. Some of the closest friendships I have seen form at university grew out of exactly this kind of shared responsibility. When you are choosing a role, the following considerations are worth weighing. Start with a cause you actually care about. As with clubs, the single most important factor is genuine interest. If you care about education, look for tutoring or mentoring. If you care about food insecurity, look for food banks, pantries or community kitchens. If you care about the environment, look for conservation or sustainability projects. If you care about mental health, look for peer support or awareness programmes, bearing in mind that these often require training. The friendships you want are with people who share your values, and you find them by following your values. Look for regularity. A role that involves the same shift each week with the same colleagues is far better for friendship than one where you are assigned to different shifts each time with different people. If a placement offers a choice, ask for a regular slot. Find out whether there is a team. Some volunteer roles are essentially solitary, such as remote transcription or occasional one-to-one visits. These can be valuable and satisfying, but they do not build friendships among volunteers. If friendship is part of what you are seeking, look for roles where you work alongside other volunteers. Check the commitment honestly. Many placements, especially those involving children or vulnerable adults, require training and background checks and ask for a commitment of a term or more. That is appropriate, and it is also good for friendship, since it ensures the same people stay. But be honest with yourself about whether you can sustain it. Dropping out of a placement after a few weeks is unfair to the people you were serving and does little for your friendships. Ask about the social side. Many volunteer programmes, especially student-run ones, have an informal social life: a meal after the shift, an end-of-term gathering, a group chat. Ask about this when you apply. It is a legitimate question and says nothing bad about your motives. Your campus will probably have a volunteering office, a community engagement centre, or a students' union volunteering team. They can match you with placements and often run introductory sessions. Religious and cultural societies frequently organise service projects as well, and local charities often advertise directly to students at the start of term. If a weekly commitment is genuinely impossible, because you work long hours, commute, or have caring responsibilities, volunteering can still play a part. A monthly role with the same team, such as a regular Saturday conservation day, keeps some of the repetition that friendship needs. Some universities also offer service-learning or community engagement courses, in which a volunteer placement forms part of a credit-bearing module. These can be a good option for students whose time is tightly constrained, because the service counts toward the degree rather than competing with it, and the seminar that accompanies the placement puts you in regular contact with other students doing the same work. When you make first contact with a programme, it helps to be straightforward. A short message is enough: "I'm a second-year student interested in volunteering with your programme. I can commit to one regular afternoon a week for at least this term. I'd like to work alongside other volunteers if possible. Could you tell me what the next step is?" This tells the organiser that you are reliable and serious, and it lets you ask about team-based roles without awkwardness. Doing it well, and the pitfalls to avoid Volunteering is not a friendship vending machine, and it carries some particular risks that are worth naming. The first is instrumentalism. Some students approach volunteering mainly as a line on a CV or a route to a medical school application. There is nothing wrong with volunteering helping your career, and many worthwhile programmes are glad of anyone who shows up reliably. But if the career benefit is your only motive, you are likely to treat the placement as a transaction, and both the people you serve and your fellow volunteers will notice. The friendships this book is about grow from shared values, and they are hard to form if you are not really sharing them. The second is what is sometimes called the saviour mentality: the sense that you, as a university student, have arrived to rescue people less fortunate than yourself. This attitude is corrosive to good service and to friendship. The best volunteers approach the people they serve, and the community organisations they work within, with humility and respect, recognising that the people who live and work in a place usually understand its needs better than a newcomer does. Fellow volunteers who have been doing the work for years tend to notice quickly whether a newcomer comes to listen or to lecture. Approach a placement as a learner, follow the lead of experienced staff and volunteers, and ask questions before offering opinions. The third is burnout. Some forms of volunteering are emotionally demanding, particularly those involving crisis support, homelessness, illness or bereavement. Good programmes provide training, supervision and debriefing. If yours does not, be cautious. Take the support that is offered, talk with fellow volunteers about difficult shifts, and step back if you find the work is taking a toll on your own mental health. Your fellow volunteers are often the best people to talk to after a hard shift, and those conversations are frequently where deep friendships begin, because they involve exactly the kind of honest disclosure that builds closeness. The fourth is overcommitment. Volunteering can be so rewarding that students take on more than they can manage, stacking up placements and committee roles until their studies and health suffer. One or two regular commitments are usually enough. The aim is a sustainable rhythm, not a heroic one. Within those limits, the practical advice from Chapter 4 applies here too. Arrive a few minutes early. Learn names, including the names of the people you serve where that is appropriate. Take on small responsibilities once you have settled in: training newcomers, organising the rota, coordinating supplies. Stay for the informal time afterward. And consider, after a term or two, taking on a leadership role in a student-run programme, which brings you into close, regular contact with the others who share that responsibility. There is also a benefit to volunteering that goes beyond your own friendships, and it is worth naming because it may shape which programmes you choose. Volunteering often brings together people who differ from each other in background, faith, politics or class but share a commitment to the same cause. The psychologist Gordon Allport, in The Nature of Prejudice (1954), proposed that contact between members of different groups is most likely to reduce prejudice when those involved have equal status, share common goals, cooperate rather than compete, and have the support of relevant authorities. A well-run volunteering programme meets most of those conditions almost by design. The friendships that form there can cross lines that ordinary campus social life rarely crosses, and those friendships tend to widen what each person understands about the world. On campuses where students increasingly sort themselves into groups of the like-minded, this is no small thing. A composite story Consider a composite student I will call Elena, a second-year in economics who arrived at university from a small town and found the social scene overwhelming. She went out with her flatmates for most of her first year but felt, in her words, like she was always performing. In the autumn of her second year, a notice in the library advertised a student-run programme that ran weekly homework clubs for children at a nearby community centre. She had tutored younger children at home and missed it. The programme had about twenty volunteers, each committed to one weekday afternoon a week, with a planning meeting every other Sunday. Elena was assigned to Wednesdays, alongside four other students and a retired teacher who had been volunteering at the centre for years. For the first few weeks she spoke to her fellow volunteers mostly about the children: who was struggling with fractions, who needed encouragement, who was disruptive and why. Walking back to campus together afterward, the conversations began to widen. One volunteer, a physics student, had grown up in a neighbourhood like the one the centre served and talked about what the homework club might have meant to him. Another was thinking about leaving her course and training as a teacher. Elena found herself talking about her own family, her doubts about economics, and her homesickness in a way she had not with her flatmates in a whole year. By spring, the Wednesday group had started meeting for dinner after the planning meetings. Two of them became what Elena described as her real friends at university: people who knew what was going on in her life, whom she could call when things were difficult, and who she was confident would still be in her life after graduation. Her flatmates remained people she liked and went out with. But the friendships that mattered most to her had grown out of shared work that mattered, done week after week with the same small group. Elena's story is not unusual. It illustrates the basic pattern of this book: a recurring structure, shared activity, shared values and gradual deepening. Volunteering happens to supply all four of these in a particularly concentrated form. The pattern also shows something else. Elena did not set out to find friends. She set out to do something she cared about, and friendship followed as a consequence. This is worth remembering if you feel self-conscious about seeking friendship too deliberately. The best approach is usually to choose a structure for its own sake and then pay attention to the people you meet there. The next chapter is about how to pay that attention well, and how to turn the familiar faces of a club or a volunteer shift into friends. Chapter 6: From Familiar Face to Friend By now you may have one or two structures in your week: a club you attend faithfully, a volunteer shift you would not miss. You recognise people. You know their names and a few facts about them. Some of them seem to like you, and you like some of them. And yet there is a gap between this and the friendship you are hoping for. You see these people every week in one setting and nowhere else. You talk easily about the club or the work, but not much about anything else. You would not quite call any of them if something went wrong. This gap is normal, and crossing it is a skill. Some people cross it without thinking, because they learned the moves early or because their temperament makes it easy. Many do not, and they can stay for years in the pleasant but limited territory of friendly acquaintance. This chapter describes how closeness actually grows, and offers practical ways to help it along. How closeness grows Two ideas from the psychology of relationships are particularly useful here. The first is social penetration theory, proposed by the psychologists Irwin Altman and Dalmas Taylor in the early 1970s. Their image was of personality as an onion, with many layers. On the outside are the things anyone can know about you: your course, your hometown, your taste in films. Further in are your opinions, your hopes and worries, your family relationships, and at the centre your deepest fears and most private sense of yourself. Relationships deepen, in their account, as people gradually reveal more of these inner layers to each other, and as the range of topics they talk about widens. The process tends to be gradual and reciprocal. When one person reveals a little more, the other usually responds in kind. The second idea is the intimacy process model developed by Harry Reis and Phillip Shaver in 1988. Their insight was that disclosure alone does not create closeness. What matters is how the disclosure is received. When one person reveals something personal and the other responds in a way that makes them feel understood, validated and cared for, intimacy grows. When the response is dismissive, distracted or judgmental, it does not, and the person who disclosed learns to be more guarded next time. Closeness, in this model, is built through repeated cycles of disclosure and responsiveness. These ideas received a memorable demonstration in a 1997 study by Arthur Aron and colleagues, published in Personality and Social Psychology Bulletin. Pairs of strangers spent about forty-five minutes taking turns to answer a series of questions that became gradually more personal, moving from questions such as whom they would choose as a dinner guest toward questions about their relationships with their families and their most treasured memories. Pairs who worked through these escalating questions reported feeling considerably closer afterward than pairs who spent the same time on small talk. The study is sometimes presented in popular culture as a formula for falling in love, which overstates it. Its real lesson is more modest and more useful: sustained, reciprocal, gradually deepening conversation generates closeness, even between people who have just met. Put together, these ideas explain why friendly acquaintances often stay stuck. In most club and volunteer settings, conversation stays on the outer layers, because the setting provides an obvious shared topic and there is little time or privacy for anything else. Nobody reveals much, so nobody has the chance to respond warmly, so the cycle never starts. The task is to create occasions where the conversation can go a little deeper, and then to respond well when it does. Extending the connection beyond the setting Most friendships that begin in a club or volunteer role grow by gradually extending beyond it. The movement usually follows a recognisable sequence, which Table 4 sets out. It is not a rigid ladder, and friendships skip and repeat steps all the time, but it helps to see where a given relationship stands and what a natural next step might be. Table 4. Stages of moving from acquaintance to friend, with a small next step for each. Stage What it looks like A natural next step Familiar face You recognise each other and exchange greetings Learn their name and one thing about them Setting friend You chat easily, but only within the club or shift Join or suggest the informal time afterward Group friend You socialise together as part of a small group Suggest something one-to-one or in a pair Emerging friend You meet one-to-one, and talk about more than the setting Share something personal, and ask about their life Close friend You rely on each other and know each other's lives Keep investing time, and show up in hard moments The most difficult transitions are usually the second and third: from the club to the informal time afterward, and from the group to something one-to-one. Both involve a small risk, because they require someone to issue an invitation, and invitations can be declined. This is where the liking gap described in Chapter 2 does its damage. Both people would welcome the next step; neither takes it, because each assumes the other is less keen. The practical answer is to become the kind of person who issues small, easy invitations. The key word is small. An invitation does not need to be a declaration of friendship. It needs only to offer a specific, low-stakes opportunity to spend a little more time together. Some examples: • "A few of us are going to get food after this. Want to come?" • "I'm going to the talk on Thursday that the club mentioned. Do you want to go together?" • "I'm heading back toward the library, if you're going that way." • "I've got a spare ticket for the concert on Friday. Would you be up for it?" • "I'm cooking a big pot of soup on Sunday. Do you want to come over and eat some?" Notice what these have in common. They are specific, with a time and an activity, rather than vague ("we should hang out sometime"). They are easy to say yes to. They are also easy to decline without awkwardness, which makes them less risky for both people. And they often involve an activity, which relieves the pressure of pure conversation. Vague invitations are the enemy of new friendships. "We should get coffee sometime" is a pleasant thing to say and almost never leads to coffee, because nobody owns the next step. If you catch yourself saying it, follow it immediately with a specific proposal: "How about Thursday after the meeting?" If someone says it to you, do the same. The person who converts vague goodwill into a specific plan is doing the essential work of friendship formation, and most people are quietly grateful when someone else takes it on. It helps to know that a declined invitation almost always means less than it feels like. People decline for many reasons: an essay due, a prior commitment, fatigue, anxiety of their own. A single no tells you very little. A useful rule is to try at least twice more, on different occasions, before concluding that someone is not interested. If they decline but suggest an alternative, that is a strong signal of interest. If they decline three times without suggesting anything else, it is reasonable to let that particular connection rest and focus your energy elsewhere, without taking it as a verdict on your worth. The one-to-one step deserves special attention. Groups are comfortable, and many students are content to stay at the group-friend stage for a long time. But close friendships usually require some one-to-one time, because it is in one-to-one conversation that people are most willing to share the inner layers of their lives. Moving a group friend into one-to-one time can be done gently. Walking back from the meeting together, arriving at a group event a few minutes early and chatting with whoever is there, or inviting one person to something that suits their particular interests are all natural ways to create pairs within a group without making anyone feel singled out. The craft of good conversation Invitations create occasions. What happens within those occasions determines whether closeness grows. Here the research offers some encouragingly simple guidance. Ask questions, and especially follow-up questions. In a 2017 paper in the Journal of Personality and Social Psychology, Karen Huang, Alison Wood Brooks and colleagues found that people who asked more questions in conversation, and especially more follow-up questions that built on what their partner had just said, were better liked by their conversation partners. Follow-up questions signal that you have been listening and want to know more. They are also a gentle way of inviting someone to go a layer deeper. If someone mentions that they nearly did a different degree, asking "What made you choose this one in the end?" opens a door they can walk through or not, as they choose. Respond to disclosures warmly. When someone tells you something personal, even something small, the way you respond matters more than anything clever you might say. Reis and Shaver's model suggests that people feel close when they feel understood, validated and cared for. In practice, this usually means reflecting back what you have heard ("That sounds like a lot to deal with"), asking a gentle follow-up question, and resisting the urge to immediately change the subject to yourself or offer advice. Advice has its place, but it is often not what people want in the first moment of disclosure. Share in return, at a matching depth. Social penetration theory emphasises reciprocity. If someone tells you they are worried about their exams, you might say that you have been feeling the same pressure. If they talk about missing home, you might share something about your own homesickness. Matching depth matters: responding to a light disclosure with a heavy one can overwhelm, while responding to a heavy disclosure with a joke can wound. The aim is to keep pace with each other, each going a little further as trust grows. Celebrate good news well. Friendship is not only about difficulties. In 2004, Shelly Gable and colleagues published research in the Journal of Personality and Social Psychology on what happens when people share good news with others. They found that the way the listener responds matters a great deal. Responses that are enthusiastic and engaged, which they called active-constructive responses, were associated with stronger relationships, while responses that are muted, distracted or deflating were associated with weaker ones. When a friend tells you they got the internship, passed the exam or finally finished the project, show genuine interest: ask how they found out, what it means to them, how they plan to celebrate. It is a small act with a large effect. Remember what people tell you. A great deal of friendship rests on continuity: remembering that someone had a job interview on Tuesday and asking how it went, remembering their sister's name, remembering that they were dreading the chemistry lab. People feel known when their lives are remembered. If your memory is unreliable, there is no shame in keeping brief notes, and a quick message such as "How did the interview go?" on the right day can do more for a friendship than an entire evening out. If conversation beyond the surface does not come naturally to you, it can help to have a few questions ready that invite a slightly deeper answer without prying. These work best in relaxed settings, such as a walk back from a meeting or a meal afterward, and they should be asked out of real curiosity rather than recited. Some that students have found useful: • "How did you end up getting into this?" (about the club, the cause, or their course) • "What's been the best part of this term for you, and the hardest?" • "Is university what you expected it to be?" • "What do you miss about home, if anything?" • "If you weren't studying this, what do you think you'd be doing?" • "What's something you're looking forward to at the moment?" Each of these can be answered lightly or deeply, which is what makes them good. They leave the other person in control of how much to share. If the answer is brief, you have lost nothing and can move on. If the answer opens up, follow it with genuine questions and, when the moment is right, offer something of your own in return. A small exercise can make these ideas concrete. Choose one person from your club or volunteer setting whom you would like to know better. Over the next three weeks, set yourself three modest goals: first, to have one conversation with them about something other than the shared activity; second, to issue one specific invitation to spend time together outside the usual setting; third, to remember one thing they tell you and ask about it later. Notice how each step feels, and what happens as a result. Most students who try this find that at least one of the three steps is warmly received, and that the relationship feels noticeably different by the end. Pacing, reciprocity and patience A few final principles help keep the process of deepening healthy. Go at a pace that matches the other person. Some people open up quickly; others take months. Pressing someone to disclose before they are ready, or disclosing a great deal very early yourself, can feel intrusive. Watch for signs of comfort: whether someone returns your invitations, whether they share things voluntarily, whether they seem relaxed with you. Let those signs guide how quickly you move. Watch the balance of initiative. In a developing friendship, both people should eventually take turns reaching out. Early on, it is natural and fine for one person to do more of the initiating, especially if the other is shy or busy. But if, after several months, you are always the one making plans and they never do, it is worth noticing. It may mean they value the friendship less than you do, or it may mean they are simply someone who rarely initiates. A gentle, honest conversation can help: "I always enjoy our time together. I've noticed I'm usually the one who suggests things. I'd love it if you felt free to as well." Accept that some connections will not deepen. Not every familiar face will become a friend, and that is fine. Some people are already fully occupied with existing friendships. Some simply do not click with you, or you with them, despite goodwill on both sides. Hall's research suggests that closeness takes hundreds of hours, and nobody has hundreds of hours for everyone. Part of the art is to invest in several connections at once, notice which ones seem to be growing, and give those more time. Consider a composite student I will call Marcus, who had been attending a weekly climbing club for a term. He knew everyone by name and chatted easily at the wall, but had no friendships beyond it. He decided to try the approach described in this chapter. He started staying for the pizza some members ate afterward. After a few weeks, he noticed that one member, Ade, was taking the same statistics course, and suggested they work through a problem set together. Over the next month, the problem sets became a weekly habit; conversation drifted from statistics to their families, their plans, Ade's worries about money and Marcus's uncertainty about his degree. When Marcus's relationship ended in the spring, Ade was the first person he told. None of the individual steps had been dramatic. Together, over a few months, they had turned a climbing acquaintance into a friend. Marcus's story captures the core of this chapter. Closeness is not a mysterious chemistry that either happens or does not. It is built through a sequence of small, reciprocal steps: an invitation, a conversation that goes a little deeper than before, a warm response to a disclosure, a remembered detail, another invitation. Most people can learn to take those steps. The next chapter considers what makes them harder for some students than others, and what can help. Chapter 7: What Gets in the Way The advice in the previous chapters is simple to describe: choose a few recurring settings organised around things you care about, show up faithfully, and take small steps to deepen the connections you find there. For many students, following it is also fairly simple, if not always comfortable. For others, real obstacles stand in the way. Anxiety makes every new room feel dangerous. A packed schedule of paid work leaves no evenings free. The campus social culture assumes a drink in every hand. The people around you do not share your background, your language, your faith or your way of seeing the world. This chapter takes those obstacles seriously. None of them makes meaningful friendship impossible, but each changes the route. The aim here is to describe the adjustments that help, and to be honest about when an obstacle calls for more help than a book can give. Anxiety and the inner critic Almost everyone feels some nervousness in new social situations. For some students, though, the nervousness is intense enough to shape their whole university experience. They skip the club meeting they had planned to attend because the thought of walking in alone is unbearable. They rehearse conversations for hours beforehand and replay them for hours afterward. They avoid the dining hall at busy times. They feel, much of the time, that others are judging them and finding them lacking. It is useful to distinguish between shyness, which is a common temperamental tendency toward caution in new social situations, and social anxiety disorder, a recognised mental health condition in which fear of social situations is persistent, intense and interferes significantly with daily life. The line between them is not always sharp, and you do not need a diagnosis to benefit from help. But if social fear is regularly stopping you from doing things you want to do, attending classes, eating in public places, speaking in seminars or joining activities, it is worth talking to a professional. Social anxiety is common and treatable. Cognitive behavioural therapy, in particular, has a strong evidence base for it, and many campus counseling services offer it or can refer you to someone who does. Whether or not your anxiety reaches that level, a few ideas drawn from cognitive behavioural approaches can help. The first concerns safety behaviours. These are the small things anxious people do to get through social situations: staying on their phone, rehearsing what to say, avoiding eye contact, hovering near the door, leaving early. They feel protective, but they tend to keep anxiety going, for two reasons. They prevent you from learning that the situation would have been manageable without them, and they often make you seem less approachable, which can lead to the very coolness you feared. Gradually dropping safety behaviours, one at a time, is uncomfortable at first but tends to reduce anxiety over time. The second concerns graded exposure. Rather than throwing yourself into the most frightening situation, you build a ladder of situations from least to most anxiety-provoking and work up it step by step. For a student who wants to join a club, the ladder might begin with reading the club's online page, then attending a large, anonymous taster session, then attending a small meeting and saying only a greeting, then attending and asking one question, then staying for the informal time afterward, and so on. Each step is repeated until it feels manageable before moving to the next. The structures recommended in this book suit this approach well, because their regularity means you can climb the ladder gradually, week by week, in the same setting. The third concerns the inner critic. Anxious people tend to judge their own social performance harshly, focusing on every awkward moment and discounting evidence that things went well. The liking gap described in Chapter 2 is a mild version of this tendency that affects almost everyone; in anxious people it is often much stronger. A useful practice is to ask, after a social interaction, not "what went wrong?" but "what actually happened?", listing facts rather than interpretations. "She asked me two questions about my course and laughed at my joke about the projector" is a fact. "She was probably just being polite" is an interpretation. Learning to separate the two takes time, but it weakens the critic's grip. It helps, too, to know that the structures described in this book are unusually friendly to anxious people. A clear role, a shared task and a predictable routine all reduce uncertainty, which is what anxiety feeds on. Many students who find parties unbearable find a weekly volunteer shift perfectly manageable, and some discover in those settings a social ease they had never experienced before. Anxiety should also be distinguished from introversion, which is not a problem at all. Introverted students tend to find large, loud gatherings tiring and to recharge in solitude or in the company of one or two people. The writer Susan Cain made the case for this temperament to a wide audience in Quiet (2012), arguing that many institutions are built around an extroverted ideal that undervalues people who prefer depth to breadth. University social culture is often one of those institutions. Introverted students sometimes conclude that they are bad at friendship because they do not enjoy the party scene, when in fact they are simply suited to a different route. The approach in this book, a small number of recurring settings, one-to-one conversation, depth over breadth, is close to how many introverts naturally prefer to connect. What introverts may need to watch is not their preference for smaller settings but a tendency to let time alone crowd out the regular contact that friendship requires. Protecting two structures in your week, and resting generously around them, is usually a better balance than either forcing yourself to go out constantly or withdrawing altogether. Loneliness and anxiety can also be part of something heavier. If you find yourself persistently low, losing interest in things you used to enjoy, sleeping far more or far less than usual, or thinking that others would be better off without you, please seek help promptly. Campus counseling services are a good first step; your doctor or a local health service is another. If you are having thoughts of suicide or are in crisis, in the United States you can call or text 988 to reach the 988 Suicide & Crisis Lifeline; in the United Kingdom and Ireland, Samaritans can be reached free on 116 123. Services differ elsewhere, and your campus will be able to tell you what is available locally. If you or someone else is in immediate danger, contact emergency services. Counseling conversations are generally treated as confidential, but there are limits, especially where someone's safety is at risk, and it is reasonable to ask a counselor to explain those limits at the start. Not drinking in a drinking culture On many campuses, alcohol is woven into social life so thoroughly that students who do not drink can feel excluded by default. They may abstain for religious or cultural reasons, for health reasons, because they are in recovery, because of family history, because of medication, because they are under the legal drinking age, or simply because they do not enjoy it. Whatever the reason, they often find that invitations assume drinking, that conversations at social events revolve around it, and that their choice attracts questions or pressure. The first thing to say is that non-drinking students are far from alone. The Prentice and Miller research described in Chapter 2 is a reminder that visible drinking norms can overstate how comfortable students actually are with them. Many students drink less than their peers assume, and many would welcome more social options that do not centre on alcohol. The structures this book recommends, niche clubs and volunteering, are in most cases not organised around drinking at all, which makes them a natural fit. The second thing to say is that you do not owe anyone an explanation. If you are asked why you are not drinking, a short, relaxed answer is usually enough: "I don't drink, but I'm happy to be here," or simply, "Not tonight, thanks." Some students find it easier to hold a soft drink so that the question does not arise. If someone keeps pressing, that says something about them, not you, and you are entitled to move on. Third, you can shape the social life of your structures. If a club's informal tradition is always a trip to the bar, suggest an alternative some weeks: a café, a late-night food place, a walk. Many members may be glad of it. If you start your own standing event, as Chapter 4 described, you can make it alcohol-free from the start. Some campuses also have alcohol-free social programmes, faith-based societies with active social lives, and, especially in the United States, collegiate recovery programmes that offer community for students in recovery from addiction. If any of these apply to you, they are worth seeking out. Finally, a word for students who do drink but are uneasy about how much. If you notice that drinking has become the main way you socialise, that you rarely spend time with your friends sober, or that you are drinking more than you want to because it seems expected, the advice in this book may be especially relevant to you. Building friendships in settings that do not involve alcohol can loosen its hold on your social life without requiring you to give up nights out altogether. If you are worried that your drinking is becoming a problem, campus health services can help, and in the United States the SAMHSA National Helpline, 1-800-662-4357, offers free, confidential referral information. Time, money and distance Some of the most significant obstacles to friendship are practical. The advice to join clubs and volunteer assumes that you have some free time, a little spare money, and the ability to be on campus at the times things happen. Many students have none of these in abundance. Students who work long hours for pay face a particular squeeze. When evenings and weekends are taken by shifts, there is little room for anything else. For these students, it can make sense to look for friendship within settings they already inhabit, and to be strategic about the few hours that are free. The workplace itself can be a structure: colleagues who share shifts see each other repeatedly and share a task, and some of those relationships can be developed deliberately in the way Chapter 6 describes. If you have a little flexibility, choosing a single structure that meets at a time you can reliably protect, even once a fortnight, is better than trying to fit in several. Money shapes social life in less visible ways. Nights out, club memberships, trips and meals can add up quickly, and students without much money can find themselves declining invitations and gradually drifting to the edge of groups. The structures recommended here are often cheaper than party-centred social life. Many clubs have low or no membership fees, many universities offer hardship funds or fee waivers for student activities, and volunteering is free. When you issue invitations yourself, you can choose low-cost options: a walk, a shared meal cooked at home, a free talk or concert. Being matter-of-fact about money also helps. "I'm on a tight budget this month, but I'd love to do something cheap" is a perfectly ordinary thing to say, and it often gives others permission to say the same. Commuter students, who live at home or off campus and travel in for classes, face the problem of distance. They often miss the unplanned encounters that residents take for granted, and evening events can mean a long journey home late at night. For commuters, the principle of choosing structures carefully becomes even more important. Look for clubs and volunteer roles that meet during the day, or on days when you are already on campus. Many universities have commuter student societies or lounges, which can be good places to meet others in the same situation. And consider whether some of your friendship-building can happen near home: volunteering in your own community, for example, can provide the same benefits without the commute. Students with caring responsibilities, whether for children, parents or other family members, face similar constraints, often more severely. For them, the goal may be a smaller number of carefully chosen connections rather than a busy social life, and the most valuable structures may be those that welcome their circumstances: student-parent networks, mature student societies, or flexible volunteer roles. Difference and belonging A final set of obstacles concerns difference: being in some important way unlike most of the people around you. This can mean being the first in your family to attend university, being an international student far from home, being older than most of your peers, being a member of a racial, ethnic or religious minority on your campus, being LGBTQ+, being disabled, or being neurodivergent. Each of these experiences is distinct, and I will not pretend that a few paragraphs can do justice to any of them. But a few general principles apply across them. The first is that identity-based communities can be a vital source of belonging, especially early on. Societies for international students from a particular country or region, first-generation student networks, LGBTQ+ societies, faith groups, disabled students' associations and cultural societies offer something distinctive: the relief of not having to explain yourself. For many students, these communities are where their first real friendships at university form, and there is nothing limiting about that. They are structures like any other, and they often combine regular contact, shared activity and shared values in exactly the way this book recommends. The second is that it can be valuable, over time, to have friendships that cross lines of difference as well as friendships within your own community. Niche clubs and volunteering are particularly good at this, because they bring people together around shared interests and values that cut across other identities. A student who finds her first friends in the international student society may later find others in a chess club or a tutoring programme, and the combination of both kinds of friendship can be a source of strength. The third is that some students face specific barriers that deserve specific help. International students may be navigating a new language, unfamiliar social conventions and long distances from family; many universities have international student offices that run social programmes and can connect students with peers. First-generation students may feel that others understand an unwritten code they have never been taught; first-generation networks and mentoring programmes can help decode it. Neurodivergent students may find the unwritten rules of socialising particularly opaque, and may benefit from structured settings with clear roles and routines, which are exactly the settings this book recommends. Disabled students may find that some venues and activities are inaccessible, and are entitled to ask for adjustments; disability services offices can often help. Consider a composite student, Wei, who came from China to study engineering in the United Kingdom. In his first months he spent almost all his social time with other Chinese students, which gave him comfort and support but left him feeling that he was living alongside British university life rather than within it. He was anxious about his spoken English and worried about misunderstanding jokes. In his second term he joined a small robotics club, drawn by the work itself. The shared task gave him something concrete to talk about and a way to contribute that did not depend on fluency. Over months, two club members became real friends, and his confidence in English grew alongside the friendships. He did not abandon his existing community; he added to it. By his final year, he described his social life as having two homes, and he valued both. The obstacles described in this chapter are real, and for some students they are heavy. But none of them changes the basic logic of how friendship forms. It still grows from repeated contact, shared activity, shared values and gradual deepening. What the obstacles change is which structures are realistic, how much time and support you may need, and how patient you need to be with yourself. The next chapter turns to what happens once friendships have formed: how to keep them, how to repair them, and how to let them change. Chapter 8: Keeping Friends Most advice about friendship at university focuses on making friends, as though the job were finished once you had found your people. But friendships are not possessions that, once acquired, stay put. They are more like gardens, or perhaps like fires: they need tending, and they change over time whether or not you tend them. The student who has worked hard to build two or three close friendships during the first year may find, by the third, that one has faded, one has become strained, and one has become the most important relationship in their life. Understanding how friendships are maintained, how they go wrong and how they change is the final piece of the picture. Maintenance: the unglamorous work Robin Dunbar's research, introduced in Chapter 1, has a sobering implication for anyone who has made friends: without contact, friendships decay. Dunbar and his colleagues have found that emotional closeness tends to decline when people see each other less often, and that the decline can be surprisingly quick. His work with Sam Roberts on young people moving from school to university found that friendships with those left behind weakened unless both people made a deliberate effort to stay in touch, while relationships with family were more resilient. Friendship, in other words, is a relationship with no built-in obligation to continue. It survives only because both people keep choosing it. At university this has two practical consequences. The first is that friendships formed in a particular setting are vulnerable when the setting changes. The climbing club friend becomes harder to see when one of you stops climbing; the volunteer friend becomes more distant when your shifts change. Friendships that have extended beyond their original setting, in the way Chapter 6 described, are more resilient, because they are no longer dependent on one structure. But even these need care when routines change at the start of a new term or year. The second consequence is that maintenance does not have to be dramatic. The research on friendship decay is about frequency of contact more than intensity. A friendship can be kept alive by modest, regular gestures: a message when you see something that reminds you of them, a quick coffee between classes, a standing arrangement to eat lunch together on Fridays. In his research on couples, the psychologist John Gottman has written about bids for connection, the small moments in which one person reaches for another's attention, such as pointing out something interesting or sharing a passing thought, and about how much relationships depend on those bids being noticed and returned. Friendships work in a similar way. Their health depends less on grand occasions than on small, repeated acts of attention. Rituals are especially valuable. A ritual is a recurring shared activity that belongs to a particular friendship: the Sunday evening film, the walk to the market every Saturday, the late-night toast after exams. Rituals make maintenance automatic. Nobody has to issue an invitation; the time together simply happens. They are, in effect, a friendship's own private structure, and they draw on exactly the same principle that makes clubs and volunteering effective. If you have a close friend with whom you do not yet have a ritual, consider proposing one. "Should we make Thursday lunch a regular thing?" is a small question with large consequences. Maintenance also means showing up at the moments that matter. There are occasions in every friendship when presence is disproportionately important: a birthday, a breakup, a family illness, a failed exam, a success that deserves celebration. Friends remember who was there at those moments, and who was not. You cannot be there for every moment, and a good friendship allows for that. But when you know a friend is going through something significant, make the effort: a message, a visit, a meal, an offer of help. These are the moments in which friendships of pleasure show their limits and friendships of character show their strength. Sometimes showing up means supporting a friend through something serious: depression, an eating disorder, a sexual assault, a crisis at home, thoughts of self-harm. These situations can be frightening, and students often worry about saying the wrong thing. A few principles help. Listen more than you speak, and take what your friend tells you seriously. Do not promise to keep something secret if it involves a risk to their safety, because that is a promise you may not be able to keep. If you are worried that a friend may be thinking about suicide, it is appropriate to ask them directly; the evidence suggests that asking about suicide does not put the idea into someone's head, and it can be a relief for the person to be asked. Encourage them to seek professional help and, if they are willing, offer to go with them to the counseling service or sit with them while they make a call. In the United States, the 988 Suicide & Crisis Lifeline can also be contacted by someone worried about another person. If there is immediate danger, call emergency services. Just as important, look after yourself. Supporting a friend in crisis can be exhausting, and you are not their therapist. It is not disloyal to tell a counselor, a trusted staff member or another friend that you are worried and need support of your own. Friendships that carry one person through a hard time are among the strongest there are, but only if the person doing the carrying does not collapse under the weight. Finally, maintenance includes the ordinary work of reciprocity. Over time, both people should feel that they are giving and receiving roughly equally, though the balance will shift from month to month. A friendship in which one person is always the listener, always the planner or always the one who adjusts will eventually strain. It is worth checking, now and then, whether your important friendships feel balanced, and gently raising it if they do not. Conflict and repair Every close friendship eventually encounters conflict. A friend says something hurtful, cancels plans once too often, shares something that was told in confidence, or behaves in a way that clashes with your values. How these moments are handled determines whether the friendship deepens or dissolves. Many students, especially those who have only recently found close friends, are frightened of conflict. They fear that raising a grievance will end the friendship, so they swallow it. But unspoken grievances do not disappear; they accumulate, and they tend to leak out as coolness, sarcasm or withdrawal. Friendships often end not through a dramatic falling-out but through a slow cooling that neither person names. The alternative is to raise concerns early, while they are small, and to do so in a way that invites repair rather than defence. A useful structure has four parts: describe the specific thing that happened, say how it affected you, express that you value the friendship, and invite the other person's perspective. For example: "When you cancelled on Saturday for the third time, I felt pretty hurt, because I'd been looking forward to it. I really value our friendship, so I wanted to say something rather than let it build up. Is everything okay on your side?" This approach works because it is specific rather than general ("you always cancel"), it describes your experience rather than attacking their character, it frames the conversation as care for the friendship, and it leaves room for information you do not have. Often there is such information. The friend who kept cancelling may be struggling with depression, drowning in coursework, or dealing with something at home. A conversation that begins with curiosity is much more likely to uncover this than one that begins with accusation. When you are the one who has caused hurt, repair works best when it is direct and does not come with excuses. "You're right, I did cancel three times, and I'm sorry. I can see why that felt hurtful." Explanation can come afterward, if it is needed, but leading with it tends to sound like self-justification. A good apology acknowledges what happened, recognises its effect and, where possible, commits to doing differently. Repair does not always succeed, and not every conflict should end in reconciliation. Some friendships reveal, under strain, that they are built on something other than mutual care. If a friend repeatedly belittles you, controls whom you see, betrays confidences, pressures you into things you do not want to do, or leaves you consistently anxious and diminished, it is reasonable to step back, and sometimes necessary. Chapter 1 noted that part of building meaningful friendships is being able to recognise relationships that take more than they give. Having several friendships, rather than depending entirely on one, makes it much easier to make that judgement clearly, because you are not choosing between a difficult friendship and no friendship at all. Stepping back from a friendship does not always require a confrontation. Sometimes it is enough to invest less, to see the person in groups rather than alone, and to let the relationship move outward from the inner circle. At other times, especially where there has been serious harm, a clearer ending is healthier. If a relationship involves threats, harassment or abuse, campus support services and, where necessary, the police are there to help, and you should not have to handle it alone. Change, distance and the long view Friendships change not only through conflict but through the ordinary movement of life. At university, that movement is unusually rapid. People change courses, move residences, take up new interests, fall in love, go abroad for a term, take a year out, graduate at different times. Each of these changes can reshape friendships, sometimes painfully. The first thing to understand is that some drift is natural and not a sign of failure. A friendship that was central in your first year may become peripheral in your third, not because anything went wrong, but because you have both grown in different directions. Dunbar's layers help here: friends move between the circles over time, and a friend who is no longer in your innermost five may still be a valued member of your wider fifteen or fifty. It is possible to hold such friendships lightly, with gratitude for what they were, without insisting that they remain what they were. The second is that some friendships are worth deliberate effort to carry across a change. When you or a close friend goes abroad for a term, or when summer scatters everyone to different places, the friendship is at risk precisely because the structure that sustained it is gone. The Roberts and Dunbar findings suggest that effort matters: friendships that are actively maintained across distance hold up better than those left to chance. A regular call, even a short one, a shared online activity, or a plan to see each other on return can bridge the gap. It helps to make the plan explicit before the separation begins: "Let's call every other Sunday while you're away." The third is that friendships from home deserve thought too. Many students arrive at university with close friends from school, and those friendships can come under strain as each person's new life takes shape. Some fade, as the research predicts; others survive and become lifelong. There is no rule about which should be kept, but it is worth noticing which of your home friendships you actually want to maintain, and investing in those deliberately, rather than letting all of them drift by default or clinging to all of them out of guilt. The fourth concerns graduation, the largest transition of all. Many students worry, as the end of university approaches, that the friendships they have built will not survive the scattering that follows. Some will not, and that is a real loss. But friendships that have been built on shared values and mutual knowledge, rather than on shared circumstances alone, are the ones most likely to last. In this sense, the argument of this book looks beyond university. Friendships of character, formed through repeated shared commitment and gradual deepening, are precisely the ones that tend to survive when the setting that produced them is gone. And the skills you have practised in building them, choosing structures, showing up, issuing invitations, deepening conversation, repairing conflict, will serve you in every new place you live. Consider a composite student, Aisha, who graduated with three close friends from her university years: one from a student newspaper, one from a mentoring programme, one she had met on her residence floor in the first week. In the years after graduation they lived in four different cities. They kept a group chat going, called one another on birthdays, and made a point of meeting for a weekend together once a year. The annual weekend became their ritual, the structure that replaced the structures of university life. Their friendships changed in shape; they no longer knew the small details of each other's daily lives. But they remained in each other's inner circles, because each of them kept choosing to keep them there. It is worth returning, at this point, to the exercise from the end of Chapter 1: the three circles, with the people you would call in a crisis at the centre, the people you see and enjoy regularly in the middle, and friendly acquaintances on the outside. Redraw them at the end of each term. You are likely to see movement: a name from the outer circle that has moved inward, a name from home that has drifted outward, perhaps an empty inner circle that now has one or two names in it. The exercise is not a scorecard. Its value is in making visible where your time and attention are going, and in prompting a few honest questions. Which of these friendships do I want to invest in next term? Which structures are feeding them? Is there anyone in the middle circle I have been meaning to know better? Is there anyone I have been neglecting? A few minutes of reflection, repeated each term, keeps the process of building and keeping friends deliberate rather than accidental. A last word belongs to the subject this book began with. It would be easy to read this book as an argument against parties. It is not. Parties, nights out and large social events can be genuinely enjoyable, and they have a real place in university life. The argument has been that they are a poor primary strategy for building close friendships, not that they are worthless or that students who enjoy them are doing something wrong. In fact, once you have close friends, parties often become more enjoyable. Going out with people you know well, who will look out for you, with whom you can have a real conversation in the corner or on the walk home, is a different experience from going out with a group you barely know in the hope that something will happen. The party becomes an occasion to celebrate friendships that already exist, rather than a stressful search for friendships that might not arrive. Many students who follow the approach in this book find that their social lives include both the quieter structures where friendships are built and the livelier occasions where those friendships are enjoyed. The point, finally, is balance and intention. A good social life at university is not defined by how often you go out or how many people you know. It is defined by whether you have a few people who know you well, whom you trust, and with whom you share something that matters. Every other feature of social life, including the parties, is worth judging by whether it helps or hinders that. Conclusion The argument of this book can be put in a sentence: close friendship at university is built less by social talent than by commitment to a few recurring settings organised around what you care about, followed by small, deliberate steps toward depth. Having followed that argument through its evidence and its practice, it is worth asking what follows from it, because some of the implications are not obvious at the start. Commitment in a culture of options The first implication is that the main enemy of friendship on campus is not shyness, or a lack of charm, or even the party culture as such. It is the habit of keeping options open. University life is saturated with choice: dozens of events every week, hundreds of clubs, constant invitations flowing through group chats. The natural response to so much choice is to sample, to drift from thing to thing, to say "maybe" to everything and commit to little, in case something better comes along. That habit feels like freedom and social openness. In practice it is almost perfectly designed to prevent close friendships from forming, because it denies them the one resource they cannot do without: repeated time with the same people. Seen this way, the students who seem to find close friends easily are often not the most sociable. They are the ones who, early on, narrowed their options: who committed to the choir, the tutoring programme, the small club, and kept turning up. They accepted a cost, the parties they missed and the other clubs they never tried, in exchange for depth. The practical lesson is that choosing well at university involves choosing against things as well as for them. A student who wants closer friendships should probably say no to more events, not fewer, and yes to fewer structures, more consistently. This also changes what it means to feel lonely in a busy social life. The student who is out every night and still lonely is not failing at socialising. They are succeeding at a kind of socialising that does not produce what they want. The remedy is not more effort in the same direction, but a change of direction. What institutions could do differently The second implication concerns universities themselves. If friendship grows best in structures, then institutions that care about student wellbeing and retention should invest in structures, not only in events. Much of the energy of campus social programming goes into welcome weeks, one-off gatherings and large occasions. These have a place as entry points, but they cannot do the work that small, recurring, purposeful groups do. An institution that took the argument of this book seriously might do several things. It might make it easier for students to form and sustain small groups, with rooms that are free and simple to book and modest funding for student-run programmes. It might invest in volunteering schemes that place students in regular teams rather than one-off service days. It might design first-year experiences around small cohorts that meet repeatedly, rather than large orientation groups that meet once. It might ensure that a meaningful share of social life is available without alcohol, not as a token alternative but as a genuine part of campus culture. And it might communicate honestly with new students about how friendships actually form, including the reassurance that feeling adrift in the first term is common and that the second term is a second chance. None of these ideas is new, and many campuses already pursue some of them. But they deserve more weight than they usually receive, because the evidence on why they matter is consistent. What remains unsettled It would be misleading to present the science of friendship as complete. Much of the research described in this book relies on people's own reports of their relationships and feelings, which are imperfect measures. Many studies were conducted with particular populations, often university students in North America or Western Europe, and may not generalise neatly to every culture or campus. The estimates of how many hours it takes to make a friend are rough averages that conceal wide variation. The association between volunteering and wellbeing runs in both directions, and disentangling cause from effect is difficult. Some important questions remain genuinely open. We do not yet fully understand how online connection interacts with in-person friendship: when it supplements, when it substitutes and when it erodes. We know less than we should about how friendship formation differs for students from different backgrounds, and about what institutions can do most effectively to help. Readers should hold the specific findings in this book with appropriate modesty. But the broad pattern, that close friendship grows from repeated shared time, shared activity, shared values and gradual reciprocal disclosure, is supported from many directions at once: philosophy, psychology, sociology and the ordinary experience of most people who have ever had a close friend. What to do this week The final implication is practical, and it is where this book should end. Knowing how friendship forms is not the same as forming friendships, and the gap between the two is closed only by action. If you have read this far, here is what the argument suggests you do in the next seven days. Look at your week and identify whether it contains any setting that combines regular contact, conversation, shared activity and shared values. If it does not, choose two candidates, one club and one volunteer role or one of each kind, and find out when they next meet. Put the dates in your calendar and commit to six visits before deciding. At the first one, arrive early and learn two names. Look at the people you already know, choose one you would like to know better, and issue one specific, small invitation. And when a conversation leaves you feeling that it went "fine but probably not great", remember that you are probably underestimating how much you were liked, and act accordingly. None of this requires you to change who you are. It asks you to choose where you spend your time, to keep showing up, and to take the small risks that friendship has always required. The friends you are hoping for are probably already on your campus, looking for the same thing and wondering whether anyone else is. Many of them are one recurring Tuesday evening away. Hashtags: #AuthenticConnections #CampusFriendships #MeaningfulFriendships #HealthyFriendships #CollegeFriendships #UniversityLife #StudentWellness #CampusCommunity #BeyondPartyCulture #GenuineRelationships #SocialConnection #BuildingFriendships #FriendshipPsychology #SenseOfBelonging #StudentBelonging #CampusInvolvement #StudentClubs #NicheClubs #CampusVolunteering #SharedInterests #SharedValues #SocialWellbeing #OvercomingLoneliness #StudentMentalHealth #SocialAnxiety #InterpersonalRelationships #EmotionalSupport #RelationshipBuilding #FriendshipMaintenance #CollegeSocialLife

Download the Book (PDF): Introduction A bottling line running at 60,000 containers an hour presents a new bottle to the inspection station roughly every sixty milliseconds. In that interval a camera must be triggered, an image exposed and transferred, a decision computed, and a signal sent to a reject mechanism that will fire at precisely the moment the faulty bottle passes it a metre or two further down the conveyor. The decision cannot be late, because a late decision rejects the wrong bottle. It cannot be timid, because every good bottle thrown away is margin lost. And it cannot be careless, because the one bad bottle that escapes may carry a misprinted expiry date, a cracked neck or a foreign body into a customer's hands, and from there into a recall. This is the setting for applied computer vision in manufacturing, and it is a different world from the one most people picture when they hear the phrase. The public story of computer vision over the past decade has been a story about models: networks that classify a thousand categories of photograph, detect pedestrians in street scenes, segment anything in an image on request, or answer questions about a picture in fluent prose. Those achievements are real, and they have changed what is possible on the factory floor. But engineers who have actually put vision systems into production lines know something the benchmark tables do not show. The model is rarely the part that fails. What fails is the light, when a skylight lets afternoon sun fall across a conveyor that was commissioned on a cloudy morning. What fails is the trigger, when a photo-eye misfires on a reflective label and the camera images the gap between two parts. What fails is the definition of a defect, when the quality department, the production supervisor and the customer each hold a slightly different idea of how long a scratch must be before it matters. What fails is the timing, when an inference that took eight milliseconds on the developer's workstation takes forty on the embedded computer in a hot electrical cabinet, and the reject gate fires too late. What fails, eventually and almost always, is the assumption that the world the system was trained on will remain the world it operates in. The argument of this book This book argues a single proposition: an industrial vision system is a measurement instrument, and it succeeds or fails on the engineering that surrounds the neural network rather than on the network itself. Upstream of the model sits the physics of image formation — illumination, optics, sensors, triggering — which determines whether the defect is visible at all. Downstream sits the decision system — thresholds, reject timing, operator review, validation, monitoring — which determines whether a visible defect leads to a correct action, repeatably, for years. The model in the middle matters, and this book treats it seriously, but it is the most replaceable component in the chain. A good model cannot rescue a bad image, and a good image with a mediocre model will often outperform a poor image with an excellent one. The practical consequence of treating vision as measurement is a change of discipline. Measurement engineers ask questions that machine learning practitioners are often not trained to ask. What is the smallest feature this instrument must resolve, and with what margin? How repeatable is its answer when the same part is presented twice? How does its output change with temperature, with ageing of the light source, with a different operator loading the fixture? How will anyone know when it has drifted? These questions have well-developed answers in metrology and quality engineering, and they apply to a camera running a convolutional network as directly as they apply to a coordinate measuring machine. What the book covers The description of this book promises three things: deploying object detection models on edge devices, overcoming lighting variance on the factory floor, and automating optical character recognition on high-speed conveyors. Each receives a full chapter, but each also depends on material that comes before it, so the book is ordered by the path a photon takes from the light source to the reject gate. Chapter 1 defines the inspection problem itself: the economics of escaped defects and false rejects, the limits of human inspection, the difference between rule-based machine vision and learned models, and the often-neglected work of turning a quality specification into something a machine can decide. Chapter 2 treats illumination as the first and most powerful algorithm in the system, explaining how the geometry and wavelength of light decide which defects become visible. Chapter 3 covers lenses, sensors, interfaces and the timing problem of imaging parts that are moving at metres per second. Chapter 4 turns to data, and specifically to the central difficulty of industrial learning: defects are rare, varied and expensive to collect. It explains why unsupervised anomaly detection has become a core technique and where its limits lie. Chapter 5 covers supervised detection and segmentation models, including the YOLO family that dominates edge deployments, and argues that the operating point chosen on a model's precision-recall curve is a business decision rather than a technical one. Chapter 6 takes those models onto edge hardware, through compilers and quantisation, and into conversation with the programmable logic controllers that actually run the line. Chapter 7 returns to light, this time from the operational side: how lighting changes once a system is installed, how to design models and pipelines that tolerate it, and how to detect the changes that cannot be tolerated. Chapter 8 is devoted to optical character recognition on fast conveyors — date codes, lot numbers, serialised pharmaceutical packs — where the task is not just to read text but to verify, at line speed, that what was printed matches what should have been printed. Chapter 9 closes the technical arc with validation and life-cycle management: how to prove a vision system works, how to keep proving it, and how to change it without breaking the evidence. Who this book is for The intended reader is a practitioner: a controls or automation engineer who has been asked to add deep learning to an inspection cell, a machine learning engineer who has been asked to make a model work in a factory, a quality engineer who must sign off on a system neither of them fully explains, or a manager who needs to understand why the pilot that worked so well in the lab is struggling on the line. No particular mathematical background is assumed beyond comfort with basic probability and geometry. A few short code fragments appear where they make a point more precisely than prose can, but the argument never depends on them. The book names real products, standards, datasets and research papers throughout, because concrete examples are more useful than abstractions. Product names are given as illustrations of a category, not as endorsements, and hardware specifications in particular change quickly. Where figures are quoted, they are figures that manufacturers or authors have published; where a precise figure is not available or not reliable, the text says so rather than inventing one. A note on what has changed Industrial vision is an old field. Cognex was founded in 1981, and smart cameras that could locate a part, measure a gap or read a barcode were commonplace on production lines by the late 1990s. For most of that history, inspection meant hand-built rules: find this edge, measure that distance, count the dark pixels in this region, compare against a tolerance. Those methods remain excellent for problems that can be stated geometrically, and a large share of installed systems still use them. What has changed in the past decade is that deep learning made a different class of problem tractable: the cosmetic defect that no one can describe precisely, the texture that varies naturally from part to part, the handwritten-looking dot-matrix code smeared by a printer running low on ink. Cognex's acquisition of the Swiss deep learning company ViDi Systems in 2017 marked the moment the established vendors accepted this shift. Since then the tooling has matured rapidly. Open-source anomaly detection libraries can learn what a good part looks like from a few hundred images. Real-time detectors run at hundreds of frames per second on modules the size of a credit card. Optical character recognition engines that once needed font training out of the box now read degraded industrial print with no font file at all. The tools are now good enough that the bottleneck has moved. It sits where it always sat in measurement engineering: in the conditions under which the instrument works, and in the procedures by which it is trusted. That is where this book spends most of its time. CHAPTER 1 The Inspection Problem Every automated inspection system makes the same decision millions of times: accept this part, or reject it. Everything else — the cameras, the lights, the networks, the dashboards — exists to make that binary choice correctly, quickly and consistently. Before any of that equipment is specified, an engineer must answer a question that sounds trivial and almost never is: what, precisely, counts as a defect, and what does it cost to get the answer wrong in each direction? This chapter treats inspection as a decision problem with asymmetric costs, explains why human inspection sets a lower bar than most people assume, and distinguishes the classes of problem that suit rule-based machine vision from those that require learned models. It ends with the step most projects skip: turning a quality specification into something a machine can decide. Two kinds of error, two kinds of cost An inspection system can be wrong in two ways. It can pass a bad part, which quality engineers call an escape and statisticians call a false negative. Or it can reject a good part, which is a false reject, a false positive, or in plant-floor language simply "overkill". Every design decision in a vision system trades one of these against the other, and the right trade depends entirely on what each error costs. The cost of an escape varies by orders of magnitude. A slightly off-centre label on a bottle of shampoo may cost nothing at all; nobody returns shampoo because the label is two millimetres to the left. A missing or illegible expiry date on a food product can trigger a withdrawal from retail shelves. A mislabelled allergen is among the most common causes of food recalls in both the United States and Europe, and a recall's cost includes not just the product but the logistics of retrieval, the retailer penalties, the regulatory attention and the brand damage. In automotive supply, an escaped defect on a safety-relevant component can lead to a customer line stoppage, charge-backs that run to thousands of dollars per hour of stopped production, and, in the worst case, a field campaign. In pharmaceutical packaging, an incorrect batch number or serial code is a compliance failure that regulators treat with corresponding seriousness. The cost of a false reject is usually smaller per event but far more frequent. A false reject throws away the material, energy and labour already invested in the part. On a high-volume line, a false reject rate of half a per cent can represent hundreds of thousands of good parts a year. Worse, false rejects corrode trust. Operators who watch a system reject parts that look perfectly good to them will, sooner or later, find a way to bypass it — by widening a tolerance, by re-feeding rejected parts, or by switching the station to "pass-through" mode during a busy shift. A vision system that is technically sensitive but practically ignored has an effective escape rate equal to that of no system at all. This asymmetry means there is no such thing as a single "accuracy" figure for an inspection system. A model that is 99 per cent accurate on a dataset in which one per cent of parts are defective could achieve that score by passing everything. What matters is the pair of rates — the fraction of real defects caught and the fraction of good parts rejected — at a chosen operating point, and the economic weight attached to each. Chapter 5 returns to this in detail when discussing thresholds. The point here is that the weights must be decided before the system is designed, because they determine how much resolution, how much lighting control and how much model capacity the problem deserves. A useful exercise at the start of any project is to write down, with the quality and production managers in the room, three numbers: the expected defect rate, the maximum tolerable escape rate, and the maximum tolerable false reject rate. If the defect rate is one in a thousand and the customer will tolerate one escape per million parts, the system must catch at least 99.9 per cent of defects. If the line produces fifty million parts a year and the finance department will tolerate the loss of ten thousand good parts, the false reject rate must be below 0.02 per cent. Those two numbers together define a target that may or may not be physically achievable, and it is much better to discover that in a meeting than after the equipment has been bought. How good is a human inspector? Most automated inspection projects replace or supplement people, so the natural baseline is human performance. It is widely assumed that a careful human inspector is close to perfect and that the machine must strive to match them. The research literature on visual inspection says otherwise. Human-factors studies of inspection go back more than half a century, to work on industrial inspectors in the 1950s and 1960s, and they have been surveyed many times since. Judi See's review of the visual inspection literature for Sandia National Laboratories in 2012 drew together decades of studies across manufacturing, aviation maintenance and security screening, and its picture is sobering: inspectors routinely miss a substantial fraction of the defects in front of them, with miss rates in the range of twenty to thirty per cent appearing frequently in the studies reviewed, and performance degrading with defect rarity, task duration, time pressure and the complexity of the item. Colin Drury's work on aircraft inspection and on the human factors of quality control reached compatible conclusions over a long career: inspection is a search-and-decision task, and both the search and the decision are fallible in predictable ways. Three effects are especially relevant to anyone designing an automated replacement. The first is the prevalence effect. When defects are rare, people stop expecting them and miss more of them. Studies of baggage screening and medical imaging have demonstrated this repeatedly, and it applies with full force to a production line where one part in several thousand is defective. The second is vigilance decrement: sustained attention to a monotonous task declines over tens of minutes, which is why inspection stations staffed by people usually rotate operators. The third is criterion drift. The threshold at which a person decides that a mark is "bad enough" moves with fatigue, with recent experience, with production pressure and with the person. Two inspectors given the same borderline parts will disagree, and the same inspector given the same parts on two different days will disagree with themselves. This last point matters more than it first appears. If the humans whose labels will be used to train and evaluate a model do not agree with one another on borderline cases, the model cannot be held to a standard of agreement higher than theirs. The ceiling on measurable model performance is set by the consistency of the ground truth. Chapter 9 describes how to measure that consistency formally, using attribute agreement analysis. For now the lesson is that the human baseline is weaker and noisier than intuition suggests, which is both encouraging, because a machine can plausibly do better, and complicating, because "better" has to be measured against a reference that is itself uncertain. None of this means people should be removed from inspection. They remain far better than machines at reasoning about novel situations, at noticing that something about the line is wrong in a way no one anticipated, and at deciding what to do about it. The most effective arrangements use the machine for exhaustive, consistent screening and the person for review of the uncertain cases and for investigation of patterns. That division of labour is a design choice, and it shapes the rest of the system. Rules, learning and the shape of the problem Industrial machine vision has two broad toolkits, and choosing between them is one of the first architectural decisions. The first is rule-based or "traditional" machine vision. Here an engineer composes a pipeline from well-understood operations: locate the part using pattern matching against a trained template, define regions of interest relative to that location, apply edge detection, blob analysis, intensity thresholds or caliper measurements within those regions, and compare the results against numeric tolerances. Commercial libraries such as MVTec HALCON, Cognex VisionPro, Matrox Imaging Library and the tools built into Keyence and Omron smart cameras have refined these operations over decades. Open-source equivalents exist in OpenCV and elsewhere. Rule-based methods have virtues that are easy to undervalue. They are deterministic: the same image always produces the same answer. They are explainable: when a part is rejected, the system can say that the measured gap was 2.31 millimetres against a maximum of 2.25. They need little data: an engineer can configure a gauging tool from a handful of good parts and a drawing. And they are fast, often requiring only a few milliseconds on modest hardware. For any problem that can be stated geometrically — presence or absence of a feature, position, dimension, count, alignment, barcode grade — rule-based vision remains the right first choice, and a sensible engineer reaches for it before reaching for a neural network. The weakness of rules appears when the problem cannot be stated geometrically. Consider scratches on a brushed aluminium housing, where the brushing itself is a pattern of fine lines. A threshold that catches the scratch also catches the brushing. Consider porosity in a casting, where acceptable surface texture and unacceptable pores overlap in size and contrast. Consider a weld bead, where every good weld looks slightly different and the bad ones are bad in ways that are easier to recognise than to describe. Consider wood, leather, textiles or food, where natural variation in the good product is larger than many defects. In such cases engineers using rules end up stacking exception upon exception, tuning dozens of parameters until the system passes the validation set and then failing as soon as the raw material changes supplier. Learned models address exactly this class of problem. Rather than describing the defect, the engineer provides examples, and the model learns a representation in which good and bad separate. Deep convolutional networks, and more recently transformer-based architectures, can learn texture, context and shape cues that no one could write down as a rule. They tolerate natural variation in the good product far better than thresholds do. They can be retrained when the product changes rather than re-engineered. They also bring costs that rule-based systems do not. They need data, and specifically data that represents the variation they will meet in production. They are probabilistic, producing scores that must be thresholded, and the thresholds must be chosen and justified. Their failures are harder to explain. They require more computation, which on a production line means specialised hardware. And their behaviour can change in ways that are difficult to predict when the input distribution shifts, which on a factory floor it always eventually does. Most production systems now combine the two. A typical arrangement uses rule-based pattern matching to locate and align the part, rule-based gauging for dimensions, a learned model for cosmetic defects, and rule-based code reading or a learned OCR engine for text. Each tool is used where its strengths are decisive. The art lies not in choosing a side but in decomposing the inspection into sub-problems and assigning each to the method that suits it. From specification to decision The most consequential work in an inspection project is also the least glamorous: converting a quality specification into a decision rule that a machine can apply and an auditor can check. Quality specifications are written for people. A typical cosmetic standard for a painted part might say that no scratch longer than 5 millimetres is permitted on a Class A surface, that up to three inclusions smaller than 0.3 millimetres are acceptable per 100 square centimetres provided they are not visible at arm's length under normal lighting, and that "no visible defects" applies to the logo area. Every phrase carries assumptions. Visible to whom, from what distance, under what lighting? Is a scratch's length measured end to end or along its path? Does a scratch that fades out have a definite end? Is an inclusion of 0.29 millimetres really acceptable when the measurement uncertainty is 0.05? A human inspector resolves these ambiguities using judgement and experience, and resolves them differently from the next inspector. A machine cannot resolve them at all unless someone has decided in advance. This is why experienced integrators insist on a defect catalogue before they quote a system: a document, built with the customer, that lists every defect type with its definition, its accept-reject criterion, a set of physical reference samples and photographs of each, and a severity class that ties it back to the cost of an escape. Boundary samples — parts that sit just on either side of the line — are the most valuable items in the catalogue, because they are where the system's threshold will be set and where disputes will arise. Building the catalogue often reveals that the specification is not what anyone actually wants. A food manufacturer may discover that its specification for a date code says only that the code must be "legible", and that the quality team, the retailer and the regulator would each accept or reject different degraded prints. An automotive supplier may find that its customer's cosmetic standard is applied leniently in practice and that a vision system enforcing it literally would reject five per cent of production. These discoveries are uncomfortable but useful. They force a decision that the organisation has been avoiding, and they produce a specification that can be tested. The catalogue also defines the physical requirements of the imaging system. If the smallest defect that must be caught is a 0.2 millimetre pinhole, and a sound rule of thumb is that a defect should cover at least three to four pixels across to be detected reliably, the system needs a pixel footprint on the part of roughly 0.05 to 0.07 millimetres. Over a field of view 200 millimetres wide, that implies at least 3,000 to 4,000 pixels across — a 12-megapixel area sensor or a line-scan camera. Over a field of view twice as wide, the requirement doubles. Arithmetic like this, done in the first week, prevents the most common and most expensive mistake in industrial vision: buying a camera that cannot see the defect. Inline, at-line and offline A final framing decision concerns where the inspection happens relative to the flow of production. Inline inspection examines every part as it passes, at the speed of the line, and drives an automatic reject. It is the most demanding configuration and the main subject of this book. At-line inspection takes parts off the line, sometimes all of them and sometimes a sample, and examines them at a dedicated station nearby, where the part can be fixtured, rotated and imaged more carefully. Offline inspection happens in a laboratory, often on samples, and serves process control and investigation rather than sorting. The choice changes the engineering completely. Inline systems are governed by cycle time and must cope with whatever presentation the conveyor offers: parts that are slightly rotated, slightly tilted, touching, wet, vibrating. At-line systems can afford mechanical fixturing, multiple views and longer exposure times, which often makes a hard vision problem easy. Many successful projects begin by asking whether the hard part of an inline problem could be moved to an at-line station, or whether a cheap mechanical change upstream — a guide rail, a singulating conveyor, a change in how parts are oriented — would make the inline problem tractable. It is also worth asking whether the inspection is meant to sort parts or to improve the process. A system that only sorts treats the symptom. A system whose results are fed back to the process — by tracking which mould cavity, which print head or which weld gun produced each defect — can reduce the defect rate itself. That feedback requires traceability: each image and each decision must be linked to a part identity, a time and the process parameters that produced it. Designing for traceability from the outset costs little. Retrofitting it later costs a great deal. What follows from this The inspection problem, properly framed, is a decision under asymmetric costs, made against a ground truth that is itself fuzzy, on images whose quality is fixed long before any model sees them. The rest of this book follows that framing. The next two chapters deal with forming an image in which the defect is visible and measurable, because no algorithm can decide correctly about information the image does not contain. Later chapters deal with learning a decision from scarce examples, executing it within a time budget, keeping it stable as conditions change, and proving that it works. The model is one stage among many, and the stages before and after it decide most of the outcome. CHAPTER 2 Light Is the First Algorithm Machine vision engineers have a saying that the three most important factors in a vision system are lighting, lighting and lighting. Like most industrial folk wisdom it exaggerates, but not by much. Illumination determines which physical properties of a part are converted into contrast in the image. A defect that produces strong contrast can be found by almost any method. A defect that produces none cannot be found by any method at all. Every hour spent designing illumination saves many hours of algorithm development and data collection, and no amount of model capacity compensates for light that hides the defect. This chapter explains how light interacts with surfaces, describes the main illumination techniques and the defects each reveals, and covers the use of wavelength, polarisation and timing as additional design variables. The operational problem of lighting that changes after installation is the subject of Chapter 7; this chapter concerns the design choices that come first. How surfaces return light When light strikes a surface, some is reflected, some is absorbed, and, for translucent or transparent materials, some is transmitted or scattered beneath the surface. The reflected portion has two components that behave very differently. Specular reflection is mirror-like: light leaves the surface at the same angle at which it arrived, on the other side of the surface normal. Diffuse reflection scatters light in many directions, roughly independent of the angle of arrival. Polished metal, glass, glossy paint and wet surfaces are predominantly specular. Paper, matte plastics, cast surfaces and most textiles are predominantly diffuse. Most real surfaces mix the two. This distinction is the key to almost every lighting design. On a specular surface, a camera sees a reflection of the light source only if the source sits at the mirror angle relative to the camera. Move the light a few degrees and the surface goes dark. A defect that changes the local angle of the surface — a scratch, a dent, a ripple — redirects the specular reflection, so it appears either as a bright mark on a dark background or a dark mark on a bright background, depending on where the light sits. On a diffuse surface, the brightness seen by the camera depends mainly on the reflectance of the material and the amount of light arriving, and is relatively insensitive to small changes in surface angle. A defect on a diffuse surface becomes visible mainly if it changes the material's colour or reflectance, or if it casts a shadow. From this follows the central technique of industrial illumination: choose the geometry of the light so that the feature of interest changes the path of light into the lens, while the background does not. The techniques below are all variations on that idea. The geometries of illumination Bright-field illumination places the light so that its specular reflection from a flat surface enters the lens. The flat surface appears bright; anything that tilts the surface or scatters light — a scratch, a dent, an embossed character, a particle — throws light away from the lens and appears dark. A ring light mounted close to the lens and pointing straight down, or a coaxial light, produces bright-field conditions on a flat specular part. Dark-field illumination does the opposite. Light arrives at a low, grazing angle, so its specular reflection from a flat surface misses the lens entirely and the surface appears dark. Any feature that rises above or cuts into the surface catches the grazing light and scatters some of it upward into the lens, appearing bright against a dark background. Dark-field is the classic technique for surface scratches, embossed and engraved characters, dust, and edge chips on glass or polished metal. A low-angle ring light, or a set of linear bars mounted at shallow angles around the part, provides it. The effect can be striking: a scratch invisible under room lighting stands out as a crisp white line. Backlighting places the light behind the part, facing the camera. Opaque parts appear as sharp black silhouettes against a bright field, which is ideal for measuring outlines, holes, gaps and dimensions, and for detecting presence or absence. Because the contrast is between full light and no light, backlit measurements are among the most robust in machine vision, largely insensitive to the part's colour or surface finish. Backlighting also reveals defects in transparent and translucent materials — bubbles in glass, fill levels in bottles, contamination in film — and with a collimated backlight, which emits nearly parallel rays, it produces edges sharp enough for precision gauging. Diffuse dome illumination surrounds the part with a hemisphere of evenly lit, scattering surface, so light arrives from almost every direction. Specular reflections, which depend on the angle of the source, are averaged away. The result resembles a part photographed on an overcast day: shiny, curved and crinkled surfaces appear flat and evenly lit, and differences in colour and printed content become clear while glare and surface texture disappear. Domes are the standard choice for reading labels on foil pouches, inspecting printing on curved cans, and examining components on reflective circuit boards. Their weakness is the mirror image of their strength: because they suppress surface angle information, they also suppress scratches and dents. Coaxial or on-axis illumination sends light down the optical axis of the camera using a beam splitter, so the part is lit from exactly the direction the camera views it. On a flat specular surface, light reflects straight back into the lens and the surface appears uniformly bright; any feature that tilts the surface appears dark. Coaxial lighting is excellent for flat, mirror-like parts such as silicon wafers, polished metal faces, and laser marks on shiny metal, where it renders the marked area dark against a bright background. Structured light projects a known pattern — a line, a set of stripes, a grid — onto the part and observes its deformation from another angle. A laser line projected across a conveyor, viewed by a camera at an angle, reveals the height profile of the object beneath it through triangulation; each scan produces one profile, and the motion of the conveyor builds a three-dimensional surface. Commercial laser profilers from LMI Technologies, Keyence, Cognex and others are built on this principle. Fringe projection, using shifted sinusoidal patterns, extends it to full-field 3D in a single station. Structured light turns problems that are ambiguous in brightness into problems that are explicit in height: a dent becomes a measured depression, a missing component becomes a measured absence, and the part's colour becomes irrelevant. Photometric stereo, sometimes called shape from shading in its simpler forms, captures several images of the same stationary or synchronised part, each lit from a different direction, and computes the surface orientation at every pixel from how its brightness changes between images. The output is a map of surface normals or curvature that suppresses colour and printed content and amplifies three-dimensional texture. Commercial implementations such as the shape-from-shading tools in several vendors' software have made the technique practical for inspecting embossed codes, tyre sidewall markings, and fine surface defects on leather, plastics and paint. Table 1 summarises the principal techniques, the contrast mechanism each relies on, and the defects for which each is a common first choice. Table 1. Principal illumination geometries and what they reveal. Technique Light position Contrast mechanism Typical first choice for Bright-field Near the lens axis, at the mirror angle Surface tilt scatters light away; defects dark Dents and debris on flat shiny parts Dark-field Low grazing angle around the part Raised or cut features scatter light in; defects bright Scratches, engraving, edge chips, dust Backlight Behind the part, facing the camera Silhouette or transmission Dimensions, holes, fill level, glass bubbles Diffuse dome Hemisphere around the part Glare suppressed; colour and print emphasised Labels on foil, print on curved or crinkled surfaces Coaxial Along the optical axis via beam splitter Flat surfaces bright; tilted areas dark Wafers, polished faces, laser marks on metal Structured light Projected line or pattern at an angle Height measured by triangulation Dents, warpage, missing parts, volume No table can capture the full design process, because real parts rarely have a single surface type and real defects rarely have a single signature. A machined aluminium housing may have a polished sealing face, a cast exterior, a printed label and threaded holes, each needing a different light. The practical answer is often multiple lighting stations or, increasingly, a single station that fires several lights in rapid sequence and captures an image under each, letting the downstream algorithm choose or combine them. Wavelength, filters and polarisation The geometry of light is the first design variable. Its spectrum is the second. Different materials absorb and reflect different wavelengths, and choosing a light colour that maximises the difference between feature and background is one of the cheapest ways to increase contrast. A long-standing rule of thumb is that a coloured feature appears bright under light of its own colour and dark under light of its complementary colour. Red print on a white label nearly vanishes under red light and stands out strongly under green or blue. This is why industrial lights are sold in narrow-band versions — commonly around 470 nanometres for blue, 525 for green, 630 or 660 for red — rather than only as white light, and why monochrome cameras paired with a well-chosen light colour frequently outperform colour cameras on contrast-limited tasks. Wavelength also affects how light interacts with surface texture. Shorter wavelengths scatter more strongly from fine surface roughness, so blue light emphasises fine scratches and texture, while longer wavelengths are more forgiving of it. Near-infrared light, typically around 850 nanometres, passes through many dyes and inks that are opaque in visible light; it can see through a coloured plastic cap to check the seal beneath, read a code printed under a decorative overprint, or suppress the variation in a printed background so that a dark code stands out. Ultraviolet light makes fluorescent materials glow, which is exploited deliberately when adhesives, lubricants or security inks are formulated with fluorescent tracers so that their presence and coverage can be inspected. Beyond the near infrared lies the short-wave infrared band, roughly 900 to 1,700 nanometres, accessible with indium gallium arsenide sensors. SWIR imaging can reveal moisture, because water absorbs strongly around 1,450 nanometres; it can distinguish materials that look identical in visible light, such as different plastics in recycling streams; and it can see through some packaging films and silicon. SWIR cameras remain expensive compared with silicon sensors, but prices have fallen substantially and they are now routine in food sorting and recycling. Optical filters on the lens complete the spectral design. A bandpass filter matched to the light source passes the system's own illumination while blocking most other light. This is one of the most important defences against ambient light, a theme Chapter 7 develops: if the inspection uses 660-nanometre red light and the lens carries a narrow red bandpass filter, the fluorescent lights overhead and much of the sunlight through a window contribute far less to the image. Polarisation is the third spectral-domain tool. Specular reflection from non-metallic surfaces is partially polarised, while diffuse reflection is largely depolarised. Placing a linear polariser over the light and a second, crossed polariser over the lens therefore removes much of the specular glare while preserving the diffuse image — a standard technique for reading labels through shrink-wrap, inspecting glossy printed packaging, or imaging wet food products. Polarisation-sensitive sensors, such as Sony's Polarsens line, which places a micro-polariser array over the pixels so that each group of four pixels samples four polarisation angles, can compute degree and angle of polarisation in a single shot. They reveal stress in transparent plastics and glass, and surface orientation on shiny parts, in ways that ordinary intensity images cannot. Strobing, exposure and timing Light also has a temporal dimension. On a moving conveyor, the time over which light reaches the sensor determines motion blur, a constraint Chapter 3 quantifies. Short exposures need intense light, and the standard industrial answer is to strobe: rather than leaving an LED on continuously, the controller drives it with a short, high-current pulse synchronised to the camera's exposure. Because the LED is on only a small fraction of the time, it can be overdriven well beyond its continuous rating without overheating. Manufacturers of industrial lighting publish the permitted overdrive for each product as a function of pulse width and duty cycle, and the brightness gains available this way are large. Strobing brings a further benefit. If the exposure is short and the strobe is intense, ambient light contributes only a small fraction of the energy collected during the exposure, because the ambient light is weak relative to the strobe and is integrated for only a few microseconds. A system exposing for 50 microseconds under a strobe that is many times brighter than the room lighting is largely immune to changes in that room lighting. Combined with a bandpass filter, this is often enough to make a vision station indifferent to whether the factory lights are on. Timing must be precise. The strobe pulse, the camera exposure and the part's position must align within microseconds. Industrial lighting controllers from specialists such as Gardasoft, and lighting manufacturers' built-in drivers accept the camera's trigger or strobe output signal and generate a current pulse of programmable width and delay. Multi-light sequencing — firing a dark-field ring, then a backlight, then a coaxial light for three consecutive exposures as the part passes — depends on this same control. Designing light in practice Lighting design remains more empirical than theoretical. The physics constrains the options, but the particular combination of material, finish, colour, geometry and defect on a given part is rarely predictable from first principles. Experienced integrators therefore build a lighting lab: a bench with an assortment of ring lights, bars, domes, coaxial units and backlights in several colours, a range of lenses and cameras, polarisers and filters, and a collection of good and defective sample parts. The first days of a project are spent there, trying combinations and capturing images, before any software is written. The criterion for success at this stage is simple to state: can a person looking at the raw image, without any processing, see every defect in the catalogue clearly and distinguish it from the acceptable variation? If the answer is yes, the algorithm will probably succeed. If the answer is no, the algorithm will almost certainly struggle, and data collection should not begin until the lighting has been improved. This test is crude but it captures something essential. Neural networks are powerful at learning patterns that people struggle to articulate, but they are not magic: they work with the information the sensor captured. When the defect's contrast is low relative to the noise and the natural variation of the background, a model must learn a fragile boundary from many examples, and that boundary will break when anything changes. A quantitative version of the same test measures contrast-to-noise ratio: the difference in mean grey level between defect and surrounding background, divided by the standard deviation of the background. A ratio of several units makes detection straightforward; a ratio close to one means the defect is buried in the background's variation. Measuring this for each defect type under each candidate lighting arrangement turns subjective judgement into a comparison that can be documented. Mechanics matter as much as optics. Lights and cameras must be mounted rigidly so that their geometry does not drift with vibration or maintenance. Enclosures that shroud the inspection area from ambient light are cheap and effective. Lens and light surfaces must be protected from dust, oil mist and spray, and the maintenance schedule must include cleaning them. A light that is moved two centimetres by a technician cleaning the conveyor can change the image enough to invalidate the model trained on the original geometry, a problem Chapter 7 returns to. The lesson for the rest of the system The deep learning revolution in vision has sometimes encouraged a belief that careful image acquisition is old-fashioned, that a sufficiently large model trained on sufficiently many images can learn to see past poor lighting. In consumer applications, where the camera cannot be controlled, there is some truth in this. In a factory, where the camera and light can be controlled completely, it is a costly mistake. Every unit of contrast the optics provide is a unit the model does not need to learn, and a model that does not need to learn something cannot forget it when conditions change. Light is the first algorithm in the pipeline, and it is the one that runs at the speed of light, consumes no compute, and never drifts unless someone moves it. CHAPTER 3 Optics, Sensors and the Moving Part Illumination makes a defect produce contrast. The lens and sensor must then capture that contrast with enough resolution, sharpness and signal to be measured, and they must do it while the part is moving. This chapter covers the arithmetic of resolution, the choice of lens, the properties of industrial sensors, the interfaces that carry images to a computer, and the timing chain that ties an image to a physical part on a conveyor and a reject decision back to that same part. Each of these is a place where a project can fail before a single model has been trained. Resolution, field of view and the pixel budget Chapter 1 introduced the basic calculation: the pixel footprint on the part, which is the field of view divided by the number of pixels across it, must be small enough that the smallest defect of interest covers several pixels. The practical figure depends on contrast and on the method. A high-contrast feature under backlight can be located to a fraction of a pixel by edge-fitting algorithms, and gauging tools routinely achieve sub-pixel repeatability. A low-contrast cosmetic defect that must be recognised by its shape and texture needs considerably more: a scratch three pixels wide gives a model little to work with, while one ten pixels wide carries recognisable structure. For learned defect detection, many integrators aim for the smallest defect to span at least five to ten pixels in its shortest dimension. The field of view is set by the part and its presentation. It must cover the area to be inspected plus a margin for positional variation on the conveyor. A part that may arrive anywhere within a 30-millimetre band needs a field of view that accommodates the whole band, which wastes pixels on empty conveyor. This is one reason mechanical guides and singulation upstream of a vision station pay for themselves: tighter presentation means a smaller field of view and finer resolution from the same sensor. When the pixel budget exceeds what one sensor offers, there are three options. A larger sensor — industrial area cameras now range from a fraction of a megapixel to well over a hundred — increases resolution at the cost of frame rate, data volume and lens expense. Multiple cameras can tile the field of view, at the cost of calibration and synchronisation. Or a line-scan camera can build the image one row at a time as the part moves past, which is often the most economical way to inspect long, continuous or cylindrical objects at high resolution. Lenses, depth of field and telecentricity A lens maps the part onto the sensor, and its choice involves several coupled quantities: focal length and working distance, which together set the magnification; aperture, which sets brightness and depth of field; and resolving power, which sets how much fine detail survives the trip. Depth of field matters more in industry than many newcomers expect. Parts vary in height, sit at slightly different positions on the conveyor, and may have features at several depths. If the depth of field is shallower than this variation, some images will be out of focus. Depth of field increases as the aperture is closed — a higher f-number — but closing the aperture reduces light, which then requires longer exposure or more intense strobes, and beyond a certain point diffraction begins to soften the image. For small pixels, diffraction becomes a real constraint at f-numbers that a newcomer might otherwise choose for depth of field. There is a genuine trade-off here, and its resolution depends on the specific sensor, lens and part. The resolving power of the lens must match the sensor. A lens designed for large pixels will blur detail on a sensor with small ones, so that a 20-megapixel camera behind it delivers the effective resolution of a much smaller sensor. Lens manufacturers publish modulation transfer function curves showing contrast as a function of spatial frequency, and a sound specification checks that the lens retains useful contrast at the sensor's Nyquist frequency across the whole field, not only at the centre. For gauging, conventional lenses have a specific problem: perspective. Objects closer to the lens appear larger, so a part that sits a millimetre higher than its neighbour appears slightly bigger, and the sides of a tall feature are visible at the edges of the field. Telecentric lenses solve this by accepting only rays parallel to the optical axis, which makes magnification independent of distance within the lens's telecentric range. A part measured by a telecentric lens appears the same size whether it sits slightly high or slightly low, and the walls of a bore are not seen at an angle. The cost is that the front element of the lens must be at least as large as the field of view, which makes telecentric lenses large, heavy and expensive for anything but small fields. Where dimensional accuracy is the point of the inspection, they are often worth it. Sensors and shutters Almost all industrial cameras now use CMOS image sensors. The properties that matter most are pixel size, quantum efficiency, read noise, full-well capacity, dynamic range and shutter type. The EMVA 1288 standard, maintained by the European Machine Vision Association, defines how these properties should be measured and reported, and reputable camera manufacturers publish EMVA 1288 data sheets. Comparing cameras on these measured values rather than on marketing claims is one of the simplest ways to avoid a poor choice. Shutter type is decisive for moving parts. A global shutter exposes every pixel in the sensor over the same time interval, so a moving object is captured as it was at one instant. A rolling shutter exposes rows in sequence, so different rows see the object at slightly different times; a moving part appears sheared or distorted, and a strobe that fires during the readout illuminates only part of the frame. Rolling shutter sensors dominate consumer devices because they are cheaper and can achieve smaller pixels with lower noise, but for inline inspection of moving parts a global shutter is the default choice. Sony's Pregius line of global-shutter sensors, and comparable sensors from Gpixel, onsemi and others, underlie a large share of industrial area cameras. Dynamic range becomes important when a scene contains both very bright and very dark regions, such as a shiny metal part with dark recesses, or a dark label on a reflective can. A sensor with limited dynamic range must choose between saturating the bright regions and losing the dark ones in noise. Some industrial sensors offer high-dynamic-range modes that combine multiple exposures or use dual-gain readout. Often, though, the better answer is the one Chapter 2 recommends: change the lighting so that the scene no longer spans such a range. Monochrome sensors remain the norm for most inspection. A colour sensor uses a mosaic of filters so that each pixel sees only red, green or blue, and colour is reconstructed by interpolation, which reduces effective resolution and sensitivity. Unless colour itself carries the information — sorting by colour, checking that the right coloured cap is fitted, detecting discolouration — a monochrome sensor with an appropriately coloured light usually gives a better image. Motion blur and exposure A moving part smears across the sensor during the exposure. The size of the smear in pixels is the distance the part travels during the exposure divided by the pixel footprint. If a conveyor moves at one metre per second and each pixel covers 0.1 millimetres of the part, the part moves one pixel's width in 100 microseconds. An exposure of 100 microseconds produces a blur of one pixel, which is usually acceptable; an exposure of a millisecond produces ten pixels of blur, which is not. This arithmetic is simple, but it ties together the decisions of the previous chapter and this one. Finer resolution, which the defect demands, shortens the allowable exposure. A shorter exposure collects less light, which demands a brighter source, which usually means a strobed LED overdriven for a few microseconds. Closing the aperture for depth of field reduces light further. A short calculation at the start of a project establishes whether the light budget closes at all. def imaging_budget(fov_mm, pixels, speed_m_s, max_blur_px=1.0, min_defect_mm=0.2): footprint_mm = fov_mm / pixels # mm per pixel on the part defect_px = min_defect_mm / footprint_mm # size of smallest defect in pixels max_exposure_us = max_blur_px footprint_mm / (speed_m_s 1000) * 1e6 line_rate_hz = speed_m_s * 1000 / footprint_mm # for a line-scan camera return footprint_mm, defect_px, max_exposure_us, line_rate_hz # 250 mm field, 4096 px across, conveyor at 1.5 m/s print(imaging_budget(250, 4096, 1.5)) # about 0.061 mm/px, defect about 3.3 px, exposure under about 41 us, line rate about 24.6 kHz The example shows the kind of conclusion such a calculation produces: a 4,096-pixel sensor over a 250-millimetre field gives a 0.2-millimetre defect only about three pixels, which is marginal for a learned detector, and the allowable exposure is only around 40 microseconds. Either the field must shrink, the sensor must grow, or the defect specification must be reconsidered. Line-scan imaging and encoders A line-scan camera has a sensor one or a few pixels tall and thousands of pixels wide. It captures one line at a time as the object moves past, and software stacks the lines into an image. Line scan suits continuous materials — steel strip, paper, film, textiles, glass sheet — and objects that can be rotated in front of the camera, such as cans, bottles and cylindrical machined parts, whose entire circumference can be unrolled into a flat image. It also suits very wide fields, because sensors of 8,000 or 16,000 pixels are standard, and lighting needs to illuminate only a narrow line rather than a whole area, which makes intense illumination easier to achieve. The essential discipline of line scan is synchronisation. Each line must correspond to the same distance of travel, or the image will be stretched or compressed along the direction of motion. The usual solution is a rotary encoder attached to the conveyor or a roller, which emits pulses at fixed increments of travel; the camera triggers one line per pulse, or per a configured number of pulses. As the conveyor speeds up or slows down, the line rate follows, and the image geometry stays correct. The required line rate is the conveyor speed divided by the pixel footprint along the direction of travel: one metre per second at 0.1 millimetres per line requires 10,000 lines per second, and modern line-scan cameras reach rates of several hundred thousand lines per second. Time delay integration sensors extend the line-scan principle by having many rows that pass the charge or signal from row to row in step with the motion, so that each point on the object is exposed many times and the results are summed. This multiplies the effective exposure without blur, which is invaluable in light-starved applications such as fast web inspection or fluorescence imaging. Getting images into the computer Images must travel from the camera to the processor, and the interface determines bandwidth, cable length, latency and cost. The machine vision industry has standardised these interfaces through bodies including the Association for Advancing Automation in North America, the European Machine Vision Association and the Japan Industrial Imaging Association, and the GenICam standard provides a common programming interface so that software can control cameras from different vendors through the same calls. Table 2 compares the main options as they stand in 2026. Table 2. Common industrial camera interfaces. Interface Nominal bandwidth per link Typical cable reach Typical use GigE Vision (1GigE) 1 Gb/s About 100 m on copper Multi-camera cells, long runs to a central PC GigE Vision (10GigE and faster) 10 Gb/s and above About 100 m on copper; further on fibre High-resolution area cameras USB3 Vision 5 Gb/s A few metres passive; longer with active or optical cables Compact single-camera stations CoaXPress 2.x (CXP-12) 12.5 Gb/s per coax; links aggregate Tens of metres High-speed line scan and large sensors MIPI CSI-2 Several Gb/s per lane, multiple lanes Board level; extended by serialiser links Embedded cameras on edge modules Source: A3 (GigE Vision, USB3 Vision), JIIA CoaXPress roadmap, MIPI Alliance; reach figures are typical rather than normative and depend on cable and speed. GigE Vision is the most widely deployed, because Ethernet is cheap, cables are long and many cameras can share network infrastructure. Its standard transport sends image packets that the host must reassemble, which consumes CPU and can drop packets under load; careful network configuration — dedicated network interface cards, jumbo frames, adequate buffers — is needed for reliable multi-camera systems. GigE Vision 3.0, published by A3 in 2026, adds transport over RDMA over Converged Ethernet, letting the network card write image data directly into host memory and reducing CPU load and latency at high data rates. CoaXPress dominates the high end, where very high frame rates or line rates demand bandwidth that Ethernet struggles to deliver deterministically, and CoaXPress 3.0, aimed at 25 gigabits per second per link, was in its final stages of standardisation in mid-2026. On embedded edge modules, MIPI CSI-2 cameras connect directly to the processor's image pipeline, and serialiser links such as GMSL extend them over several metres of cable — an architecture borrowed from automotive cameras. Triggering and tracking The final link in the acquisition chain is the one most often underestimated: knowing which physical part each image shows, and making sure the reject decision acts on that same part. The camera must be triggered when the part is in the right position. The usual sensor is a photoelectric switch — a photo-eye — mounted upstream, whose signal goes to the camera's trigger input directly or through a programmable logic controller. Every element of this chain introduces latency and jitter: the photo-eye's response time, the PLC's scan cycle if it sits in the path, the camera's trigger latency. If the total jitter, multiplied by conveyor speed, is significant compared with the field of view's margin, parts will sometimes be imaged off-centre or partly outside the frame. Wiring the photo-eye directly to the camera's hardware trigger input, rather than routing the signal through software, removes most of the variable delay. Photo-eyes also misfire: a reflective label, a gap in a part, a transparent bottle or a dangling strap can produce a double trigger or a missed one, and robust systems detect these events — by checking that the expected part is present and centred in each image — rather than assuming every trigger is correct. After the decision is made, the reject mechanism, typically an air blast, a pusher or a diverter gate, sits some distance downstream. The part must be tracked from the camera to the reject point. The standard approach is a shift register in the PLC, clocked by an encoder on the conveyor: when the part is imaged, its position is recorded in encoder counts; when the inspection result arrives, it is attached to that position; and when the encoder count shows the part has reached the reject point, the mechanism fires if the result says reject. The inspection result must arrive before the part reaches the reject point, which sets the hard deadline for the entire processing chain — a deadline Chapter 6 treats as the central constraint on edge inference. Tracking errors are a classic source of expensive failures. A conveyor that slips, a part that is knocked out of position after inspection, or a result that arrives a few milliseconds late can cause the system to reject the good part behind the bad one and pass the bad one. Good practice includes a confirmation sensor after the reject point that checks the reject actually happened, and a verification that every imaged part received a result. When either check fails, the line should stop or divert to a quarantine bin rather than guess. The acquisition chain as a whole Taken together, illumination, optics, sensor, interface and triggering form the part of a vision system that converts physical reality into data. Every downstream stage inherits its limitations. An under-resolved defect cannot be recovered by a larger model; a blurred image cannot be sharpened without inventing detail; a mistracked result cannot be corrected by better confidence scores. The engineering in this chapter is unglamorous and largely settled, but it is where the measurable performance of the whole system is first capped, and an inspection that respects these limits can afford a much simpler decision stage than one that ignores them. CHAPTER 4 Learning From Scarce Defects The defining statistical fact of industrial inspection is that defects are rare. A well-run process may produce one defective part in a thousand, or one in a hundred thousand. The defects that occur are not a fixed set of categories but a long tail: a few common types and many that appear a handful of times a year, plus genuinely novel failures that have never been seen before because the process has never failed in that way. Meanwhile, examples of good parts are effectively unlimited. Any learning method used on a production line must be designed around this imbalance. This chapter explains how the imbalance shapes data collection, why unsupervised anomaly detection has become a central technique, what its benchmarks do and do not show, how synthetic defects can supplement real ones, and why the labelling of data deserves the same engineering rigour as the design of the model. The shape of industrial data Consumer computer vision grew up on datasets such as ImageNet and COCO, in which every category has hundreds or thousands of examples and the categories are, by construction, roughly balanced. A model trained on such data learns to distinguish known classes. Industrial inspection rarely looks like this. A typical project starts with tens of thousands of images of good parts, a few dozen to a few hundred images of the common defect types, and a handful of examples, or none, of the rare ones. The defect samples that exist are often skewed toward whatever failure happened to occur during the collection period — a run of bad castings from one mould, a week when a print head was clogging — and may not represent the defects that will appear next year. Three consequences follow. First, a purely supervised classifier trained to recognise named defect types will be blind to defect types it has not seen. A model trained on scratches and dents may pass a part with a crack, because nothing in its training taught it that cracks are bad. Second, the good class must be represented as carefully as the defect classes, because the main risk of false rejects comes from good parts that look unlike the good parts in training: a different batch of raw material, a different shift's surface finish, a change in lighting. Third, evaluation is statistically difficult. With only thirty examples of a defect type, the uncertainty in any measured detection rate is large, and claims of 99 per cent recall based on such numbers are not credible. The practical response begins with data collection. Good-part images should be gathered over a long enough period, and across enough sources of variation — several material batches, shifts, machines, seasons of ambient temperature and humidity, and maintenance cycles — that the training set represents what "normal" really looks like on this line. A dataset collected in a single afternoon represents that afternoon. Defect images should be collected deliberately rather than opportunistically, by retaining physical defective parts in a library and re-imaging them under the production setup, by asking the process engineers to produce controlled defects where that is safe and economical, and by capturing every rejected part during pilot runs. Splitting data for training and evaluation needs equal care. A random split of images into training and test sets can leak information when several images come from the same part, the same batch or the same hour: the model learns features of that batch, and the test set rewards it for doing so. Splitting by time — training on earlier production and testing on later — gives a much more honest estimate of how the model will behave in the future, which is the only estimate that matters. Anomaly detection: learning what normal looks like If defects are rare and unpredictable, an obvious alternative to learning them is to learn only what good parts look like and to flag anything that deviates. This is unsupervised anomaly detection, sometimes called one-class learning, and over the past several years it has moved from research curiosity to a standard tool in commercial inspection software. The modern era of the field began with a dataset. In 2019, Paul Bergmann and colleagues at MVTec Software published the MVTec Anomaly Detection dataset at the Conference on Computer Vision and Pattern Recognition. It contains 5,354 high-resolution images across fifteen categories of objects and textures — bottles, cables, capsules, carpets, hazelnuts, metal nuts, screws, transistors, wood, zippers and others — with training sets containing only defect-free images and test sets containing both good images and images with a variety of real defects, annotated at pixel level. For the first time, researchers had a common, realistic industrial benchmark, and progress on it was rapid. Amazon's VisA dataset, published by Yang Zou and colleagues in 2022, added 10,821 images across twelve subsets, including complex structures such as printed circuit boards and multiple instances per image. Methods for anomaly detection fall into a few families. Reconstruction-based methods train a model, such as an autoencoder, to reproduce good images; at test time, regions the model cannot reconstruct well are flagged as anomalous. The idea is intuitive, but in practice such models often reconstruct defects too well, because the network generalises beyond what it was trained on, and their accuracy on benchmarks fell behind other families. Feature-embedding methods proved more effective. They take a network pretrained on a large general dataset such as ImageNet, pass good training images through it, and record the distribution of intermediate features at each location in the image. At test time, each patch of a new image is compared against this record of normal features; patches whose features are far from anything seen in training are anomalous. PaDiM, published by Thomas Defard and colleagues in 2020, modelled the features at each position as a Gaussian distribution. PatchCore, published by Karsten Roth and colleagues at Amazon at CVPR 2022, stored a memory bank of representative normal patch features, reduced in size by a coreset selection procedure, and scored each test patch by its distance to the nearest neighbour in the bank. PatchCore reported image-level detection performance on MVTec AD with an area under the ROC curve of up to 99.6 per cent, and its simplicity and accuracy made it a default baseline for practitioners. Student-teacher methods train a small "student" network to imitate the features of a pretrained "teacher" on normal images only; on anomalous regions the student fails to imitate, and the discrepancy marks the defect. EfficientAD, published by Kilian Batzner, Lars Heckler and Rebecca König of MVTec at the Winter Conference on Applications of Computer Vision in 2024, combined a lightweight student-teacher design with an autoencoder for logical anomalies and reported detection at a latency of about two milliseconds and a throughput of around six hundred images per second on a modern GPU, while matching or exceeding the accuracy of slower methods. Normalising-flow methods such as FastFlow and CFlow-AD model the density of normal features explicitly and score test features by their likelihood. These methods share important practical properties. They need only good images to train, often a few hundred. They produce an anomaly map that localises the suspected defect, which helps operators understand rejections. And they can, in principle, detect defect types never seen before. Intel's open-source Anomalib library, described in a 2022 paper by Samet Akcay and colleagues, collects implementations of most of these methods behind a common interface, with export to inference runtimes, and it has made the techniques accessible to engineers who do not want to reimplement research code. What the benchmarks hide By 2023 the leading methods were reporting image-level AUROC scores above 99 per cent on MVTec AD, and the benchmark was effectively saturated. This did not mean industrial anomaly detection was solved. It meant the benchmark had stopped distinguishing between methods. MVTec's own researchers made the point by publishing a harder successor. The MVTec AD 2 dataset, released in 2025 by Lars Heckler-Kram and colleagues, contains more than 8,000 high-resolution images across eight scenarios chosen to reflect difficulties practitioners actually meet: transparent and overlapping objects, dark-field and backlit illumination, high variance in the normal data, and extremely small defects. Crucially, it includes test sets captured under changed lighting conditions, to measure robustness to the kind of distribution shift discussed in Chapter 7. The authors reported that state-of-the-art methods achieved below 60 per cent average AU-PRO, a localisation metric, on the new benchmark. The gap between near-perfect scores on the old benchmark and modest scores on the new one is a fair measure of the distance between research conditions and factory conditions. Several limitations of anomaly detection explain that gap. The first is that "anomalous" and "defective" are not the same thing. An anomaly detector flags anything unusual, including harmless variation that the training set happened not to include: a slightly different shade of a material from a new supplier, a smear of permitted lubricant, a reflection from a repositioned light. On a production line, such benign novelties are far more common than true defects, so an anomaly detector without care will produce many false rejects. The training set's coverage of normal variation is therefore the main lever on performance, which is why the data collection practices above matter. The second limitation is that AUROC, the standard benchmark metric, summarises performance across all possible thresholds, while a production system operates at one. A method with excellent AUROC can still be unusable if, at the threshold needed to catch the required fraction of defects, it rejects too many good parts. Because defects are rare, even a small false-positive rate translates into many false rejects. Metrics that focus on the low false-positive region of the curve, or simply reporting the false reject rate at a required detection rate, are far more informative for deployment. The third limitation is threshold selection itself. A supervised model can set its threshold using validation examples of both classes. An anomaly detector trained only on good parts must set its threshold using the distribution of scores on held-out good parts, and perhaps a few defect examples. Choosing a threshold at, say, the 99.9th percentile of good scores fixes the false reject rate but says nothing about recall unless defects are available to test against. In practice, every production deployment of anomaly detection still needs a set of real defect examples for validation, even if none were needed for training. The fourth limitation concerns logical anomalies — parts in which every local patch looks normal but the arrangement is wrong, such as a connector with two pins swapped, a kit missing one component, or a label applied to the wrong product. Patch-based methods are blind to these by design. The MVTec LOCO dataset, published by Bergmann and colleagues in 2022, was built specifically to test them, and dedicated components such as the autoencoder branch in EfficientAD were introduced in response. Many logical checks remain easier to implement with rules or with a supervised detector that counts and locates components. The practical conclusion is that anomaly detection is best used as a safety net and a bootstrap, not as the sole decision. It is excellent for getting a system running before defect examples are available, for catching novel defect types that a supervised model would miss, and for flagging images for human review. As defect examples accumulate, supervised models trained on them, discussed in Chapter 5, usually offer better precision on the known types, and the two are often run together: a supervised detector for the catalogued defects, an anomaly detector to catch what the catalogue does not anticipate. Zero-shot and few-shot methods Large vision-language models have introduced another option: describing normality and abnormality in words rather than examples. WinCLIP, published by Jongheon Jeong and colleagues at CVPR 2023, used OpenAI's CLIP model to compare image windows against text prompts such as "a photo of a damaged object" and "a photo of a flawless object", achieving useful zero-shot anomaly detection on MVTec AD without any training images from the target product. AnomalyCLIP, presented at the International Conference on Learning Representations in 2024, learned object-agnostic text prompts that generalise across product categories. More recent multimodal models can be asked directly whether an image shows a defect and why. These approaches are valuable for rapid prototyping, for triage, and for generating initial labels, and their zero-shot accuracy has improved steadily. They are not yet a substitute for a model trained on the specific product and lighting of a specific line when the requirement is a fraction of a per cent of false rejects at millisecond latency. Their size also makes them difficult to run within the time budgets of fast lines on edge hardware. Their most productive current role is upstream of the production model: helping engineers find, sort and label images faster. Synthetic defects When real defects are scarce, an obvious idea is to manufacture artificial ones. The idea has a long history and several current forms. The simplest approach pastes or blends defect patches onto good images. CutPaste, published by Chun-Liang Li and colleagues at Google at CVPR 2021, cut a patch from a good image and pasted it elsewhere in the same image, creating a local irregularity that a network could learn to recognise as anomalous; this self-supervised task produced representations useful for detecting real defects. DRAEM, published by Vitjan Zavrtanik, Matej Kristan and Danijel Skočaj at the International Conference on Computer Vision in 2021, generated synthetic anomalies by blending texture patterns into good images within random masks, and trained a model to both reconstruct and segment them. Copy-paste augmentation in supervised detection, where real defect cut-outs are pasted onto new backgrounds, is a standard technique for multiplying the useful size of a small defect set. Physically based rendering offers more control. Using a CAD model of the part, a description of its material and a model of the lighting, a renderer can produce images of the part with defects of specified type, size, position and orientation. Domain randomisation, introduced by Josh Tobin and colleagues in 2017 for robotics, varies lighting, texture and camera parameters widely in rendered data so that a model trained on it treats the real world as one more variation. Rendering is most effective for geometric defects — missing features, deformations, misassemblies — and less reliable for subtle surface defects, whose appearance depends on microscopic material properties that are hard to model. Generative image models, including diffusion models, can now synthesise realistic defects conditioned on masks or text, and research and commercial tools have applied them to industrial augmentation. Their realism is impressive, but realism to a human eye is not the same as fidelity to the physics of the specific camera and light. A synthetic scratch that looks convincing may differ from a real one in its noise characteristics, edge profile or interaction with the illumination, and a model can learn to detect those differences rather than scratches. The rule for all synthetic data is the same: use it to train, never to validate. Synthetic defects can enlarge and diversify a training set, but the evidence that a system catches real defects must come from real defects. A project that reports high recall on synthetic test defects has measured its generator, not its inspection. Labels are measurements too Supervised learning requires labels, and in industrial inspection labels are produced by people making the same fallible judgements Chapter 1 described. A label is a measurement with its own error, and its error propagates into both training and evaluation. The work of the data-centric AI movement, associated particularly with Andrew Ng and his company Landing AI, which built its LandingLens platform around industrial inspection, emphasised a point practitioners had long known: on small industrial datasets, improving label consistency often helps more than changing the model. If one labeller draws tight boxes around scratches and another draws loose ones, or one labels faint marks as defects and another does not, the model receives contradictory instructions and its performance plateaus. A labelling guide, written with reference to the defect catalogue and illustrated with boundary examples, is the cheapest performance improvement available. So is periodic review in which several labellers label the same images and their disagreements are discussed and resolved into clearer rules. Annotation tools have improved substantially. Meta's Segment Anything Model, released in 2023, and its successor SAM 2 in 2024, can produce a pixel-accurate mask from a single click, which turns segmentation labelling from tedious polygon-drawing into a quick review task. Anomaly maps from an unsupervised model can pre-highlight candidate defects for labellers to accept, reject or refine. Active learning — selecting the images the current model is least certain about for labelling next — concentrates human effort where it improves the model most. The deeper point is that the label set defines the system's notion of a defect more concretely than any written specification. Whatever labellers consistently mark as defective, the model will learn to reject; whatever they inconsistently mark, the model will treat inconsistently. Governing the labels — who produces them, under what guide, with what review — is governing the specification itself. Chapter 9 returns to this when it discusses how a system is validated and how changes to it are controlled. Data as the durable asset Models change quickly. The architectures that lead benchmarks this year will be replaced, and a well-run inspection programme expects to retrain or replace its models several times over a system's life. What persists is the data: the curated archive of good images across years of normal variation, the library of real defects with consistent labels, the boundary samples that define where the threshold sits, and the record of which images the system got wrong. An organisation that treats that archive as a durable engineering asset, versioned and governed like drawings and specifications, can move to new models with confidence. One that treats data as a by-product of model training will find itself starting over every time. CHAPTER 5 Detectors, Segmenters and the Operating Point Once defect examples exist in useful numbers, supervised models become the workhorses of learned inspection. They are trained on labelled images to recognise, locate and often outline specific defect types, and on the defects they know they are usually more precise than anomaly detectors. This chapter covers how to frame the learning task, the model families that dominate industrial use, the particular problem of small defects in large images, and — most importantly — how to turn a model's continuous scores into an accept-or-reject decision that reflects the costs set out in Chapter 1. Choosing the task Three supervised formulations are common, and the choice among them shapes everything that follows. Image classification assigns a label to the whole image: good, or one of several defect classes. It is the simplest to label and to train, and it suits problems in which the region of interest is small and well-aligned, such as a cropped image of a single solder joint or a single capsule. Its weakness is that it says nothing about where the defect is or how big it is, which makes it hard to apply size-based acceptance criteria, hard to explain decisions to operators, and prone to learning shortcuts — a classifier may learn that images from a particular fixture are more often defective, rather than learning the defect. Object detection predicts bounding boxes with class labels and confidence scores. It localises defects, counts them, and handles images in which several defects or several parts appear. Detection is the natural formulation for presence and absence of components, for counting, and for defects that have a compact extent, such as chips, dents or foreign objects. Bounding boxes are, however, a poor description of elongated or irregular defects: a diagonal scratch's box is mostly background. Semantic or instance segmentation predicts a class for every pixel. It is the most expensive to label but the most informative, because a mask allows the system to measure the defect directly — its length along the path, its area, its maximum width — and to compare those measurements with the numeric criteria of the defect catalogue. When the specification says that scratches shorter than five millimetres are acceptable, a segmentation model followed by a measurement step implements the specification literally, rather than hoping a classifier has learned the length threshold implicitly from labels. Segmentation also tends to learn efficiently from few images, because every pixel provides a training signal. Domen Tabernik and colleagues at the University of Ljubljana, working with the Slovenian manufacturer Kolektor, demonstrated in a 2020 paper in the Journal of Intelligent Manufacturing that a segmentation-based network could detect cracks on electrical commutators accurately from a few dozen defective training samples, and their KolektorSDD datasets became standard benchmarks for surface defect detection. A sound general pattern is to use segmentation or detection to find candidate defects and a deterministic measurement and rule step to decide whether each candidate violates the specification. The learned component answers "where is something that looks like a scratch?"; the rule answers "is this scratch long enough, in the right zone, to reject?". This separation keeps the acceptance criteria explicit, auditable and changeable without retraining: if the customer relaxes the scratch limit from five millimetres to seven, only the rule changes. The model families The most widely deployed real-time detectors in industry descend from YOLO, introduced by Joseph Redmon and colleagues in 2016. YOLO's insight was to predict boxes and classes in a single pass of a convolutional network over the whole image, rather than proposing regions and classifying them in a second stage, which made it fast enough for real-time use. Successive versions from many groups improved accuracy and speed. Since 2020, the Ultralytics company has maintained the most popular line — YOLOv5, YOLOv8, YOLO11 and, in January 2026, YOLO26 — with a training and export toolkit that has made custom detection accessible to engineers who are not machine learning specialists. These models come in sizes from "nano" to "extra large", and most offer detection, segmentation, classification, pose and oriented-box variants from the same framework. Two recent changes are especially relevant to edge deployment. Traditional detectors produce many overlapping candidate boxes and rely on a post-processing step, non-maximum suppression, to remove duplicates. That step runs outside the neural network, is awkward to accelerate on dedicated inference hardware, and adds latency that varies with the number of detections. YOLOv10, from Tsinghua University in 2024, introduced a training scheme that removes the need for non-maximum suppression, and Ultralytics' YOLO26 adopted end-to-end, NMS-free inference along with other simplifications aimed at easier export to edge hardware and faster inference on CPUs. Meanwhile, transformer-based detectors became competitive at real-time speeds: RT-DETR, from Baidu, presented at CVPR 2024, showed that a DETR-style detector, which is inherently free of non-maximum suppression, could match YOLO models in speed and accuracy, and several successors have followed. One practical consideration often surprises engineering teams late in a project: licensing. The Ultralytics models and code are distributed under the GNU Affero General Public License, version 3, with a commercial licence available from the company. For a manufacturer embedding a model in a proprietary inspection product, or an integrator delivering systems to customers, that licence has real implications, and legal review belongs at the start of model selection rather than the end. Alternatives under permissive licences exist, including several YOLO-family models and transformer detectors released under Apache 2.0 terms, and some teams choose them for this reason alone. For segmentation, the U-Net architecture introduced by Olaf Ronneberger and colleagues in 2015 for biomedical images remains a strong default for defect masks, because its encoder-decoder design with skip connections preserves fine spatial detail. Instance segmentation heads attached to real-time detectors serve when separate defect instances must be counted and measured. Whatever the family, transfer learning is standard: models are initialised from weights pretrained on large general datasets and fine-tuned on the industrial data. Pretraining provides general visual features — edges, textures, shapes — that make training on a few hundred defect images feasible. The pretraining domain is very different from industrial images, but the early layers' features transfer well, and the gains in data efficiency are large. Small defects in large images Industrial defects are often tiny relative to the image. A 0.3-millimetre pinhole in a 20-megapixel image of a 300-millimetre panel may span a dozen pixels in an image five thousand pixels wide. Standard detectors downscale input images to a fixed size, commonly 640 pixels on the long side, for speed. At that scale the pinhole occupies less than two pixels and disappears. There are several remedies, usually combined. The first is to avoid downscaling by tiling: divide the full-resolution image into overlapping tiles at the model's native input size, run the model on each, and merge the results. Fatih Akyon and colleagues formalised this approach in 2022 as Slicing Aided Hyper Inference, with an open-source library, and showed substantial gains on small-object benchmarks. Tiling multiplies the inference cost by the number of tiles, which must fit within the time budget, so the region of interest should be cropped as tightly as possible first, typically using a rule-based locator. The second remedy is architectural. Feature pyramid networks, introduced by Tsung-Yi Lin and colleagues in 2017, combine high-resolution, low-level features with low-resolution, semantically rich features, so that small objects are represented at a scale where they still exist. Most modern detectors include such a pyramid, and for very small defects it can help to add an extra high-resolution detection head or remove the lowest-resolution one. The third is to work at the native resolution of the defect by cropping. If the part's location is known from a locator, the inspection regions can be cut from the full-resolution image at a scale where the defect spans enough pixels, and each region processed separately, possibly by different models suited to different surfaces. Class imbalance compounds the small-object problem, because nearly every pixel or candidate location in a training image is background. Focal loss, introduced by Lin and colleagues in the 2017 RetinaNet paper, down-weights the easy background examples so that training concentrates on the rare, hard ones, and it or a similar weighting is now built into most detector training pipelines. Metrics that matter on a line Research on object detection reports mean average precision, averaged across classes and across overlap thresholds between predicted and true boxes. This metric served academic progress well, but it answers a question few factories ask. It rewards precise box placement, which hardly matters for a reject decision, and it averages over all confidence thresholds, while the line runs at one. The metrics that matter are part-level. For each part, did the system reject it or not, and was it truly defective? From these outcomes come the two numbers Chapter 1 asked the project to fix in advance: the detection rate on defective parts, sometimes called recall or catch rate, and the false reject rate on good parts. Both should be reported per defect class where classes have different severities, because catching 99 per cent of cosmetic blemishes and 90 per cent of cracks is very different from the reverse. Because defects are rare, the statistics need care. If a validation set contains 200 defective parts and the system catches all of them, the observed detection rate is 100 per cent, but the true rate could plausibly be lower. A useful approximation known as the rule of three says that if zero events are observed in n independent trials, the upper bound of a 95 per cent confidence interval on the event rate is about 3 divided by n. Zero escapes in 200 defects therefore supports a claim that the escape rate is below about 1.5 per cent, not that it is zero. Demonstrating an escape rate below 0.1 per cent at the same confidence requires zero escapes in about 3,000 defective parts — which, for rare defects, may be more than the plant produces in a year. This arithmetic is humbling, and it argues for collecting and preserving defect samples aggressively from the start of a project. The false reject rate is easier to measure, because good parts are plentiful, but it needs measuring in production conditions rather than on a curated validation set. A model that rejects 0.05 per cent of curated good images may reject 2 per cent of real production, because production contains variations the curated set did not. A shadow-mode period, in which the system runs on the live line and logs its decisions without acting on them, while humans or the existing system make the actual decisions, is the most reliable way to measure the real false reject rate before the system is trusted. Setting the operating point A trained model outputs scores. A decision requires a threshold. Every threshold yields one pair of detection rate and false reject rate, and sweeping the threshold traces the trade-off curve. Choosing the point on that curve is the most important decision in deployment, and it is a business decision informed by technical evidence, not a technical decision informed by convention. The common default of a 0.5 confidence threshold has no particular merit. If the costs of an escape and a false reject can be estimated, the expected cost per part at each threshold is the escape cost multiplied by the defect rate multiplied by the miss rate, plus the false reject cost multiplied by the good rate multiplied by the false reject rate. The threshold that minimises this sum is the economically optimal operating point. In practice the costs are uncertain and often asymmetric by orders of magnitude, so a simpler and more defensible procedure is common: fix the detection rate required by the customer or the risk assessment for each defect class, then choose the threshold that achieves it on validation data with a safety margin, and accept the false reject rate that results. If that false reject rate is unacceptable, the answer lies upstream — better lighting, better resolution, better data — rather than in a lower threshold. import numpy as np def threshold_for_recall(defect_scores, good_scores, target_recall=0.995): # highest threshold that still catches target_recall of known defects t = np.quantile(defect_scores, 1.0 - target_recall) false_reject_rate = np.mean(good_scores >= t) return t, false_reject_rate Model scores are often poorly calibrated: a score of 0.9 does not mean the defect is present with 90 per cent probability. Chuan Guo and colleagues showed in 2017 that modern deep networks are systematically overconfident, and that simple post-hoc corrections such as temperature scaling improve calibration substantially. Calibration matters whenever scores from different models or classes are combined, or whenever the score is shown to operators as if it were a probability. Thresholds should normally be set per defect class rather than globally. A crack in a pressure-bearing component and a cosmetic smudge on a non-visible face carry very different escape costs, and a single threshold applied to both either over-rejects smudges or under-rejects cracks. Per-class thresholds let each defect type sit at the operating point its severity deserves, and the defect catalogue's severity classes from Chapter 1 provide the natural grouping. When a supervised detector and an anomaly detector run together, as Chapter 4 recommended, the combination rule also needs to be explicit. The simplest and most common rule rejects a part if either model's score exceeds its threshold, which preserves the anomaly detector's role as a safety net for unknown defects at the cost of adding its false rejects to the detector's. Each model's threshold must then be set with the other in mind: the combined false reject rate is roughly the sum of the two individual rates where their errors do not overlap, and a budget that was comfortable for one model alone can be exceeded by two. Measuring the combined decision, rather than each model separately, on the validation set and in shadow mode is the only reliable way to know what the system as a whole will do. The three-way decision A binary accept-or-reject decision forces every uncertain case into one of two bins. Many successful systems add a third: uncertain parts go to a review lane, where a person examines them, or to a re-inspection station with a different view. Two thresholds replace one. Scores below the lower threshold pass automatically, scores above the upper threshold reject automatically, and scores between them go to review. The three-way design has several advantages. It lets the automatic thresholds be set conservatively without driving the false reject rate up, because the borderline cases are resolved by a person rather than discarded. It concentrates human attention on a small fraction of parts — often well under one per cent — where human judgement adds most value, which counters the vigilance and prevalence problems of full manual inspection. And it generates a continuous stream of labelled borderline examples, exactly the data most useful for improving the model, as Chapter 9 describes. The design has costs too. It requires a physical path for diverted parts and staff to review them, and it requires discipline to ensure reviewers' decisions are recorded and fed back rather than lost. A review lane that fills up faster than it is cleared becomes a bottleneck, and one whose decisions are never examined becomes a place where systematic errors hide. The width of the uncertain band is therefore a capacity decision as well as a quality one: it must be narrow enough that the review staff can keep up and wide enough that it captures the cases the model genuinely cannot decide. Explaining decisions Operators and quality engineers need to understand why a part was rejected, both to trust the system and to investigate the process. Detection and segmentation models provide natural explanations: the box or mask shows where the defect is, and the measurement step shows why it violates the specification. Displaying the image with the overlay at the station, and archiving it with the decision, is far more useful than any abstract explanation method. For classification models, saliency methods such as Grad-CAM can indicate which regions influenced the decision, but they are approximate and sometimes misleading, and a system whose decisions can only be explained by saliency maps is harder to defend than one built on explicit localisation and measurement. The broader lesson of this chapter is that the model's raw accuracy is only one input to the system's performance. The task formulation determines whether the specification can be implemented literally. The resolution strategy determines whether small defects exist in the model's input at all. The metrics determine whether anyone knows how well the system really works. And the operating point determines what the system actually does. Each of these is a deliberate engineering choice, and each matters as much as the architecture behind it. CHAPTER 6 Inference at the Edge A model that performs well on a workstation has not yet become an inspection system. It must run on hardware installed beside the line, often in an electrical cabinet, often without a fan, within a deadline set by the distance between the camera and the reject gate, and it must exchange signals with the controllers that run the machine. This chapter treats the deployment of detection and anomaly models on edge devices as a real-time engineering problem: why inference belongs at the edge, how to construct and respect a latency budget, how models are compiled and quantised for edge accelerators, what hardware is available, and how the vision system talks to the rest of the factory. Why the edge The case for running inference beside the line rather than in a data centre or public cloud rests on four arguments, each sufficient on its own for most inline applications. The first is latency and its predictability. Chapter 3 showed that the inspection result must reach the reject mechanism before the part does. On a fast line that leaves tens to a few hundred milliseconds for everything. A round trip to a remote server adds network delay that is not only longer but variable, subject to congestion, retransmission and outages that the vision system cannot control. An inspection that is usually on time is not acceptable; it must be on time for every part. The second is bandwidth. A single 5-megapixel monochrome camera running at 30 frames per second produces about 150 megabytes per second of raw data. A cell with four such cameras produces 600. Sending this continuously off-site is expensive and often physically impossible over a plant's uplink. Even compressed, the volume is large, and lossy compression can destroy exactly the fine detail the inspection depends on. The third is availability. A production line must keep running when the internet connection fails, when a cloud provider has an incident, or when the plant's IT network is being maintained. Inspection that depends on remote infrastructure turns every external failure into a line stoppage or a period of uninspected production. The fourth is data governance. Many manufacturers are unwilling to send images of their products and processes outside the plant, for reasons of intellectual property, customer contracts or regulation. Running at the edge keeps raw images local by default and lets the organisation decide deliberately what leaves. None of this excludes central infrastructure. Training, which is computationally heavy but not time-critical, normally happens on servers or in the cloud. Archives of images, dashboards, and fleet management of many edge devices also sit centrally. The decision for each part, however, is made beside the line. The latency budget The deadline for a decision follows from the geometry and speed of the line. If the reject mechanism sits 600 millimetres downstream of the camera and the conveyor runs at two metres per second, the part reaches the reject point 300 milliseconds after it is imaged. Subtract the reject actuator's own response time — a solenoid valve and air jet may need ten or twenty milliseconds to act — and a safety margin, and the vision system might have 200 milliseconds from trigger to result. On a faster line with a closer reject point, the budget can fall to a few tens of milliseconds. That budget must cover every stage, not just the model. Exposure and sensor readout take a few milliseconds. Transfer over the interface depends on image size and bandwidth: a 5-megapixel, 8-bit image over a 1-gigabit link takes roughly 40 milliseconds, which on its own may consume a large part of a tight budget, and is one reason high-speed systems use faster interfaces or process images on the camera itself. Pre-processing — debayering, cropping, resizing, normalisation, tiling — can take several milliseconds on a CPU and much less if done on a GPU or in the camera. Inference takes whatever the model and accelerator dictate. Post-processing, including box decoding, non-maximum suppression where it remains, measurement and rule evaluation, adds more. Finally, the result must be communicated to the controller that drives the reject, through digital I/O or an industrial network, each with its own delay. The figure that matters is not the average latency but the worst case, or at least a very high percentile. A system with a mean latency of 20 milliseconds that occasionally takes 400 will, at the rate of millions of parts, miss its deadline many times. Common sources of latency spikes include operating system scheduling on a general-purpose Linux or Windows installation, memory allocation and garbage collection in interpreted languages, logging that blocks on slow storage, thermal throttling when the processor overheats, contention when several cameras' images arrive at once, and variable post-processing time when the number of detections changes. Each must be measured on the target hardware, under realistic load, at realistic temperature, over long periods. A latency histogram collected over several days of shadow-mode operation is the evidence that the system will meet its deadline. Engineering responses are familiar to anyone who has built real-time systems. Pre-allocate memory and reuse buffers rather than allocating per frame. Pin time-critical processes to dedicated cores and give them real-time scheduling priority. Keep logging and image archiving off the critical path, in separate threads or processes with bounded queues that drop rather than block when full. Avoid batching frames for throughput if it delays individual results. Consider a real-time-capable kernel: the PREEMPT_RT patches, which reduce worst-case scheduling latency on Linux, were merged into the mainline kernel in version 6.12 in 2024 after nearly two decades of development out of tree. And design the system to fail safe: if a result is not ready by the deadline, the controller should treat the part as rejected or divert it for re-inspection, never pass it by default. Compiling and quantising models Models are trained in frameworks such as PyTorch, which are designed for flexibility rather than deployment speed. Before running on edge hardware, a model is usually exported to an intermediate representation, most commonly ONNX, and then compiled by a vendor-specific toolchain into an optimised engine for the target accelerator. NVIDIA's TensorRT does this for Jetson modules and NVIDIA GPUs; Intel's OpenVINO for Intel CPUs, integrated GPUs and neural processing units; Hailo's Dataflow Compiler for its accelerators; and ONNX Runtime provides a common runtime with pluggable execution providers for several of these. The compilers fuse layers, choose efficient kernels for each operation on the specific chip, plan memory, and convert numerical precision. Precision conversion — quantisation — is the largest single source of speed-up and the largest single source of risk. Networks are trained in 32-bit floating point. Converting weights and activations to 16-bit floating point usually costs almost nothing in accuracy and roughly halves memory and compute. Converting to 8-bit integers can roughly double throughput again on hardware with integer tensor units, and many dedicated edge accelerators run only integer arithmetic. Post-training quantisation determines the scale of each layer's activations by running a calibration set of representative images through the network and recording their ranges. Quantisation-aware training simulates the integer arithmetic during training so that the network learns to tolerate it, and generally recovers more accuracy for sensitive models. # Build an INT8 TensorRT engine from an exported ONNX model, # calibrating on images drawn from production (not from the internet) trtexec --onnx=defect_seg.onnx --int8 --fp16 \ --calib=calib_production.cache \ --saveEngine=defect_seg_int8.engine The risk in quantisation is specific to inspection. Aggregate metrics such as mean average precision may change by a fraction of a point after INT8 conversion, and a team may conclude the conversion is harmless. But defects are often subtle, low-contrast signals at the edge of the model's sensitivity, and quantisation error can disproportionately affect exactly those cases. Anomaly detectors that rely on small differences between feature vectors can be especially sensitive. The only valid test is to rerun the full part-level validation — detection rate per defect class and false reject rate — on the quantised engine running on the target hardware, and to compare the score distributions, not just the summary numbers. The calibration set should be drawn from production images, including defects and the full range of lighting and material variation, because a calibration set of easy images sets activation ranges that clip the unusual ones. A compiled engine is tied to the hardware, the compiler version and often the driver version. Upgrading any of them can change the numerical results. This is one of several reasons edge deployments should treat the compiled engine, not just the trained model, as the versioned artefact that is validated and released, a point Chapter 9 develops. The hardware Edge inference hardware for vision falls into several categories: embedded system-on-modules with integrated GPUs, dedicated neural network accelerators added to a host processor, industrial PCs with discrete GPUs, smart cameras with built-in processors, and general-purpose CPUs with integrated neural processing units. Table 3 lists representative devices with their published peak figures. Table 3. Representative edge inference hardware for industrial vision. Device Vendor peak AI throughput Module power range Main software stack NVIDIA Jetson Orin Nano 8GB (Super mode) 67 TOPS (sparse INT8) 7–25 W JetPack, TensorRT NVIDIA Jetson Orin NX 16GB (Super mode) 157 TOPS (sparse INT8) 10–40 W JetPack, TensorRT NVIDIA Jetson AGX Orin 64GB 275 TOPS (sparse INT8) 15–60 W JetPack, TensorRT NVIDIA Jetson AGX Thor (T5000) 2,070 TFLOPS (sparse FP4) 40–130 W JetPack, TensorRT Hailo-8 M.2 accelerator 26 TOPS (INT8) A few watts typical Hailo Dataflow Compiler Hailo-8L M.2 accelerator 13 TOPS (INT8) A few watts typical Hailo Dataflow Compiler Source: NVIDIA Jetson product pages and JetPack 6.2 release notes (2025); NVIDIA Jetson Thor announcement (August 2025); Hailo product documentation. Figures are vendor peaks, not measured inspection throughput. Several cautions apply to such tables. Peak throughput figures are not comparable across vendors or even across product generations, because they assume different numerical formats and, in NVIDIA's case, structured sparsity, which doubles the headline number relative to dense computation. NVIDIA's own JetPack 6.2 documentation lists the dense INT8 figure for the Orin Nano 8GB in Super mode as 33 TOPS — half the headline. The Thor figure is quoted in four-bit floating point, a format aimed largely at generative models. More importantly, the peak figures say little about how fast a specific detection or segmentation model will run, which depends on whether its layers map efficiently onto the accelerator, how much memory bandwidth it needs, how much work remains on the CPU, and how fast images can be moved in and out. The only reliable comparison is to compile the actual model for each candidate and measure end-to-end latency on representative images. The Jetson family dominates embedded industrial deployments, largely because its software stack, built around CUDA and TensorRT, is the same one used for training and server inference, which shortens the path from prototype to production. Jetson modules also include deep learning accelerators separate from the GPU, which can run suitable layers at lower power and free the GPU for pre-processing or a second model. Dedicated accelerators such as Hailo's, and others from vendors including Axelera, DEEPX and the neural processing units in recent Intel and AMD processors, can offer better efficiency per watt for supported model types but demand that the model fit their compilers' supported operations. An unsupported operation either fails to compile or falls back to the host CPU, often at a severe speed penalty. Smart cameras, which integrate sensor, processor and I/O in one housing, are the traditional form factor of industrial vision, and the major vendors now offer models with on-board deep learning: Cognex's In-Sight series, Keyence's IV and VS families, and others. They trade flexibility for integration: the model types and sizes they support are constrained, but they are designed for factory conditions, certified, and configured through tools that automation engineers already know. For many single-station applications they are the most practical option. Environmental conditions are part of the hardware decision. Electrical cabinets beside production lines can reach 50 degrees Celsius or more in summer. Modules that perform well on an open bench can throttle in a sealed enclosure, and the throttling appears precisely as the latency spikes discussed above. Industrial-grade module variants rated for wider temperature ranges and longer availability, fanless enclosures with heat sinks sized for the worst case, and thermal tests under maximum ambient and full load belong in the qualification plan. So does product lifecycle: an inspection system may need to run, and to be repairable with identical spares, for ten years, which favours hardware with published long-term availability commitments. Talking to the machine An inspection result is useful only when the machine acts on it. On a production line, the machine is controlled by a programmable logic controller, and the vision system must integrate with it on its terms. The most time-critical signal is the reject decision. The simplest and most deterministic path is a digital output from the vision system wired to a PLC input or directly to the reject actuator's driver, combined with the encoder-based tracking described in Chapter 3. Many smart cameras and industrial vision controllers include optically isolated digital I/O for this reason. Where the result must carry more information — which defect, which lane, a measured value — it travels over an industrial Ethernet protocol: PROFINET, common in European plants and associated with Siemens; EtherNet/IP, common in North America and associated with Rockwell Automation; EtherCAT, used where very tight synchronisation is needed; or Modbus TCP for simpler integrations. Vision vendors' devices support these natively; a custom Jetson-based system needs a protocol stack or a gateway, and the integration effort should not be underestimated. For non-time-critical data — statistics, recipe changes, model version, health status — OPC UA has become the common language of industrial automation. The VDMA, the German mechanical engineering industry association, developed an OPC UA companion specification for machine vision, published as OPC 40100, which defines a standard information model for controlling vision systems and retrieving their results so that a line controller or a manufacturing execution system can interact with vision systems from different vendors in a uniform way. MQTT, a lightweight publish-subscribe protocol, is widely used to stream telemetry from edge devices to central dashboards and data platforms. Recipe management deserves explicit design. Many lines run several products, and a change of product means a change of model, thresholds, regions of interest and perhaps lighting. The line controller should tell the vision system which recipe to load, the vision system should confirm which recipe is active, and every result should carry the recipe identifier. A product changeover that leaves the previous product's model running is a classic and entirely preventable cause of mass false rejects or, worse, mass escapes. Operating a fleet A single inspection station is a project; a plant with fifty stations, or a company with fifty plants, is a fleet, and it needs to be operated as one. Models and compiled engines must be deployed, versioned, rolled back if a release misbehaves, and kept consistent with the validated configuration. Containerisation, with tools such as Docker on the edge device and an orchestration or device-management layer above it, has become the common pattern, and NVIDIA, the cloud providers and specialised vendors offer fleet-management services for edge AI devices. Security belongs in the same conversation. Edge devices on the plant network are computers running complex software stacks, and they are attack surfaces. The IEC 62443 series of standards for industrial automation and control system security provides the framework most manufacturers use: network segmentation into zones and conduits, hardened device configurations, controlled remote access, signed software updates and a vulnerability management process. A vision station that can be updated over the network by anyone who reaches it is a risk to the whole line. The central idea of this chapter is that edge inference is not a matter of finding a fast enough chip. It is the engineering of a deterministic system that meets a hard deadline for every part, at the worst temperature, after every update, and that tells the machine what to do in a language the machine understands. The model's speed is one term in that equation, and rarely the one that causes trouble. CHAPTER 7 When the Light Changes A vision system is commissioned on a particular day, under particular conditions, with a particular set of parts. It is validated against images captured under those conditions, and it performs well. Then the factory carries on. The sun moves across a skylight. A maintenance crew replaces the overhead lighting. The LED in the ring light loses a few per cent of its output every few thousand hours. Oil mist settles on the lens window. A supplier changes the resin in a plastic housing, and the parts come back a shade glossier. Each change is small. Together, over months, they move the images the system sees away from the images it was trained on, and the system's error rates move with them, usually without anyone noticing until a customer complains or the false reject bin overflows. Lighting variance on the factory floor is the problem practitioners most often cite when inspection systems that worked in trials fail in production. This chapter treats it as an engineering problem with layered defences: remove variation physically where possible, compensate for what remains, train models to tolerate the rest, and detect, rather than silently absorb, the variation that cannot be tolerated. Where the variation comes from It helps to separate the sources of variation by their mechanism, because each calls for a different remedy. Ambient light is the most obvious. Factories mix daylight, which varies by hour, weather and season, with artificial lighting that may be switched by zone and shift. Light from windows and skylights can sweep across a conveyor over the course of a day, and a system commissioned in winter may meet direct sun in summer. Overhead lighting retrofits, such as replacing fluorescent tubes with LED fixtures, change both the intensity and the spectrum of the ambient contribution. Many artificial sources also flicker at twice the mains frequency — 100 hertz in Europe, 120 in North America — and with exposures of a millisecond or less, consecutive images can catch the ambient light at different phases of its cycle, adding frame-to-frame variation that is invisible to the eye. The system's own illumination changes too. LED light output declines with operating hours; manufacturers specify this as the time until output falls to a percentage of its initial value, such as 70 per cent, and in the strobed, overdriven conditions of industrial vision the rate depends heavily on drive current and temperature. LED output also falls as junction temperature rises, so a light that warms up over the first hour of a shift, or sits in a hot summer cabinet, is dimmer than the same light at commissioning. The peak wavelength of LEDs shifts slightly with temperature as well, which matters when narrow bandpass filters or wavelength-dependent contrast are involved. The optical path degrades physically. Dust, oil mist, fingerprints and spatter accumulate on lens windows, protective covers and diffusers, reducing contrast and adding scatter. Mechanical mounts loosen with vibration, or are moved during cleaning and maintenance, changing the lighting angle — and Chapter 2 showed that for specular surfaces a change of a few degrees can turn bright-field into something else entirely. Cameras are replaced after failures, and a new unit of the same model may have a slightly different response, or be installed with different exposure and gain settings. Finally, the parts themselves vary. Raw material batches differ in colour and reflectance; surface finishes vary with tool wear and process settings; coatings, oils and moisture levels change with upstream conditions. From the camera's point of view, a part that is three per cent more reflective is indistinguishable from a light that is three per cent brighter. All of these shift the distribution of input images. Research on robustness has repeatedly shown that deep networks can be sensitive to such shifts. Dan Hendrycks and Thomas Dietterich's ImageNet-C benchmark, published at the International Conference on Learning Representations in 2019, applied systematic corruptions — changes of brightness and contrast, noise, blur, simulated fog and frost — to standard test images and documented substantial accuracy losses in models that performed well on clean images. In the industrial setting, the MVTec AD 2 dataset described in Chapter 4 includes test images under changed lighting precisely because anomaly detectors, which by design flag anything unfamiliar, are especially prone to reading lighting changes as defects. Removing variation physically The most effective defence is to keep variation out of the image in the first place, using the methods introduced in Chapter 2. Shrouding the inspection area with an enclosure, even a simple sheet-metal hood with brush strips where the conveyor passes, removes most ambient light. Where an enclosure is impractical, strobing at high intensity with short exposures reduces the ambient contribution to a small fraction of the collected energy, and a bandpass filter matched to the system's light source blocks most of the ambient spectrum. These three measures together — enclosure, strobe, filter — make many installations practically immune to the factory's own lighting. Exposures that are short relative to the ambient flicker period still sample the flicker, but if the ambient contribution has been reduced to a few per cent of the signal, its flicker becomes negligible. The system's own lights can be stabilised. Industrial lighting controllers can regulate current precisely, and some lights include internal photodiodes that measure their own output and adjust drive current to maintain it as the LEDs age or warm. Running lights below their maximum rated drive leaves headroom for this compensation and extends their life. Thermal management — heat sinks, air flow, avoiding mounting lights against hot machinery — reduces temperature-driven drift. Mechanical rigidity and maintenance discipline address the optical path. Lights and cameras should be mounted on rigid brackets with pinned or keyed positions, so that after removal for cleaning they return to exactly the same place. Protective windows should be specified for easy cleaning and inspected on a schedule. Air knives can keep lens windows clear in dusty or misty environments. The maintenance procedure for the vision station should be written down, include photographs of the correct configuration, and end with a verification step, described below, that confirms the station still produces correct results. Compensating for what remains Some variation survives the physical defences, and much of it can be measured and corrected before the image reaches the model. Flat-field correction addresses spatial non-uniformity — the tendency of illumination to be brighter at the centre of the field than at the edges, and of lenses to transmit less light toward the corners. The procedure is old and reliable: capture a dark frame with no light, capture a flat frame of a uniform reference target under the normal illumination, and correct each subsequent image pixel by pixel by subtracting the dark frame and dividing by the normalised flat frame. The result is an image in which a uniform surface appears uniform. Many industrial cameras perform this correction on board. Repeating the flat-field calibration periodically, or whenever the lighting is serviced, keeps it current as the illumination ages unevenly. Closed-loop exposure control addresses overall intensity drift. A small reference patch — a grey ceramic tile, or a region of the fixture that is always in view and never changes — gives the system a constant against which to measure its own illumination. If the patch's measured grey level falls, the system can raise exposure, gain or light current to compensate, within defined limits. This is more robust than auto-exposure based on the whole image, because auto-exposure responds to the part as well as the light: a dark part would be brightened and a pale part darkened, destroying exactly the differences inspection needs to see. The rule is to correct the instrument against a reference, never against the thing being measured. Software normalisation of image statistics — histogram equalisation, per-image standardisation of mean and variance — is tempting but must be used carefully for the same reason. If a defect changes the overall brightness of an image, normalisation removes part of the signal. If the background varies naturally, normalisation can amplify noise. Normalisation applied consistently to a well-defined region, such as the reference patch or a known background area, is safer than normalisation of the entire image. Training for tolerance No amount of physical control removes all variation, and models should be trained to tolerate the variation that remains. There are two complementary methods: collect real variation, and simulate it. Real variation is best. If the training set includes images captured across seasons, shifts, light ages, material batches and maintenance cycles, the model learns that these differences are not defects. This is the strongest argument for the long data collection periods Chapter 4 recommended, and for continuing to collect images after deployment so that later retraining can include conditions the original data lacked. When a known change is coming — a lighting retrofit, a new material supplier — capturing images under the new conditions before the change takes effect gives the model a chance to learn it in advance. Simulated variation, through data augmentation during training, extends coverage cheaply. Photometric augmentations randomly alter brightness, contrast, gamma, colour balance, noise and blur on each training image, so that the model sees each part under many simulated conditions. The key discipline is physical plausibility: augmentations should reproduce the kinds of change the line will actually produce, at magnitudes it will actually produce them. Brightness changes of ten or twenty per cent are realistic for a drifting LED; a change of eighty per cent is not, and training on it may teach the model to ignore genuine low-contrast defects. Spatially varying augmentation, such as adding a smooth gradient across the image, mimics uneven ambient light or a partially dirty diffuser. Small rotations and translations mimic mechanical mount drift. Augmentation must be applied with the defect signal in mind. If a defect is distinguishable from the background only by a slight difference in brightness, aggressive brightness and contrast augmentation can make defect and background statistically indistinguishable in training, and the model will learn to miss the defect. This is a further reason for the contrast-to-noise analysis of Chapter 2: if the optical design gives defects a robust contrast, the model can be trained with strong augmentation without losing them; if not, the augmentation must stay mild, and the system will be correspondingly fragile. For anomaly detection specifically, the good-image training set must include the lighting variation the system will meet, because anything absent from it will score as anomalous. Some teams maintain separate normal-image memory banks or models for distinct, known operating conditions — for instance, a model for each product colour — and select among them using the recipe, rather than asking one model to cover everything. Detecting what cannot be tolerated Even with every defence in place, some changes will exceed what the system can tolerate. The most important principle of operating a vision system is that such changes must be detected and flagged, not silently absorbed into the decisions. A system whose images have quietly degraded will not report that it is seeing poorly. It will simply make worse decisions. The first line of detection is image quality gating. Before an image reaches the model, simple measurements check that it is fit to be judged: mean brightness of the reference patch within limits; fraction of saturated pixels below a threshold; sharpness, measured for example as the variance of the Laplacian over a textured region, above a threshold; the part present, centred and correctly oriented according to the locator. An image that fails these checks should not be judged at all. The part should be diverted for re-inspection, and the event logged and alarmed. This converts silent degradation into a visible fault. The second line is statistical monitoring of the system's outputs. The reject rate, per defect class and per shift, should be plotted on control charts familiar from statistical process control: Shewhart charts for sudden shifts, and exponentially weighted moving average or cumulative sum charts for slow drifts. A reject rate that climbs from 0.3 to 0.8 per cent over three weeks may reflect a real process problem, or it may reflect a dimming light that is pushing good parts over the anomaly threshold. Either way, someone needs to look. Equally important, and more often forgotten, is a reject rate that falls: a camera that has gone out of focus may stop seeing defects altogether, and the reject rate drops to zero while bad parts flow through. The third line monitors the inputs. The distribution of simple image statistics — mean brightness, contrast, sharpness — can be charted over time. More sensitive is monitoring the distribution of the model's internal representations: compute an embedding for each image, summarise the distribution over each shift, and measure its distance from the training distribution. Rising distance indicates the system is seeing images unlike those it learned from, before that shows up in error rates. For anomaly detectors, the distribution of anomaly scores on parts that are subsequently accepted serves the same purpose: if the scores of good parts creep upward toward the threshold, the margin is eroding. The fourth line is direct verification with known parts. A set of challenge parts — physical samples of good parts and of each critical defect type, kept in a controlled cabinet — is run through the station on a schedule, perhaps at the start of every shift and after every maintenance intervention. The station must accept the good ones and reject each defective one. This test is crude compared with statistical monitoring, but it is direct evidence that the system can still catch the defects that matter, and it is the kind of evidence auditors understand. Pharmaceutical and medical device manufacturers routinely use such challenge tests, and many other industries have adopted the practice. How the layers work together It is worth following a single slow change through these layers to see how they interact. Consider a strobed ring light whose output declines gradually over a year of three-shift operation, on a station running a patch-based anomaly detector for surface marks on a painted part. Without any of the defences, the decline shows up first in the anomaly scores of good parts. As the images darken, more and more patches look slightly unlike the training patches, and the score distribution of good parts drifts upward. Nothing visible happens for months, because the scores remain below the threshold. Then the upper tail of the good distribution begins to cross it, and the false reject rate rises — slowly at first, then quickly, because the tail of a distribution grows rapidly as its centre approaches a fixed threshold. Operators see a station that "suddenly" started rejecting good parts, and the usual response is to raise the threshold, which restores the false reject rate while silently reducing sensitivity to real defects, especially faint ones whose contrast has also fallen with the light. With the defences in place, the same decline plays out differently. The reference patch measures the falling output within days, and closed-loop control raises the strobe current or exposure to hold the patch at its set point, so the images the model sees barely change. The control loop's own output — the drive current needed to hold the set point — is logged and charted, so maintenance can see the light approaching the limit of its compensation range and schedule a replacement during a planned stop. If the compensation is exhausted before that happens, the image quality gate flags patch brightness out of tolerance and diverts parts rather than judging them on degraded images. The score-distribution monitor provides a third, independent warning. And when the light is replaced, the flat-field calibration and challenge-part test confirm the station has returned to its validated state. Nothing in this sequence requires the model to be retrained, and at no point does the station make decisions on images it was not designed to judge. The operational posture The overall posture this chapter recommends is that of an instrument engineer rather than a model trainer. Instruments drift; that is expected, and the response is calibration, reference standards, monitoring and maintenance — not a hope that the instrument will remain as it was on the day it was installed. A vision system designed with physical control of its light, compensation against a reference, training on realistic variation, quality gates on its inputs, statistical monitoring of its outputs and regular challenge tests can tolerate the ordinary life of a factory floor. A system designed only to score well on a validation set captured on commissioning day cannot, however good its model. There is also a lesson in the order of the defences. Each layer is cheaper and more reliable than the one after it. An enclosure costs little and never fails silently; a reference patch costs little and makes drift measurable; augmentation is cheap but has limits; monitoring catches what the others miss but only after the shift has begun. The common failure pattern in industrial vision is to skip the early layers because the model seems good enough, and then to spend months tuning the model to compensate for variation that a sheet-metal hood would have removed in an afternoon. CHAPTER 8 Reading at Line Speed A large share of the vision systems on production lines are not looking for defects in the usual sense. They are reading. They read best-before dates and lot codes on food and beverage packaging, serial numbers and batch codes on pharmaceutical cartons, part numbers and date codes on automotive components, and identifiers marked directly into metal, plastic and glass. The consequences of misreading are concrete: a product shipped with a wrong or illegible date may have to be withdrawn; a pharmaceutical pack whose serial number does not match the record in the verification system cannot be dispensed; a part whose traceability code is unreadable breaks the chain that lets a manufacturer find every unit affected by a later quality problem. Industrial optical character recognition differs from document OCR in almost every respect. The text is short, the vocabulary is constrained, and the expected content is often known in advance. But the marks are produced by fast, imperfect printing processes on difficult surfaces, imaged in motion, and must be read in milliseconds with an error rate far lower than any document reader would require. This chapter covers the marking technologies and their characteristic failures, the central distinction between reading and verifying, the recognition methods now in use, the handling of machine-readable codes alongside text, and the design principles that keep misreads rare. How codes are marked The appearance of industrial text depends on how it was put there, and each marking technology produces characteristic forms of degradation that the reading system must tolerate or detect. Table 4 summarises the common methods. Table 4. Common industrial marking methods and their reading difficulties. Method How the mark is formed Typical substrates Characteristic reading difficulty Continuous inkjet (CIJ) Charged ink drops deflected into dot-matrix characters Bottles, cans, cartons, cables, eggs Separate dots, missing or merged dots, smearing, stretched characters Thermal inkjet (TIJ) Nozzle array prints high-resolution characters Cartons, pharmaceutical packs, labels Fading as cartridges empty, banding Thermal transfer overprinting (TTO) Heated printhead transfers ink from ribbon Flexible film, pouches, labels Voids from ribbon wrinkles, glare from film Laser marking Beam ablates coating, changes colour or engraves Cans, PET bottles, glass, plastics, metal Low contrast, angle-dependent visibility Dot peen and laser etch (DPM) Indentation or surface alteration in the part Metal and plastic components Contrast from geometry only; texture noise Printed labels Printed off-line and applied Cases, pallets, packs Skew, wrinkles, label applied to wrong product Continuous inkjet is the workhorse of high-speed coding on food and beverage lines because it prints without contact on curved and moving surfaces at very high speeds. Its characters are built from discrete dots arranged on a small grid, and the printing process introduces a recognisable set of problems. Dots can be missing because a nozzle is partially blocked, merged because ink has spread on a damp surface, or displaced because the product's speed or distance from the printhead has changed. If the line speed varies and the printer is not synchronised to an encoder, characters are stretched or compressed horizontally. Printing on condensation-covered cold bottles or on the bottom of a can produces smears and halos. Laser marking, increasingly common for sustainability reasons because it needs no ink or solvent, produces marks whose contrast depends on the material's response to the beam and on the viewing and illumination angles. A laser code on a PET bottle may be clearly visible from one angle and nearly invisible from another. Direct part marks on metal — dot-peened indentations or laser-etched patterns — contain no ink at all; their visibility comes entirely from geometry, which is why Chapter 2's dark-field, coaxial and photometric-stereo techniques are so often essential for reading them. The marking process is part of the reading problem. Integrators who succeed with industrial OCR typically spend time with the coding equipment as well as the cameras, synchronising printers to line encoders, adjusting print parameters to produce more readable characters, and agreeing on fonts and character sizes that suit both the printer and the reader. The cheapest way to read a code reliably is often to print it better. Reading versus verifying The single most important design decision in industrial OCR is to recognise that in most applications the system does not need to discover what the text says. It needs to confirm that the text says what it should. When a filling line runs a batch, the line controller or the manufacturing execution system knows the batch number, the production date and the expiry date that the coder has been instructed to print. The vision system can be given the same expected string. Its task then becomes optical character verification: check that each printed character matches the expected character, that none is missing, that the print is present and legible to a defined standard, and that it is in the right place. This is a closed-set comparison against a known answer, not an open-ended reading problem, and it is far more robust. A verification system that expects "L24187" does not need to decide whether an ambiguous character is an eight or a B; it needs only to decide whether the character in that position is a sufficiently good eight. Verification also catches the failures that matter most and that no amount of legibility checking would detect: a coder printing yesterday's date because a changeover was missed; a printer loaded with the previous product's message; a correct-looking code on the wrong product. These are content errors, not print-quality errors, and they arise from process mistakes rather than from the printer. They are among the most common sources of coding-related product withdrawals. Verification depends on a reliable data link. The expected string must reach the vision system from the same source that instructed the printer, at the same moment, and the vision system must confirm which job it is verifying against. Where the printer and the vision system are configured independently by operators typing the same message into two screens, the system checks only that two people typed the same thing. Integration through the line controller, or through standard protocols that coding equipment and vision systems both support, closes that gap. There remain applications that need genuine reading — capturing serial numbers that are generated by the printer itself, reading codes on incoming parts from suppliers, logging data for traceability — and for these, the constraints described later in the chapter, such as grammars and check digits, play the role that the expected string plays in verification. Recognition methods Traditional industrial OCR followed a pipeline inherited from document recognition. It located the text region, binarised the image to separate ink from background, segmented the text into individual characters by finding gaps between them, and classified each character by comparing it with stored templates or with a classifier trained on the specific font. Commercial tools such as Cognex's OCRMax and the OCR tools in HALCON and other libraries refined this pipeline over decades and work well on clean, consistent print. Its weakness is segmentation. Dot-matrix characters are not connected shapes; they are clusters of dots that must be grouped, and the gaps between dots within a character can be as large as the gaps between characters. Smeared print merges neighbouring characters; faded print breaks one character into fragments. Each such error in segmentation propagates into recognition. Engineers using traditional tools spent much of their effort tuning segmentation parameters and training font files for every product and printer. Deep learning removed much of this burden by recognising sequences rather than isolated characters. A typical modern pipeline has two stages. A text detector finds and outlines text regions in the image; widely used research designs include EAST, published by Xinyu Zhou and colleagues in 2017, and DBNet, published by Minghui Liao and colleagues in 2020, which predicts a probability map of text and binarises it with learned, differentiable thresholds. A text recogniser then reads each region as a sequence. The CRNN architecture of Baoguang Shi, Xiang Bai and Cong Yao, published in 2015 and in the IEEE Transactions on Pattern Analysis and Machine Intelligence in 2017, combined a convolutional feature extractor with a recurrent network and trained it with connectionist temporal classification, a loss introduced by Alex Graves and colleagues in 2006 that allows a network to learn from a string label without knowing where each character lies. Because the network reads the whole sequence, it never needs to segment individual characters, and it learns to read dot-matrix and degraded print as naturally as clean type. Later recognisers, including attention-based and transformer-based designs such as PARSeq, published by Darwin Bautista and Rowel Atienza at the European Conference on Computer Vision in 2022, improved accuracy further by modelling the context between characters. These methods are now available in both open-source and commercial forms. Baidu's PaddleOCR, released under the Apache 2.0 licence, offers detection and recognition models designed for efficient deployment, with its PP-OCRv5 generation released in 2025, and is widely used as a starting point for industrial projects. The major machine vision vendors offer pretrained deep learning OCR tools — MVTec's Deep OCR in HALCON, Cognex's deep learning OCR tools, and comparable tools from Keyence, Zebra and others — that read typical industrial fonts, including dot-matrix print, without font training, and can be fine-tuned with a modest number of examples where a particular application needs it. Industrial OCR is also one of the few inspection problems in which synthetic training data works very well. Text is easy to render in any font, with controlled degradation: missing dots, blur, smear, perspective, low contrast, background texture. Ankush Gupta and colleagues' SynthText work in 2016 showed the value of synthetic text for training scene-text detectors, and industrial practitioners routinely render dot-matrix strings with simulated printing defects to supplement real images. The real-world validation discipline of Chapter 4 still applies: synthetic data trains, real images validate. Constraining the answer A general-purpose recogniser can return any string. An industrial reading system almost never should. Constraints derived from the application are among the most powerful tools for reducing misreads, and they cost almost nothing at run time. The first constraint is the character set. If a lot code consists only of digits and upper-case letters excluding I and O, which many coding schemes avoid precisely because of their confusion with one and zero, the recogniser should be restricted to that set. The second is the format grammar: a lot code might be a letter followed by five digits; a date might be two digits, a space, three letters for the month and two digits for the year. Expressing the grammar as a pattern and rejecting strings that do not match it eliminates a large class of errors. The third is semantic validity: a date must be a real date, within a plausible window around today; an expiry date must follow the production date by the product's shelf life; a Julian day must be between 1 and 366. import re from datetime import date, datetime, timedelta LOT = re.compile(r"^[A-HJ-NP-Z]\d{5}$") # letter (no I or O) + 5 digits def check_code(lot, expiry_str, today, shelf_days): if not LOT.match(lot): return "REJECT: lot format" expiry = datetime.strptime(expiry_str, "%d %b %y").date() if expiry != today + timedelta(days=shelf_days): return "REJECT: expiry mismatch" return "ACCEPT" The fourth constraint is redundancy. Serial numbers in many schemes carry check digits — GS1 identifiers use a modulo-10 check digit, for example — which allow the system to detect most single-character errors. Where a code appears both as human-readable text and as a barcode or Data Matrix symbol, as it does on serialised pharmaceutical packaging, the two can be read independently and compared; agreement between an OCR reading and a decoded symbol is strong evidence that both are correct. Per-character confidence scores complete the picture. Recognisers report how confident they are in each character, and the decision logic should consider the weakest character, not the average. A string read with high confidence in five characters and low confidence in the sixth is uncertain, however good the average looks. Misreads and no-reads Reading systems fail in two ways that are not symmetric. A no-read occurs when the system cannot produce a confident answer. A misread occurs when it produces a confident wrong answer. A no-read is visible: the product is rejected or diverted, someone looks at it, and at worst a good item is lost. A misread is silent: the system reports a valid code that is not the code on the product. In verification applications a misread can pass a wrongly coded product; in traceability applications it can corrupt the record that links a unit to its history. Every design choice in industrial reading should therefore prefer no-reads to misreads. Thresholds should be set so that ambiguous characters produce rejection rather than a guess. Constraints should be strict. Redundant reads should be required to agree. And the system's performance should be reported with the two failure types separated: a no-read rate of 0.1 per cent may be an acceptable cost; a misread rate of 0.1 per cent is almost certainly not. Codes, grades and serialisation Machine-readable symbols — linear barcodes and two-dimensional codes such as Data Matrix and QR — are read by decoding rather than recognition. Decoding is deterministic, includes error correction in the case of two-dimensional codes, and when it succeeds it is almost never wrong. The industrial problem is not so much reading the symbol as ensuring it is printed well enough to be read reliably by everyone downstream, including scanners in warehouses, pharmacies and hospitals that are far less capable than the vision system on the line. International standards define how symbol quality is measured. ISO/IEC 15416 grades linear barcodes, ISO/IEC 15415 grades two-dimensional symbols printed on labels and packaging, and ISO/IEC TR 29158, derived from the AIM DPM quality guideline, adapts the grading to direct part marks, whose geometry and lighting differ from printed symbols. These standards assess parameters such as symbol contrast, modulation, fixed-pattern damage and axial non-uniformity, and assign a grade from A to F. Inline verification of print quality to these grades, rather than mere decoding, catches printer deterioration before codes become unreadable to downstream scanners. Pharmaceutical serialisation has made high-speed code reading a regulatory requirement. In the European Union, the Falsified Medicines Directive of 2011 and its Delegated Regulation (EU) 2016/161, which applied from 9 February 2019, require most prescription medicine packs to carry a unique identifier in a two-dimensional Data Matrix code — product code, serial number, batch number and expiry date — together with an anti-tampering device, and the identifier is checked against a central repository system before dispensing. In the United States, the Drug Supply Chain Security Act requires a product identifier with a serial number on prescription drug packages and, under its enhanced distribution security requirements that took effect in November 2023, interoperable electronic tracing at the package level; the Food and Drug Administration granted phased exemptions to different trading partners, the last of which, for small dispensers, has been extended to November 2027. Similar systems operate in many other countries. For a packaging line, these regulations mean that every pack must be printed with a unique code, read back immediately to verify both the code's content and its print grade, and recorded in a serialisation system — often also linked through aggregation to the case and pallet that contain it. At line speeds of several hundred packs per minute, this is a demanding inline vision application, and its failure modes are expensive: a pack whose code was printed but not recorded, or recorded but not printed correctly, becomes a pack that cannot legally be sold. The design principles of this chapter — verify against the expected content, read redundantly, prefer rejection to guessing, grade quality continuously — are the principles serialisation systems are built on. Throughput and geometry High-speed lines impose the timing constraints of Chapters 3 and 6 on reading as on inspection, with some particular twists. Codes on cylindrical containers may appear anywhere around the circumference; systems read them with several cameras arranged around the container, or with a single camera and mirrors, and must combine the views or select the one that contains the code. Text detection over a whole high-resolution image is relatively expensive, so production systems locate the code region first — often using the predictable position of the printhead relative to the product — and run recognition only on that region. Codes on can bottoms, read as cans pass over a camera looking upward, require very short exposures and careful lighting to avoid glare from the domed metal. The general lesson is the same as elsewhere in this book. Recognition models have become very good, and a modern deep learning OCR engine reads degraded industrial print that defeated older tools. But the reliability of a reading system comes from the structure around the recogniser: printing that is controlled, an expected answer that is known, constraints that are strict, redundancy that is checked, and a decision rule that prefers saying "I cannot read this" to saying something false. CHAPTER 9 Proving It Works and Keeping It Working The previous chapters described how to build a vision system that forms a good image, learns a decision from scarce data, executes that decision within a deadline, tolerates the variation of the factory floor, and reads codes without misreading them. This final chapter asks two questions that the organisation buying the system will ask, and that auditors, customers and regulators may ask on its behalf. How do you know it works? And how will you know it still works next year, after it has been retrained, updated and maintained? The answers come from metrology and quality engineering, which have long-established methods for qualifying measurement systems and controlling change. Learned models introduce new complications — their behaviour depends on data, they are retrained rather than recalibrated, and their failure modes are statistical — but the underlying discipline transfers well. This chapter covers the qualification of a vision system as an attribute gauge, the extra expectations of regulated industries, the continuous-improvement loop through which models are retrained, and the change control that keeps that loop from undermining the evidence on which the system's acceptance rests. The vision system as a gauge Quality engineering distinguishes between variable gauges, which output a number such as a diameter, and attribute gauges, which output a category such as pass or fail. A vision system that measures a dimension is a variable gauge and can be qualified with the gauge repeatability and reproducibility studies familiar to any quality engineer. A vision system that accepts or rejects parts on cosmetic grounds is an attribute gauge, and the tool for qualifying it is attribute agreement analysis, described in the Automotive Industry Action Group's Measurement Systems Analysis reference manual and widely used beyond the automotive sector. The study design is straightforward. A set of parts is assembled whose true status is known, established by expert review and, where possible, by a more accurate reference method such as microscopy or destructive testing. The set deliberately includes a substantial proportion of defective parts and of borderline parts on both sides of the acceptance limit, because a study composed mostly of obvious good parts says little. Each part is presented to the system several times, in random order, with the presentation varied the way production varies it: different positions on the conveyor, different orientations, different times of day. The same parts are often presented to the human inspectors whom the system supplements or replaces, which gives a direct comparison. From the results come several measures. Repeatability is the agreement of the system with itself across repeated presentations of the same part. For a deterministic algorithm, one might expect perfect repeatability, but the images themselves differ between presentations — slightly different position, noise, lighting phase — and borderline parts can flip between decisions. A system that gives different answers for the same part is telling the engineers where its decision boundary is unstable. Agreement with the reference — the fraction of parts the system classifies correctly in every trial — measures accuracy. The miss rate, the fraction of trials in which a defective part was accepted, and the false alarm rate, the fraction in which a good part was rejected, correspond directly to the escape and false reject rates of Chapter 1. Cohen's kappa, or Fleiss' kappa for several appraisers, measures agreement corrected for the agreement expected by chance, which matters when one class dominates. The AIAG manual offers commonly used guidelines for judging attribute systems: an effectiveness of at least 90 per cent, a miss rate no higher than 2 per cent and a false alarm rate no higher than 5 per cent are typically treated as acceptable, with kappa values above about 0.75 indicating good agreement. These are rules of thumb rather than laws, and a safety-critical defect may require far stricter limits, while a purely cosmetic one may tolerate looser ones. The value of the study lies less in passing a generic threshold than in producing documented, reproducible evidence of how the system performs on the parts that matter. The study has a statistical limitation already noted in Chapter 5: with a few dozen defective parts, it cannot demonstrate very low escape rates. A study with fifty defective parts presented three times each gives 150 trials, and zero misses supports, by the rule of three, an upper bound of about 2 per cent on the miss rate — useful, but not proof of parts-per-million performance. Claims about very low escape rates require larger defect libraries, longer shadow-mode operation in production, or a risk argument that combines the vision system with other controls. Honest reporting of the confidence interval, rather than the point estimate alone, is part of the discipline. A worked example shows how such a study shapes decisions. Suppose a supplier of moulded plastic housings assembles a study set of 120 parts: 60 good, 40 with clear defects of the four catalogued types, and 20 boundary parts, ten just inside and ten just outside the acceptance limits, chosen by the quality team from the defect library. Each part passes the station three times on different days and in different positions, giving 360 decisions. Suppose the results show no misses at all on the 40 clear defects, three false alarms across the 180 trials of good parts, all on the same two parts, and inconsistent decisions on six of the 20 boundary parts. Each finding points to a different action. The clean result on clear defects is encouraging but, at 120 trials, supports an upper bound on the miss rate for clear defects of about 2.5 per cent — enough for acceptance against the generic guideline, not enough to claim parts-per-million performance. The false alarms concentrated on two parts are not random noise; inspecting those parts may reveal a cosmetic feature, perhaps a flow line from the mould, that the training data under-represented and that should be added to the good set. The inconsistent boundary decisions show where the decision threshold sits relative to the acceptance limit. If the inconsistency falls on parts just inside the limit, the system is drawing its line slightly too strictly; if it falls on parts just outside, slightly too leniently, which for an escape-sensitive customer is the more serious finding. Either way, the study converts a vague sense that the system is "about right" into specific, documented adjustments, and repeating it after those adjustments provides evidence that they worked. The numbers in such an example are illustrative, but the structure is general: separate the clear cases from the boundary cases, look for concentration of errors on particular parts rather than treating errors as a single rate, and read the boundary results as a measurement of where the system's threshold actually lies. A study of the human inspectors on the same parts is often illuminating. It frequently shows that people disagree with each other, and with themselves, more than the organisation believed — which, as Chapter 1 noted, puts the machine's performance in context, and sometimes reveals that the reference labels themselves need to be revisited before the machine can be fairly judged. Validation in regulated industries In pharmaceutical, medical device and some food manufacturing, inspection systems are part of a regulated quality system, and their validation must meet regulatory expectations as well as engineering ones. In the United States, systems that create or maintain electronic records required by regulation fall under 21 CFR Part 11, which governs the trustworthiness of electronic records and signatures: audit trails, access controls, record retention and the like. A vision system that records inspection results as quality records, or whose configuration is part of the validated process, must meet these requirements. For medical devices, the Food and Drug Administration's Quality Management System Regulation, which aligned 21 CFR Part 820 with the international standard ISO 13485, took effect on 2 February 2026 and requires validation of software used in production and the quality system for its intended use. In September 2025 the FDA finalised its guidance on Computer Software Assurance for production and quality system software, which encourages a risk-based approach: concentrate rigorous testing on functions whose failure could affect product quality or patient safety, and use lighter, unscripted assurance activities for lower-risk functions. An inspection system that decides whether a device component is accepted is high-risk in this sense and warrants rigorous testing. In the pharmaceutical sector, the ISPE's GAMP 5 guide, whose second edition in 2022 added an appendix addressing artificial intelligence and machine learning, provides the framework most companies use for computerised system validation. It emphasises a life-cycle approach: requirements traced to risk, verification proportionate to risk, and controlled change throughout operation. Machine learning systems fit this framework if their particular features are recognised — notably that the training data is part of the system's specification, and that retraining is a change. The practical consequences for a vision system in a regulated environment are clear even where the details vary. The intended use and the acceptance criteria must be specified before validation. The training, validation and test datasets must be documented, versioned and controlled, because they determine the system's behaviour as surely as its code does. The challenge-part tests described in Chapter 7 provide recurring evidence that the system remains in its validated state. And any change — to the model, the thresholds, the lighting, the camera, the compiled inference engine, or the software stack — must go through change control with an assessment of its impact on validated performance. The improvement loop A vision system that never changes will slowly become worse as the factory changes around it. A system that changes freely will lose the evidence that justified trusting it. The resolution is a disciplined improvement loop that allows change while preserving evidence. The loop begins in production. Every decision the system makes can be logged with its image, its score, its recipe and model version, and the eventual outcome where one is known. Three streams of images are especially valuable. Images from the review lane of Chapter 5 come with a human decision attached, and they are concentrated at the model's decision boundary, which makes them ideal training material. Images of escapes — defects found later, by downstream inspection, by a customer, or by a periodic audit — are the most valuable of all, because they show exactly what the system missed. Images flagged by drift monitoring, from Chapter 7, show the conditions the model has not learned. These images are labelled under the same governed guide used originally, added to the curated dataset with their provenance recorded, and used to train a candidate model. The candidate is then evaluated against a fixed regression suite: a frozen, versioned set of test images, including every historic escape and every critical defect type, together with representative good images from across the range of production conditions. The candidate must meet the acceptance criteria on the full suite, not merely improve on the new images. Without this suite, retraining to fix one problem routinely introduces another, and the organisation discovers the regression only when production does. The candidate then passes through the same deployment gates as the original system. It is compiled for the target hardware and the compiled engine is re-validated, because Chapter 6 showed that quantisation and compilation can change behaviour on subtle defects. It runs in shadow mode alongside the production model, on live production, so that its decisions can be compared with the incumbent's on identical images before it takes control. Disagreements are reviewed. Only when the evidence is satisfactory does it replace the incumbent, with a recorded release, a rollback path, and a challenge-part test immediately after deployment. This loop resembles the continuous integration and delivery pipelines of software engineering, adapted to the facts that the "code" is partly data and that the test results are statistical. Machine learning operations tooling — experiment tracking, data versioning, model registries, automated evaluation — supports it, but the tooling matters less than the discipline. Many organisations run this loop successfully with modest tools and careful procedures; others with sophisticated platforms skip the regression suite or the shadow period and suffer for it. What counts as a change Change control requires knowing what has changed, and in a vision system that is broader than most people expect. Retraining the model is an obvious change. So is changing a threshold, even slightly, because the operating point is part of the specification. Less obvious changes include an updated version of the inference runtime or the device's operating system, which can alter numerical results; a replacement camera of the same model, whose sensor response may differ slightly; a new light after the old one failed; a moved bracket after maintenance; a different lens after a dropped one was replaced; a new recipe for a new product variant; and a change in the upstream process, such as a new material supplier, that alters what the parts look like. A practical way to manage this breadth is to define the system's validated configuration explicitly — a record of every component that affects the decision, with versions and settings — and to classify changes by their likely impact. Replacing a failed light with an identical unit, followed by flat-field recalibration and a passing challenge-part test, may be a minor change handled by a maintenance procedure. Retraining the model, changing thresholds or changing the hardware platform may be major changes requiring the full regression suite and shadow period. The classification should be decided in advance, not argued case by case after something has gone wrong. The human role, redesigned Automation of inspection does not remove people from the process; it changes what they do. The inspector who once looked at every part becomes the reviewer of the uncertain fraction, the labeller of new examples, the investigator of patterns in the rejects, and the operator who runs the challenge test at the start of the shift. The quality engineer becomes the owner of the defect catalogue, the labelling guide and the regression suite. The controls engineer owns the timing, the tracking and the reject verification. The machine learning engineer owns the training pipeline and the evaluation, but not, alone, the decision to release. These roles need explicit ownership. One of the most common organisational failures in industrial vision is a system delivered by an integrator or an internal data science team that then has no clear owner on the plant floor. When its reject rate drifts, no one is responsible for investigating. When a new product variant arrives, no one knows how to add it. When the integrator's engineer who understood it leaves, the knowledge leaves with them. A vision system is a piece of production equipment with a maintenance plan, a responsible owner and spare parts, and it should be handed over as one, with training for the people who will run it. There is also a question of authority. When the vision system and a human reviewer disagree, who decides? For a review lane, the human decision is final by design, but the disagreement should be recorded and periodically analysed: frequent overrides in one direction suggest either a model problem or a labelling inconsistency. When operators can bypass the system — and they always can, physically if not in software — the conditions under which they may do so, the authorisation required and the record that must be kept should be defined. A bypass that is undocumented is the most dangerous state a vision station can be in. Evidence as the product The theme of this chapter is that, for an inspection system, evidence is part of the product. A system that catches defects but cannot show that it does is, from the point of view of a customer or regulator, indistinguishable from one that does not. A system whose evidence was established once and never refreshed is trusted on the basis of a past that may no longer apply. The organisations that succeed with industrial vision over the long term are the ones that treat the defect catalogue, the labelled data, the regression suite, the challenge parts, the monitoring charts and the change records as durable engineering assets — more durable, in fact, than any particular model, which they expect to replace. Conclusion The argument of this book has been that an industrial vision system is a measurement instrument whose reliability is decided mostly outside the neural network: upstream in the physics of image formation, downstream in the decision, timing and evidence that surround each accept-or-reject call. Having followed that argument from the light source to the change-control record, it is possible to draw conclusions that were not available at the start, about where the field is heading, what remains genuinely unsettled, and what practitioners should do differently. The model is becoming the cheap part The first conclusion is that the trend of the past decade will continue in a particular direction. Each year, the model stage of the pipeline becomes easier to obtain. Anomaly detectors that learn normality from a few hundred images are available in open-source libraries; real-time detectors have shed their awkward post-processing and export cleanly to edge accelerators; deep OCR engines read dot-matrix print without font training; vision-language models can be asked in plain words whether an image shows a defect. The vendors of smart cameras package these capabilities so that an automation engineer can train a model in an afternoon without writing code. This does not make vision systems easy. It moves the difficulty. When every competitor can obtain a comparable model, the differences between a system that works and one that does not lie in the stages this book has emphasised: whether the lighting creates robust contrast for the defect, whether the resolution budget was done honestly, whether the data covers the variation of a real year of production, whether the worst-case latency meets the deadline in a hot cabinet, whether the system notices when its own images degrade, and whether anyone can show, with evidence, that it catches what it should. These are not glamorous capabilities, and they are not purchased with a licence. They are built by organisations that treat vision as an engineering discipline rather than a software installation, and they are where durable advantage now sits. A corollary is that the most valuable assets in an inspection programme are not models but the materials models are made from and tested against: the defect catalogue with its boundary samples, the governed label set, the library of physical defect parts, the frozen regression suite, the archive of good images across conditions. A manufacturer who has those can adopt each new generation of model quickly and safely. One who does not will find each new generation as hard to deploy as the last. What remains unsettled Several questions remain open, and practitioners should hold their views on them provisionally. The first is how to validate general-purpose models. The validation methods of Chapter 9 assume a system whose behaviour is defined by its training data and configuration, both of which the manufacturer controls. Large pretrained models, especially vision-language models used zero-shot, have behaviour shaped by training data the manufacturer has never seen and cannot inspect, and their providers update them on their own schedules. A system built on such a model can be tested on the manufacturer's parts, but its failure modes on unanticipated inputs are harder to bound. Industry practice has not settled how much reliance such models can bear in a final accept-or-reject decision, as opposed to the upstream roles of triage, labelling and prototyping where they are already useful. The second is the tension between continual learning and controlled change. Technically, it is attractive for a model to update itself continuously as new labelled images arrive from the review lane. From the standpoint of evidence, a model that changes every day is a system whose validated state never holds still. The improvement loop of Chapter 9 resolves the tension by batching changes into controlled releases, at the cost of slower adaptation. Whether more automated forms of assurance — automatic regression testing with statistical acceptance rules, perhaps — can safely shorten that cycle is an open question, and regulated industries in particular will answer it cautiously. The third is the role of synthetic data. Generative models produce increasingly convincing images of defects, and the economic pressure to use them is strong, because real defects are scarce. Chapter 4 drew a firm line: synthetic data may train, but only real data may validate. That line will be tested as synthetic images grow more realistic, and as the shortage of real defects makes strong claims about very low escape rates otherwise impossible to support. It should hold, because the purpose of validation is to show that the system catches defects as they actually appear through this camera under this light, and no generator can certify its own fidelity to that. The fourth is the benchmark gap. Public benchmarks such as MVTec AD drove a period of rapid progress and then saturated, and harder successors such as MVTec AD 2, with its changed-lighting test sets and its sub-60-per-cent localisation scores, show how much room remains. Benchmarks that measure performance at a fixed low false-positive rate, under distribution shift, with honest confidence intervals, would align research more closely with factory needs. Until they are common, practitioners should read leaderboard numbers as evidence of potential, not of fitness for a particular line. What to do differently The practical consequences can be stated plainly, because each follows from a chapter's argument. Write down the costs before buying anything. Agree, with quality, production and finance in the room, on the defect rate, the tolerable escape rate and the tolerable false reject rate, and build the defect catalogue with boundary samples. If those numbers cannot be agreed, the project's real problem is organisational, and no camera will solve it. Design the light before collecting data. Spend the first days in a lighting lab, measure contrast-to-noise for each defect type, and do not proceed until the defects are visible in raw images. Every unit of contrast gained here reduces the burden on every later stage and makes the system robust to change. Do the arithmetic early. Resolution, exposure, blur, line rate, transfer time and latency budget are simple calculations that eliminate impossible designs in an afternoon. Carry them out before choosing hardware, and recheck them when anything changes. Collect data across time, split it by time, and govern its labels. Treat the dataset as a controlled document, the label guide as part of the specification, and real defect parts as irreplaceable. Choose operating points deliberately and give uncertainty somewhere to go. Use the costs to set thresholds, report part-level detection and false reject rates with confidence intervals, and use a review lane for the cases the model cannot decide. Engineer for the worst case at the edge. Measure latency histograms on the target hardware at the highest cabinet temperature, validate the compiled and quantised engine rather than the training model, and make late results fail safe. Verify rather than read whenever the expected answer is known. In coding and serialisation, pass the expected string from the source that instructed the printer, constrain every reading, prefer no-reads to misreads, and read redundantly where codes appear twice. Instrument the instrument. Gate image quality, chart reject rates in both directions, monitor input and score distributions, and run challenge parts at every shift and after every intervention. Control change and assign ownership. Define the validated configuration, classify changes in advance, keep a frozen regression suite including every historic escape, run candidates in shadow mode, and give the system an owner on the plant floor. None of these recommendations depends on which model architecture leads next year's benchmarks. That is the point. The network in the middle of an inspection system will be replaced several times over the life of the line it guards. The light, the timing, the data, the decision rule and the evidence are what the factory actually owns, and they decide whether the next bottle, the next casting or the next serialised pack that passes the camera is judged correctly. Hashtags: #AppliedComputerVision #ComputerVision #ManufacturingQualityControl #DefectDetection #AutomatedVisualInspection #IndustrialMachineVision #ArtificialIntelligence #MachineLearning #DeepLearning #SmartManufacturing #Industry40 #ManufacturingAutomation #QualityAssurance #QualityInspection #IndustrialAI #VisualDefectDetection #AnomalyDetection #ObjectDetection #ImageSegmentation #YOLO #EdgeAI #EdgeComputing #RealTimeInspection #IndustrialCameras #MachineVisionLighting #OpticalCharacterRecognition #HighSpeedConveyors #PredictiveQuality #ProductionLineAutomation #ManufacturingInnovation

Download the Book (PDF): Introduction Every mobile generation has been sold with a number. The second generation promised digital voice, the third promised the mobile internet, the fourth promised broadband in your pocket, and the fifth arrived wrapped in claims of gigabit speeds and millisecond latency. The sixth generation already has its number too. Research roadmaps and vendor presentations speak of peak data rates of a terabit per second, of radios working at hundreds of gigahertz, of networks that do not merely carry data but see the world around them. It is easy to read those claims as the next turn of a familiar marketing wheel and stop paying attention. That would be a mistake, because the most important things happening in sixth-generation wireless are not in the headline numbers. They are in a quieter set of changes to how radio networks are physically built and what they are for. This booklet is about those changes. It takes three ideas that dominate serious discussion of 6G and examines each on its own terms: the move toward terahertz and sub-terahertz frequencies, the merging of radar-like sensing with communication, and the replacement of the classic cell tower with distributed arrays of cooperating antennas, an approach known as cell-free massive MIMO. Each idea is real, each has strong physics behind it, and each has been oversold in some quarters. The aim here is to explain the mechanisms clearly enough that a reader can tell which promises follow from the physics and which are hopes. The argument in brief The argument of the booklet can be stated simply. Sixth-generation wireless will be defined less by a single new frequency than by a change in the shape of the network. As engineers push to higher frequencies to find wide, empty channels, the physics of propagation punishes long links, thin walls become opaque, and a person's hand becomes a barrier. The response cannot be a bigger tower. It has to be more antennas, closer to users, coordinated so tightly that the old boundary between one cell and the next dissolves. The same dense, wideband, multi-antenna infrastructure that makes this possible also turns the network into an instrument capable of measuring distance, speed and direction, which is why sensing arrives alongside connectivity rather than as an afterthought. From that follows a second claim that runs against much of the hype. For the coming decade, the decisive spectrum is not the terahertz band but the middle ground between roughly 7 and 15 gigahertz, sometimes called FR3 or the centimetric range. That is where the first 6G networks will actually be built, where the regulatory fights are now being fought, and where the bulk of economic value will be created. Terahertz links will matter, but as specialised tools for short, high-capacity connections inside buildings, between equipment racks, in factories and as wireless backhaul, not as the blanket coverage layer of a national network. Where things stand Sixth generation is no longer a research slogan. It has entered the formal machinery that produces global standards, and the timetable has hardened considerably in the past year. At the international level, the International Telecommunication Union set out its framework for what it calls IMT-2030 in Recommendation ITU-R M.2160, approved in late 2023. The framework defines six usage scenarios, including integrated sensing and communication and the combination of artificial intelligence with communication, and fifteen capabilities against which candidate technologies will be judged. The ITU's Working Party 5D has published the submission process: proposals for IMT-2030 radio interface technologies may be submitted from February 2027, with the window closing in early 2029, and evaluation running alongside and after it. The industry body that writes the specifications most of the world actually uses, the Third Generation Partnership Project, has organised its work in two steps. Release 20 is the study phase, which also carries the last purely 5G-Advanced enhancements. Release 21 is where normative 6G specifications will be written. At its plenary meetings in June 2026 the project approved a Release 21 timetable that points to a final protocol freeze in March 2029, with commercial systems expected around 2030. Some foundational choices have already been made. The 6G radio will keep orthogonal frequency-division multiplexing as its downlink waveform, will reuse the low-density parity-check and polar codes from 5G with refinements, and will support channels up to 400 megahertz wide in the new 7 gigahertz spectrum. Other choices remain open. At the September 2026 plenary in Madrid, delegates endorsed a requirement for efficient sharing of control resources between 5G and 6G on the same carriers, but postponed until December the decision on whether 6G should offer any migration option beyond that spectrum-sharing baseline. Spectrum is the slowest-moving piece of all. The World Radiocommunication Conference scheduled for late 2027 will consider three candidate ranges for mobile use under agenda item 1.7: 4.4 to 4.8 gigahertz, 7.125 to 8.4 gigahertz or parts of it, and 14.8 to 15.35 gigahertz. Each is already used by satellites, military systems, radio astronomy or other incumbents. In the United States, legislation passed in July 2025 restored the Federal Communications Commission's auction authority and created an 800 megahertz pipeline, but explicitly excluded the 7.4 to 8.4 gigahertz segment used for military satellite communications. In Europe, the Radio Spectrum Policy Group has proposed giving most of the upper 6 gigahertz band to mobile operators as an early 6G home. These are not abstract debates. They will determine how much room the first 6G radios have to breathe. How the booklet is organised Chapter 1 explains why each generation of wireless has climbed the spectrum, using Claude Shannon's capacity formula to show why bandwidth is the most direct route to speed and why that route always leads upward. It also sets out the map of frequency tiers that later chapters refer to. Chapter 2 is the physics core of the booklet. It explains how radio waves at terahertz frequencies behave: the real meaning of free-space path loss, the role of antenna aperture, molecular absorption by oxygen and water vapour, the effects of rain and foliage, and the stubborn problem of blockage. It also introduces the near field, a regime that becomes practical when antenna arrays grow large compared with the wavelength. Chapter 3 turns to hardware. Generating, amplifying and detecting signals at hundreds of gigahertz pushes transistor technology to its limits, and phase noise, power efficiency, data conversion and beam alignment all become hard engineering problems. The chapter explains what has been demonstrated, including laboratory links carrying hundreds of gigabits per second, and what stands between a laboratory bench and a phone. Chapter 4 examines the middle path: the 7 to 15 gigahertz range, the upper 6 gigahertz band, and the regulatory contest over them. It sets out who uses these frequencies today, what each region is proposing, and why the outcome will shape 6G more than any terahertz breakthrough. Chapter 5 follows the standards process from the ITU's IMT-2030 framework to 3GPP's Release 20 studies and Release 21 specifications. It explains what has been decided, what remains contested, and why the question of migrating from 5G has become one of the most important arguments in the industry. Chapter 6 explains integrated sensing and communication: how a network built for data can also behave like a radar, what determines its resolution, which uses are realistic in the near term, and why privacy and security questions arrive at the same moment as the technology. Chapter 7 describes cell-free massive MIMO, the architectural idea that gives this booklet half its title. It traces the idea from early work on network coordination through to the user-centric designs now under study, and explains what cell-free systems gain, what they cost, and why scalability is the property that decides whether the idea survives. Chapter 8 deals with what it takes to deploy distributed networks in practice: fronthaul capacity, timing and phase synchronisation, energy, the use of artificial intelligence in the radio, reconfigurable surfaces, and the economics of who builds and owns dense infrastructure. The conclusion draws out what follows from all of this for operators, regulators, equipment makers and the public. A note on numbers Sixth-generation research produces a great many figures, and many of them are targets, simulations or best-case laboratory results rather than measured performance in the field. Where this booklet quotes a number, it says where the number comes from and what kind of number it is. Where the honest answer is that nobody yet knows, it says that instead. The physics is solid, the standards timetable is public, and the spectrum agenda is fixed; the commercial outcome is not. Readers should treat the 2030s network described here as a set of strong tendencies, not a finished blueprint. Chapter 1: Why Every Generation Climbs the Spectrum In 1948 Claude Shannon, working at Bell Laboratories, published the result that still governs every wireless system ever built. The maximum rate at which information can be sent reliably over a channel depends on two quantities: how wide the channel is, and how strong the signal is compared with the noise. Written compactly, capacity equals bandwidth multiplied by the base-two logarithm of one plus the signal-to-noise ratio. Everything else in radio engineering, from modulation schemes to antenna arrays, is an attempt to approach that limit or to change the terms inside it. The shape of the formula explains the history of mobile networks better than any marketing plan. Capacity grows in direct proportion to bandwidth. Double the width of the channel and, at the same signal-to-noise ratio, you double the possible data rate. Capacity grows only logarithmically with signal-to-noise ratio. Doubling the transmitted power, which raises the ratio by about three decibels, adds roughly one bit per second for every hertz of bandwidth once the signal is already strong. At high signal quality, every additional bit per hertz requires another doubling of power. Power is expensive, limited by regulation and by batteries, and it interferes with neighbours. Bandwidth, if you can find it, is a far more efficient lever. A worked example shows how strongly the formula favours width. Take a channel of 100 megahertz with a signal-to-noise ratio of 20 decibels, a factor of 100, which is a good but not exceptional mid-band link. The Shannon limit is 100 megahertz multiplied by the base-two logarithm of 101, or about 666 megabits per second. Now quadruple the bandwidth to 400 megahertz while keeping the transmitter power the same. Because the same power is now spread over four times the spectrum, and noise power grows in proportion to bandwidth, the signal-to-noise ratio falls by about six decibels, to 14 decibels, a factor of roughly 25. The limit becomes 400 megahertz multiplied by the logarithm of 26, about 1.88 gigabits per second. Even after paying the noise penalty, the wider channel offers nearly three times the capacity. To achieve the same gain by raising power on the narrow channel would require increasing the signal-to-noise ratio from 20 decibels to roughly 57 decibels, a factor of several thousand in power, which no practical system could deliver. That asymmetry has shaped the industry's behaviour for four decades. Each generation has arrived roughly ten years after the last: first-generation analogue systems in the early 1980s, GSM in the early 1990s, 3G around 2001, LTE around 2009 and 5G in 2019. Each brought a new air interface, but each also brought access to wider channels, and the wider channels delivered most of the headline gain. Improvements in modulation, coding and scheduling added real but smaller increments, because modern systems already operate within a few decibels of Shannon's limit for a single link. The logic of moving up The trouble is that bandwidth is scarce where radio propagates best. The frequencies below about 3 gigahertz travel far, bend around obstacles, and pass through walls with tolerable loss. They are also crowded with television broadcasting, aviation and maritime systems, defence radars, satellite links and every earlier generation of mobile service. In that part of the spectrum, a channel of 20 megahertz is generous, and an operator holding 100 megahertz in total across several bands is well supplied. Higher up, the picture changes. A fixed percentage of the carrier frequency yields a much larger absolute bandwidth at high frequency than at low. Ten percent of 1 gigahertz is 100 megahertz; ten percent of 30 gigahertz is 3 gigahertz; ten percent of 300 gigahertz is 30 gigahertz. Radio hardware is easiest to design when the signal bandwidth is a modest fraction of the carrier, so moving up the spectrum is the natural way to obtain wider channels. Equally important, the higher bands have historically been less used, because the equipment to exploit them was expensive and their propagation was considered poor. Unused spectrum is spectrum that can be cleared, auctioned or shared. Each generation has therefore extended its reach upward. Second-generation GSM operated mainly at 900 and 1800 megahertz. Third-generation systems added bands around 2 gigahertz. Fourth-generation LTE reached 2.6 gigahertz and, in later releases, 3.5 gigahertz. Fifth-generation New Radio defined two frequency ranges: FR1, covering bands up to about 7 gigahertz, with the 3.3 to 4.2 gigahertz range becoming the workhorse mid-band; and FR2, covering millimetre-wave bands from roughly 24 to 71 gigahertz. The 5G specifications allowed carriers of up to 100 megahertz in FR1 and up to 400 megahertz in FR2, later extended further for the highest bands. There is a less obvious reason why higher frequencies help. Antenna size scales with wavelength, and wavelength shrinks as frequency rises. At 700 megahertz the wavelength is about 43 centimetres; at 3.5 gigahertz it is about 8.6 centimetres; at 28 gigahertz it is about 1.1 centimetres; at 300 gigahertz it is one millimetre. An array of many antenna elements, each spaced half a wavelength from its neighbours, occupies a physical area that shrinks with the square of the frequency. A panel that holds 64 elements at 3.5 gigahertz could, in principle, hold several thousand at 30 gigahertz. Large arrays can focus energy into narrow beams, and those beams can raise the signal-to-noise ratio and allow many users to be served at once on the same frequency. The move upward therefore attacks Shannon's formula from both sides: it widens the bandwidth term and, through array gain, protects the signal-to-noise term. The price paid by 5G millimetre wave If the logic were that simple, 5G millimetre-wave networks would be ubiquitous. They are not, and understanding why is essential to understanding 6G. Early 5G deployments in the United States leaned heavily on bands around 28 and 39 gigahertz. Measured speeds were often spectacular, with multi-gigabit downloads in direct line of sight of a small cell. Coverage was the problem. Signals at those frequencies are attenuated sharply by building materials, especially modern energy-efficient glass with metallic coatings, by foliage, and by human bodies. A user who stepped around a corner or turned so that their hand covered the phone's antenna module could see the link collapse. Operators found that covering a city block reliably required many small cells, each needing a site, power, a backhaul connection and permission from a landlord or municipality. The cost per square kilometre was far higher than for mid-band macro cells, and the revenue did not rise to match. Millimetre wave found useful niches in stadiums, airports, dense urban hotspots and fixed wireless access to homes, but most of the world's 5G value has been delivered in the 3.5 gigahertz mid-band, where a single macro site can cover a useful area while still offering 100 megahertz channels. The economics are worth spelling out. A macro cell in mid-band spectrum might cover an area several hundred metres to a kilometre or more in radius, depending on terrain and building density. A millimetre-wave small cell typically serves a street segment of perhaps 100 to 200 metres in line of sight. To cover the same area, an operator might need dozens of small cells for every macro site, and each small cell carries fixed costs for site rental, power, backhaul and maintenance that do not shrink in proportion to its size. Unless traffic density is high enough to fill all those cells, the cost per gigabyte delivered rises sharply. The lesson that the industry drew from this experience shapes the entire 6G agenda. Bandwidth at high frequency is real, but it can only be harvested if the network architecture changes to suit it. Placing a few millimetre-wave radios on existing towers does not work. Either the network must become much denser and better coordinated, or the new spectrum must come from a band that propagates more forgivingly. In practice, 6G is pursuing both. A map of the tiers Before examining any of these ranges in detail, it helps to have a working map. Table 1 sets out the frequency tiers that appear throughout this booklet, the order of magnitude of contiguous bandwidth each can offer, and the role each is expected to play. The figures are indicative rather than precise, because channel sizes depend on national band plans and on what regulators eventually release. Table 1. Frequency tiers and their expected roles in 6G. Tier Approximate range Typical channel width Coverage character Expected 6G role Low band Below 1 GHz 5 to 20 MHz Kilometres, deep indoor Wide-area coverage, IoT Mid band (5G FR1) 1 to 7 GHz Up to 100 MHz Hundreds of metres to km Capacity layer shared with 5G Upper mid band (FR3) About 7 to 24 GHz Up to 400 MHz Hundreds of metres First new 6G capacity spectrum Millimetre wave (FR2) About 24 to 71 GHz 400 MHz to 2 GHz Tens to hundreds of metres Hotspots, fixed wireless Sub-terahertz About 90 to 300 GHz Several GHz to tens of GHz Metres to tens of metres Short links, backhaul, sensing Sources: 3GPP NR specifications for FR1 and FR2 channel widths; 3GPP 6G study agreements (2026) for the 400 MHz FR3 figure. The terahertz band, strictly, begins at 300 gigahertz and extends to 3 terahertz. In the communications literature, the range from about 100 to 300 gigahertz is usually called sub-terahertz, and most practical 6G research above the millimetre-wave bands concentrates there, especially in the D-band from 110 to 170 gigahertz and in a window around 220 to 320 gigahertz. When engineers speak loosely of terahertz communication, they usually mean these sub-terahertz windows rather than frequencies above 1 terahertz, where atmospheric absorption becomes overwhelming for anything other than very short links. What IMT-2030 asks for The International Telecommunication Union does not write detailed technical specifications, but it sets the frame within which they are judged. Its IMT-2030 framework, set out in Recommendation ITU-R M.2160, describes six usage scenarios. Three extend the 5G scenarios: immersive communication, an evolution of enhanced mobile broadband toward extended reality and holographic-style media; massive communication, extending the machine-type connections of the Internet of Things; and hyper-reliable and low-latency communication, for industrial control and similar tasks. Three are new: ubiquitous connectivity, aimed at closing the coverage gaps that leave rural areas and much of the developing world poorly served; artificial intelligence and communication, which treats the network as both a carrier and a host of distributed AI; and integrated sensing and communication, which asks the network to measure its environment as well as to carry data. The framework also lists fifteen capabilities. Nine are enhanced versions of familiar 5G metrics such as peak data rate, user-experienced data rate, spectrum efficiency, area traffic capacity, connection density, mobility, latency and reliability, plus security and resilience. Six are new: coverage, positioning, sensing-related capabilities, AI-related capabilities, sustainability, and interoperability. For peak data rate, the framework gives research target values that run from 50 to 200 gigabits per second, while noting that higher values may be considered. These are not promises of what a user will see; they describe the theoretical maximum for a single device using all available resources in ideal conditions. Two features of this framework matter for the argument of this booklet. First, the explicit inclusion of coverage and ubiquitous connectivity is a corrective to the millimetre-wave experience. A 6G system that delivers astonishing peak rates in a few spots while leaving coverage no better than 5G would fail the ITU's own vision. Second, sensing and positioning are listed as capabilities in their own right. That means the network's ability to measure range, angle and velocity will be evaluated, not merely tolerated as a side effect. Spectral efficiency and its limits If bandwidth is the main lever, spectral efficiency is the second. Spectral efficiency, measured in bits per second per hertz, describes how much data a system pushes through each unit of bandwidth. Single-antenna links are bounded by the logarithmic term in Shannon's formula. Multi-antenna systems escape that bound in a specific way: by sending several independent data streams through the same frequency at the same time, separated in space. This technique, multiple-input multiple-output transmission or MIMO, multiplies capacity by up to the number of independent spatial paths the channel can support. Fourth-generation LTE made MIMO standard, with two or four antennas at each end. Fifth generation introduced massive MIMO at base stations: arrays of 32 or 64 transceivers, and in some products more, serving many users simultaneously with beams shaped in real time. Massive MIMO has been the single most successful technology of the 5G era in mid-band spectrum, because it raises the capacity of existing sites without requiring new ones. The natural question for 6G is whether arrays can simply grow larger. In the 7 to 15 gigahertz range, the wavelength is a third to a half of its value at 3.5 gigahertz, so the same physical panel can hold roughly four to nine times as many elements. That extra array gain helps compensate for the higher propagation loss at higher frequencies, and it is one of the main reasons the industry believes FR3 can be deployed largely on existing macro sites. Equipment vendors have discussed panels with several hundred elements for this band. There is a limit, however, to what a single site can do. The users who suffer most in any cellular network are those at the edge of a cell, far from their serving base station and close to a neighbouring one whose signals arrive as interference. A larger array at the centre of the cell improves their signal but cannot remove the geometry that puts them at the edge. The 95th-percentile worst-served users in a cellular system are typically limited not by noise but by interference from nearby cells. Making every site more powerful raises everyone's signal and everyone's interference together. This is the opening through which cell-free architecture enters. If the users at the cell edge are poorly served because they sit between two sites, one answer is to stop treating the sites as competitors. Antennas at both sites, and at many smaller points between them, could cooperate to serve each user jointly, so that the signal that was interference becomes useful energy. That idea, and the practical obstacles to it, occupy Chapter 7. For now the important point is that the push to higher frequencies and the push toward distributed, cooperative antennas are not separate research programmes. They are two halves of the same response to Shannon's formula and to the geometry of real cities. Why sensing appears now A final thread connects the spectrum story to the sensing story. A radar measures distance by timing echoes, and the precision with which it can separate two objects at slightly different ranges depends on its bandwidth. The range resolution of a pulse or chirp is the speed of light divided by twice the bandwidth. A 20 megahertz LTE carrier gives a resolution of about 7.5 metres, too coarse for most purposes. A 400 megahertz carrier gives about 37.5 centimetres. A 4 gigahertz sub-terahertz channel gives about 3.75 centimetres. As communication channels have widened, they have crossed the threshold at which their signals become useful for sensing. At the same time, large antenna arrays allow angles to be measured precisely, and the short wavelengths of high frequencies make small movements, such as the rise and fall of a chest during breathing, detectable through Doppler shifts. Wireless networks are, almost by accident, becoming capable of the measurements that once required dedicated radar equipment. Sixth generation is the first generation designed with that capability in mind from the start, which is why integrated sensing appears among the IMT-2030 usage scenarios. The rest of this booklet follows the threads introduced here. The upward movement of spectrum brings with it new physics, new hardware and new politics. The densification and coordination that high frequencies demand bring with them a new architecture. And the bandwidth and arrays that serve both purposes turn the network into an instrument. None of these changes is guaranteed to succeed on the timetable that standards bodies have set, but together they explain why 6G is likely to be a deeper change than its predecessor, even if its first deployments look modest. Chapter 2: The Physics of Terahertz Propagation A common sentence in popular accounts of 6G says that terahertz waves "travel only a few metres" or "cannot pass through anything." Both statements contain a grain of truth and a large measure of confusion. Radio waves at hundreds of gigahertz can cross a room, a factory floor or, under the right conditions, a kilometre of open air. What changes as frequency rises is not a single property but a set of interacting effects, some of which are artefacts of how engineers usually describe links and some of which are genuinely physical. Untangling them is the first step to judging what terahertz communication can and cannot do. The myth and truth of path loss The most widely quoted equation in radio engineering is the formula for free-space path loss. In decibels, it says that the loss between two antennas grows by 20 decibels for every tenfold increase in distance and by another 20 decibels for every tenfold increase in frequency. A link at 300 gigahertz therefore appears to suffer 40 decibels more loss than an identical link at 3 gigahertz, a factor of ten thousand in power. Taken at face value, this looks like a law of nature that dooms high frequencies. It is not. The frequency term in the formula comes from an assumption hidden inside it: that the antennas at both ends are isotropic, or at least have a fixed gain regardless of frequency. An isotropic antenna is an idealisation that radiates equally in all directions, and its effective capture area is proportional to the square of the wavelength. As the wavelength shrinks, the area over which such an antenna collects energy shrinks with it. The extra loss at high frequency is really a statement that a small antenna catches less of the passing wave. Engineers do not build terahertz links with isotropic antennas. They build them with arrays or lenses of a given physical size. The gain of an antenna with a fixed physical aperture rises with the square of the frequency, because more wavelengths fit across it. If you hold the apertures fixed at both ends, the Friis transmission equation can be rewritten so that received power is proportional to the product of the two aperture areas, divided by the square of both the wavelength and the distance. In that form, raising the frequency tenfold improves the link by 20 decibels rather than degrading it. A worked example makes the point concrete. Suppose both ends of a link have a square aperture one centimetre on each side. At 300 gigahertz, where the wavelength is one millimetre, each aperture has a gain of roughly 31 decibels relative to isotropic. Over 10 metres, the free-space path loss with isotropic antennas would be about 102 decibels. Adding the two aperture gains reduces the net loss to about 40 decibels. At 3 gigahertz, the same one-centimetre aperture is far smaller than a wavelength and cannot provide useful gain at all; the link would be limited to the 82 decibels of isotropic path loss, minus whatever modest gain a small antenna can manage. For fixed antenna size, the high-frequency link is the stronger one. The catch is that high gain means narrow beams. A one-centimetre aperture at 300 gigahertz forms a beam only a few degrees wide. The two ends must find and point at each other, track each other as they move, and recover when something steps between them. The physics does not forbid long or strong links at terahertz frequencies. It converts the problem from one of power into one of direction. That shift underlies almost every design challenge discussed in this booklet. What the air absorbs Free-space loss is a purely geometric effect: energy spreading over a growing sphere. A real atmosphere also absorbs energy, and it does so selectively. Molecules of oxygen and water vapour have rotational and vibrational resonances that absorb radio waves at specific frequencies, producing sharp absorption peaks separated by quieter windows. Below about 50 gigahertz, these effects are mostly minor for terrestrial links. The first important peak is the oxygen absorption complex around 60 gigahertz, where attenuation at sea level reaches roughly 15 decibels per kilometre. That figure explains why the 57 to 71 gigahertz band has been used for short-range unlicensed systems such as WiGig: the absorption keeps signals from travelling far, which limits interference between neighbours. A second oxygen line sits near 119 gigahertz. Water vapour has a modest line near 22 gigahertz and much stronger ones near 183, 325, 380, 448 and 557 gigahertz, with absorption rising steadily between the lines as frequency increases. Between these peaks lie the windows where communication research concentrates: around 94 gigahertz, the D-band from 110 to 170 gigahertz, and the region from roughly 200 to 320 gigahertz. In those windows, the absorption at sea level in typical humidity is a few decibels per kilometre, rising toward ten or more in hot and humid air. Above about 1 terahertz, absorption by water vapour climbs into the hundreds of decibels per kilometre, and terrestrial links become practical only over very short distances or in dry environments. The relevance of these numbers depends entirely on distance. A molecular absorption of five decibels per kilometre costs a quarter of a decibel over 50 metres, which is negligible compared with other losses. Over 500 metres, it costs 2.5 decibels; over 2 kilometres, 10 decibels. For indoor and short outdoor links, absorption is a minor term. For wireless backhaul across a city, it is significant but manageable. For wide-area coverage of the kind a macro cell provides, it is one more reason why sub-terahertz frequencies will not replace mid-band spectrum. The International Telecommunication Union maintains the standard model for gaseous absorption, Recommendation ITU-R P.676, and terahertz link designers rely on it for precisely these calculations. Rain, fog and foliage Rain affects any radio wave whose wavelength is comparable to the size of raindrops. Drops are typically between a fraction of a millimetre and a few millimetres across, so rain attenuation rises steeply through the microwave and millimetre-wave bands and then levels off above about 100 gigahertz, where the wavelength becomes smaller than most drops and the attenuation is dominated by the geometric size of the drops rather than resonance effects. In a heavy downpour, the attenuation in the sub-terahertz windows is of the order of 10 to 20 decibels per kilometre. Like gaseous absorption, it matters for backhaul and for outdoor links of hundreds of metres, and much less for short links. Fog and cloud consist of droplets far smaller than raindrops, typically tens of micrometres, and their effect at sub-terahertz frequencies is smaller than that of heavy rain, though not zero. Dust and snow have their own effects, which depend on moisture content. For a network designer, the practical conclusion is that any outdoor sub-terahertz link longer than a couple of hundred metres must be planned with a weather margin, much as microwave backhaul links have long been planned with rain zones in mind. Foliage is more troublesome. Leaves contain water and are comparable in size to sub-terahertz wavelengths. A single tree can attenuate a millimetre-wave signal by tens of decibels, and the effect worsens with frequency and with the season. Wind-driven movement of branches also causes rapid fluctuation. Outdoor sub-terahertz links will generally need clear lines of sight above or around vegetation. Blockage, diffraction and the shrinking Fresnel zone At low frequencies, radio waves bend around obstacles through diffraction, and they pass through many building materials with moderate loss. Both effects weaken as frequency rises. Diffraction can be understood through the concept of the Fresnel zone, the region around the direct line between two antennas within which most of the energy travels. The radius of the first Fresnel zone at the midpoint of a link is the square root of the wavelength multiplied by the distance, divided by four. For a 100-metre link at 3 gigahertz, it is roughly 1.6 metres; at 300 gigahertz, about 16 centimetres. A thin pole that would barely disturb the low-frequency link can occupy a large share of the high-frequency one. Conversely, a narrow clearance that is useless at low frequency can be perfectly adequate at high frequency. The narrow Fresnel zone is both a weakness and a strength: the link is easy to block, but it is also easy to steer through small gaps and less prone to interfering with others. Penetration through materials also worsens. Concrete and brick walls that cost a mid-band signal 10 to 20 decibels can block sub-terahertz signals almost entirely. Even the human body is a near-total barrier at these frequencies, which is why a person walking between a transmitter and receiver, or a hand wrapped around a phone, can cause losses of tens of decibels. Measurements at millimetre-wave frequencies have consistently shown that self-blockage by the user's own body and hand is one of the dominant impairments for mobile devices, and the problem grows at higher frequencies. The engineering responses are known, though none is free. Devices can carry several antenna modules on different edges so that at least one is unobstructed. Networks can provide multiple access points so that a blocked path to one is replaced by an open path to another. Links can exploit reflections from walls and ceilings, provided the system can rapidly find a new beam direction when the direct path disappears. Each of these responses pushes the architecture toward redundancy and density, which is to say toward the distributed, cell-free designs examined later in this booklet. Reflection and scattering When a radio wave strikes a surface, it is reflected, scattered or absorbed. Whether a surface reflects like a mirror or scatters like frosted glass depends on its roughness compared with the wavelength. A rough rule, the Rayleigh criterion, says that a surface behaves as smooth when the height of its irregularities is small compared with the wavelength divided by the sine of the grazing angle. At 3 gigahertz, with a wavelength of 10 centimetres, almost every interior wall is smooth. At 300 gigahertz, with a wavelength of a millimetre, the texture of plaster, the grain of wood and the grooves of a brick wall are all significant. Measurements by research groups including those at New York University and at TU Braunschweig have found that indoor sub-terahertz channels still contain useful reflected paths, especially from smooth surfaces such as glass, metal and painted walls, but that diffuse scattering takes a larger share of the energy than at lower frequencies. Each reflection typically costs several decibels to tens of decibels, depending on the material. The result is a channel with relatively few strong paths. A sub-terahertz indoor channel is sparse in angle and in delay: a direct path, a handful of first-order reflections, and a weak diffuse background. Sparsity helps beam training, because there are few directions to search, and it simplifies channel estimation. It also means that losing the direct path removes a large share of the available energy, and there may be only one or two alternatives. One practical consequence is that the channel models engineers rely on thin out above 100 gigahertz. The 3GPP model used to evaluate cellular systems, TR 38.901, is defined from 0.5 to 100 gigahertz, and its parameters for the upper part of that range were fitted to relatively few measurements. Above 100 gigahertz, designers depend on measurement campaigns by individual research groups and on the channel models developed for the IEEE 802.15.3d standard, which cover specific scenarios such as data centres, kiosks and short links inside devices. Building a model trustworthy enough to compare competing system designs across indoor offices, factories and streets requires many more measurements, taken with consistent methods, than exist today. Until such a model is agreed, performance claims for sub-terahertz mobile systems rest on assumptions that are hard to check, and readers should treat simulation results in this range with corresponding caution. Table 2 summarises how the main propagation effects scale with frequency and what each means for system design. The entries are qualitative because the precise figures depend on materials, humidity and geometry. Table 2. How key propagation effects change at sub-terahertz frequencies. Effect Trend as frequency rises Matters most for Typical design response Free-space loss (fixed aperture) Improves with frequency All links Large arrays, precise pointing Gaseous absorption Rises, with sharp peaks Links over hundreds of metres Operate in absorption windows Rain attenuation Rises, then levels off above ~100 GHz Outdoor backhaul Weather margins, short hops Diffraction Weakens sharply Non-line-of-sight users Multiple access points Material penetration Weakens sharply Indoor coverage from outside Indoor deployment Surface scattering Becomes more diffuse Reflection-based links Fast beam re-acquisition Noise, bandwidth and Doppler Wider bandwidth brings a less obvious cost: more noise. Thermal noise power is proportional to bandwidth. At room temperature, the noise floor is about minus 174 decibels relative to a milliwatt per hertz. Over 100 megahertz, that becomes about minus 94 decibels relative to a milliwatt; over 10 gigahertz, about minus 74. A receiver working across a very wide channel must therefore capture correspondingly more signal power to achieve the same signal-to-noise ratio. Shannon's formula still favours bandwidth, because capacity grows linearly with bandwidth while the noise penalty affects only the logarithmic term, but the advantage is smaller than a naive reading suggests when power is limited. In practice, ultra-wideband sub-terahertz links tend to operate at modest signal-to-noise ratios with simple modulation, relying on bandwidth rather than dense constellations for their speed. Motion creates a further challenge. The Doppler shift caused by movement is proportional to the velocity divided by the wavelength. A device moving at 30 metres per second, roughly highway speed, sees a maximum Doppler shift of about 350 hertz at 3.5 gigahertz and about 30 kilohertz at 300 gigahertz. The time over which the channel stays roughly constant shrinks in proportion, from milliseconds to tens of microseconds. Narrow beams compound the problem, because a beam that is a few degrees wide must be re-aimed as the user moves. Sub-terahertz frequencies are therefore much better suited to stationary or slowly moving users than to cars and trains, at least with current techniques. The near field returns A final physical effect has become central to 6G research and illustrates how high frequencies and large arrays change the textbook picture. Most wireless engineering assumes that the receiver is in the far field of the transmitter, where the arriving wavefront is effectively flat. In the far field, an array can steer a beam toward a direction but cannot focus it at a particular distance. The boundary between near and far field is often taken as the Fraunhofer distance, twice the square of the array's largest dimension divided by the wavelength. For a 20-centimetre array at 3.5 gigahertz, it is roughly 90 centimetres, so every user is in the far field. For a 50-centimetre array at 10 gigahertz, it is about 17 metres. For a 10-centimetre array at 300 gigahertz, it is about 20 metres. As arrays grow and wavelengths shrink, a meaningful share of users finds itself inside the near field, where wavefronts are spherical. That has consequences in both directions. Standard far-field beamforming loses some efficiency for near-field users, so channel models must account for curvature; the Release 19 channel modelling study in 3GPP, completed in 2025, added near-field propagation and spatial non-stationarity across very large arrays to the standard model for the first time. But the near field also offers a new capability. An array can focus energy at a specific point in space rather than along a direction, so that two users in the same direction but at different distances can be served separately. The same property enables precise localisation, because the curvature of the wavefront reveals range as well as angle. Taken together, these effects explain why sub-terahertz communication is neither magic nor futile. With large apertures, narrow beams and short distances, the links are strong and very fast. Over longer distances, absorption and weather impose costs that must be budgeted. Around corners and through walls, the signals largely fail, and the network must provide alternatives. The physics, in other words, does not decide whether terahertz communication works. It decides where it works, and what the rest of the network must look like for it to be useful. Chapter 3: Building Radios for the Terahertz Gap For most of the twentieth century, the region between about 100 gigahertz and 10 terahertz was known to physicists as the terahertz gap. Below it, electronics worked: transistors and vacuum tubes could generate and amplify signals. Above it, photonics worked: lasers and photodetectors could handle infrared and visible light. In between, neither approach was efficient. Electronic devices ran out of speed, because charges could not cross transistor junctions fast enough, and photonic devices ran out of energy, because the photon energy at terahertz frequencies is comparable to the thermal energy of room-temperature matter. The gap has narrowed dramatically over the past two decades, but its legacy explains why the hardware for sub-terahertz communication remains the hardest part of the problem. Transistors at the edge The speed of a transistor is usually summarised by two figures. The transition frequency is the frequency at which its current gain falls to one; the maximum oscillation frequency is the frequency at which its power gain falls to one. A transistor cannot usefully amplify above its maximum oscillation frequency, and in practice amplifiers and oscillators perform well only at a fraction of it, often a third to a half. Three families of semiconductor technology dominate the sub-terahertz field. Indium phosphide, a compound semiconductor, produces the fastest transistors: both heterojunction bipolar transistors and high-electron-mobility transistors in indium phosphide have demonstrated maximum oscillation frequencies above 1 terahertz in research processes. Indium phosphide devices deliver the best gain and output power at 300 gigahertz and above, but the material is expensive, the wafers are small, and integration with large amounts of digital logic is difficult. Silicon-germanium bipolar transistors, built in BiCMOS processes that combine them with conventional silicon logic, occupy the middle ground. Commercial and near-commercial processes offer maximum oscillation frequencies in the range of about 300 to 500 gigahertz, with research processes reaching higher. Silicon-germanium is a natural choice for D-band radios and for many 140 and 300 gigahertz research chips, because it combines adequate speed with the ability to integrate control circuitry on the same die. Bulk and silicon-on-insulator CMOS, the technology of mainstream processors, is slower at the highest frequencies, with maximum oscillation frequencies of a few hundred gigahertz in advanced nodes. Its overwhelming advantage is cost and integration. A CMOS chip can carry hundreds of transceivers, phase shifters, digital beam control and baseband processing at consumer volumes. Most millimetre-wave phones today use CMOS or silicon-on-insulator radio chips, and the industry's ambition for sub-terahertz devices in consumer products depends heavily on CMOS reaching acceptable performance in the D-band and possibly beyond. Gallium nitride, which has transformed power amplifiers at microwave and lower millimetre-wave frequencies, is also being pushed upward, and research devices have reached into the D-band. Its appeal is high output power per device. Whether it becomes practical above 150 gigahertz remains an open question. The power problem Output power is the most persistent limitation of sub-terahertz electronics. A mid-band macro base station radiates tens of watts per antenna port, and a phone's power amplifier delivers around a fifth of a watt. At 300 gigahertz, a single transistor amplifier in silicon-germanium typically delivers a few milliwatts; indium phosphide amplifiers do better, reaching tens of milliwatts in research designs, but at modest efficiency. A useful record illustrates the state of the art: in June 2025, Keysight, NTT and NTT Innovative Devices reported a data rate of 280 gigabits per second in the 300 gigahertz band using an indium phosphide power amplifier operating at an output power of 0 decibels relative to a milliwatt, which is one milliwatt, with saturated output around 9 decibels relative to a milliwatt, about 8 milliwatts. The signal used 256-level quadrature amplitude modulation at 35 gigabaud, and digital predistortion was needed to correct the amplifier's non-linearity. That result is impressive and revealing at once. The data rate is extraordinary. The power is tiny. Sub-terahertz systems compensate in the only way available: by combining the output of many amplifiers in an array, so that the total radiated power and the array gain both rise with the number of elements. A 256-element array with a milliwatt per element radiates a quarter of a watt in total, and the coherent combining of the beam adds substantial gain on top. The approach works, but it requires a large number of carefully calibrated transmit chains, each consuming power and generating heat in a very small area. Efficiency compounds the challenge. A power amplifier converts direct-current power from the supply into radio-frequency output. At mid-band frequencies, well-designed amplifiers in base stations achieve efficiencies of 40 percent or more. At sub-terahertz frequencies, efficiencies are much lower, often a few percent to perhaps ten or fifteen percent in the best designs. For a handheld device with a battery and a thermal budget of a few watts, that efficiency gap is a hard barrier to sustained high-power transmission. It is one reason why most realistic scenarios for sub-terahertz devices involve short bursts, fixed installations with mains power, or downlink-heavy traffic where the base station or access point does most of the transmitting. Phase noise and frequency generation A radio's local oscillator sets the carrier frequency. Any jitter in its phase spreads energy across neighbouring frequencies and corrupts the modulated signal. This phase noise becomes worse as frequency rises, for a simple reason: high-frequency carriers are usually generated by multiplying a lower, cleaner reference. Multiplying a signal's frequency by a factor N multiplies its phase fluctuations by the same factor, which raises the phase-noise power by 20 times the logarithm of N decibels. Generating 300 gigahertz from a 10 gigahertz reference with a multiplication factor of 30 therefore raises the phase noise by about 30 decibels. High phase noise has two consequences. It limits the modulation order, because dense constellations such as 256-level QAM require precise phase. And it affects waveform design. Orthogonal frequency-division multiplexing, the waveform of 4G and 5G, divides a channel into many closely spaced subcarriers. Phase noise causes leakage between them. The standard remedies are to space the subcarriers more widely, which reduces the duration of each symbol and makes the system more robust, and to insert reference signals that track and correct phase errors. 5G already introduced a phase-tracking reference signal for millimetre-wave bands. Sub-terahertz systems would need wider subcarrier spacing still, and some researchers argue for single-carrier waveforms that are inherently more tolerant of phase noise and allow power amplifiers to run closer to saturation. This debate matters because 3GPP's first 6G decisions apply to the bands where 6G will first be deployed, not to sub-terahertz frequencies. The Release 20 study has settled on cyclic-prefix OFDM for the 6G downlink and on both OFDM and DFT-spread OFDM, a single-carrier-like variant, for the uplink. Those choices suit FR1 and FR3. If and when 3GPP specifies a sub-terahertz band, the waveform question will return in a sharper form. Converting between analogue and digital At the edge of every radio, analogue-to-digital and digital-to-analogue converters translate between the continuous signal and the numbers that processors handle. A converter must sample at least twice as fast as the bandwidth it captures, and it must resolve enough bits to support the modulation. A 10 gigahertz-wide channel requires sampling at more than 20 billion samples per second per real channel. Converters at that rate exist, but they consume significant power, and their power consumption scales roughly in proportion to the sampling rate and exponentially with the number of effective bits. Multiplying this by the number of antenna elements in a large array produces an untenable power budget if every element has its own high-resolution converter. The design space therefore splits into three options. Analogue beamforming applies phase shifts to the signal at radio frequency and combines it into a small number of converter chains; it is power-efficient but can form only one or a few beams at a time. Fully digital beamforming gives every element its own converter; it is flexible and fast but power-hungry. Hybrid beamforming uses a modest number of digital chains, each feeding a sub-array of analogue phase shifters. Most millimetre-wave products today use analogue or hybrid architectures. Research has also explored converters with very few bits, even single-bit converters, on each element. Large arrays can recover much of the lost information by combining many coarse measurements, though at a cost in spectral efficiency. The data rate after conversion presents its own challenge. A 300 gigabit per second link must be fed by processors and memory interfaces fast enough to keep up, and the channel coding and decoding for such rates require dedicated hardware. Decoder throughput, rather than radio performance, may set the practical ceiling on terahertz link speeds in some devices. Arrays, packaging and beam alignment Antenna elements at 300 gigahertz are half a millimetre apart. Feeding them with transmission lines on a circuit board would lose most of the signal, so the antennas are built directly on the chip or in the package, close to the transceiver circuits. Antenna-in-package designs, already used for millimetre-wave phones, become the only practical approach at sub-terahertz frequencies. Lenses made of dielectric materials can add gain cheaply for fixed links. Thermal management in such dense arrays is difficult. Hundreds of transceivers each dissipating tens of milliwatts in a square centimetre generate heat densities that challenge passive cooling. For a base station or access point mounted on a ceiling, active cooling or heat sinks are acceptable. For a phone, they are not. The narrow beams created by these arrays must be aligned. In a sparse sub-terahertz channel with few paths, the initial search for a good beam pair, known as beam training, can take many measurements if done naively: an array with a few-degree beamwidth must try dozens or hundreds of directions at each end. Practical schemes use hierarchical search, starting with wide beams and narrowing them; out-of-band information, such as position estimates from lower-frequency signals or from sensors on the device; and learned predictions of where the best beam is likely to be. The most practical idea is to rely on a lower-frequency link to do the hard work of discovery and control. A device might connect first at mid-band or FR3, report its position and orientation, and only then be handed a sub-terahertz beam already pointed roughly in the right direction. Wide bandwidths also cause beam squint in arrays that use phase shifters. A phase shift is the correct delay for only one frequency, so across a very wide band the beam points in slightly different directions at the low and high edges of the channel. For channels of several gigahertz, the error can exceed the beamwidth. True-time-delay elements, which delay the signal rather than rotate its phase, remove the problem but are harder to build compactly. A link budget in practice Putting these pieces together in a single calculation shows where the margins lie. Consider an access point on the ceiling of an office, serving a laptop 20 metres away at 140 gigahertz, in the D-band. The numbers below are illustrative design values chosen to be plausible for current technology, not the specifications of a particular product. Suppose the access point has an array of 64 elements, each driven by an amplifier delivering 5 decibels relative to a milliwatt, about 3 milliwatts. The total transmitted power is then about 23 decibels relative to a milliwatt, some 200 milliwatts. Coherent beamforming across 64 elements adds about 18 decibels of array gain, and each element might contribute 5 decibels of its own gain, giving an effective radiated power of about 46 decibels relative to a milliwatt in the direction of the beam. The laptop has a smaller array of 16 elements, providing about 12 decibels of array gain plus 5 decibels per element, or 17 decibels in total. Free-space path loss at 140 gigahertz over 20 metres, calculated for isotropic antennas, is about 101 decibels. Gaseous absorption over that distance is negligible. The received power is therefore about 46 minus 101 plus 17, or roughly minus 38 decibels relative to a milliwatt. On the noise side, a 4 gigahertz channel has a thermal noise floor of about minus 78 decibels relative to a milliwatt. A receiver noise figure of 10 decibels, typical of sub-terahertz front ends, raises this to minus 68. The resulting signal-to-noise ratio is about 30 decibels before any implementation losses. Allowing 5 decibels for imperfect beam alignment, phase noise and hardware impairments leaves around 25 decibels, enough in principle for dense modulation and a Shannon limit above 30 gigabits per second. Now let a person walk between the access point and the laptop. Body blockage at this frequency can easily cost 20 to 30 decibels. The signal-to-noise ratio falls to around zero, and the link either drops to a very low rate or fails. The margin that looked generous on paper is consumed by a single obstacle. That is why every serious design for sub-terahertz access assumes either a second access point at a different angle, a strong reflected path that the system can switch to within milliseconds, or both. The link budget is comfortable; the availability budget is not. Photonics and hybrid approaches Some of the fastest sub-terahertz links have used photonic generation rather than electronics. Two lasers with slightly different optical frequencies are combined and shone onto a fast photodiode, which produces an electrical signal at the difference frequency. Because laser frequencies can be tuned precisely and modulated with data using mature fibre-optic components, photonic transmitters can produce clean, very wideband sub-terahertz signals. Research groups in Japan, Germany and elsewhere have used this approach to demonstrate links of hundreds of gigabits per second. Photonic approaches fit naturally with a distributed architecture in which a central unit generates signals and sends them over optical fibre to many simple antenna heads, a concept often called radio-over-fibre. That vision connects directly to the cell-free networks examined later. For consumer devices, photonic generation is currently too bulky and power-hungry; for fixed infrastructure, it is a credible option. What exists, and what is still missing The record of demonstrations is long. Japanese, Korean, European and American companies and universities have shown sub-terahertz links carrying tens to hundreds of gigabits per second in laboratories and over outdoor distances of tens to hundreds of metres. In 2017 the IEEE published its 802.15.3d amendment, which specifies a physical layer for point-to-point wireless links in the band from 252 to 321 gigahertz, targeting data rates of up to 100 gigabits per second for uses such as data centre interconnection, kiosk downloading and fronthaul. Commercial D-band backhaul radios have appeared from microwave equipment vendors, offering capacities of tens of gigabits per second over short hops in cities. What has not yet been demonstrated at scale is a mobile, handheld sub-terahertz device that can maintain a link while its user walks, turns and puts it in a pocket. The physical obstacles discussed in the previous chapter, combined with the power, thermal and alignment problems described here, make that the hardest version of the problem. The realistic path is incremental. Fixed and nomadic uses come first: wireless backhaul and fronthaul, links between servers and racks, high-capacity connections to stationary headsets or displays, and sensing applications where the network itself carries the radio. Mobile uses follow when silicon processes, packaging and beam management have matured further. For the purposes of the 6G timeline, this has a clear implication. The first 6G specification, Release 21, focuses on the bands where commercial deployment is imminent: existing 5G bands through spectrum sharing, and the new upper mid-band around 7 gigahertz. Sub-terahertz spectrum is a candidate for later releases. The hardware work now underway will decide how much later. Meanwhile, the middle path through FR3 is where both the engineering and the spectrum politics are concentrated, and it is the subject of the next chapter. Chapter 4: The Middle Path and the Politics of Spectrum If the terahertz band is the frontier of 6G research, the range between about 7 and 15 gigahertz is its working territory. Engineers sometimes call it the upper mid-band, sometimes the centimetric range, and in 3GPP documents it has acquired the informal label FR3, a third frequency range alongside the FR1 and FR2 of 5G. Its appeal is easy to state. It offers channels far wider than today's mid-band, it propagates well enough to be deployed from existing macro sites with larger antenna arrays, and the hardware to use it is an evolution of existing 5G equipment rather than a leap into new semiconductor territory. Its difficulty is equally easy to state: almost all of it is already in use by someone else. Why the middle matters The 5G experience left the industry with a gap. Mid-band spectrum around 3.5 gigahertz delivered most of 5G's value because it balanced coverage and capacity, but operators in most countries hold only 100 to 200 megahertz of it. Mobile data traffic has continued to grow, though more slowly than in the 2010s, and the industry argues that by the end of this decade the mid-band will be saturated in dense cities. Millimetre wave can add capacity in hotspots but not across a whole city at acceptable cost. The question is where the next large block of spectrum suitable for wide-area deployment will come from. The upper mid-band is the obvious candidate. Signals at 7 to 8 gigahertz suffer roughly twice the isotropic path loss of signals at 3.5 gigahertz, a difference of about six to seven decibels, plus somewhat higher losses through walls and foliage. But the shorter wavelength allows an antenna panel of the same physical size to hold about four times as many elements, and an array with four times the elements provides about six decibels more gain at each end of a beamformed link. Vendor and operator studies have argued that, with arrays of several hundred elements on the base station side, FR3 can deliver coverage approaching that of 3.5 gigahertz from the same macro sites, at least for outdoor users and for the downlink. The uplink, where the phone's small antenna and limited power constrain the link, is harder, which is one reason the 6G design work has placed emphasis on uplink improvements and on combining FR3 with lower bands. The bandwidth on offer is the reward. 3GPP's 6G study has agreed that the 6G radio will support channels of up to 400 megahertz in the 7 gigahertz range, with devices expected to handle up to 400 megahertz in the downlink and 200 megahertz in the uplink. That is four times the largest channel common in 5G mid-band. If an operator could obtain 400 megahertz in FR3, and use it from its existing sites with new radios, it would roughly triple or quadruple the capacity of its macro layer without building a single new tower. That economic logic, more than any physics, explains why the upper mid-band has become the centre of gravity in 6G planning. Who is already there Spectrum is not empty space waiting to be claimed. The frequencies between 7 and 15 gigahertz support a dense web of existing services, many of them critical to governments. The range from about 7.25 to 7.75 gigahertz carries satellite downlinks, and 7.9 to 8.4 gigahertz carries satellite uplinks, in a part of the spectrum known as X-band that many countries reserve for military and government satellite communications. The band from 8.025 to 8.4 gigahertz also supports Earth exploration satellites transmitting imagery and science data to ground stations. Just above, around 8.4 gigahertz, space agencies receive signals from deep-space probes, signals so faint that any interference could blind the receiving antennas. Fixed point-to-point microwave links, used for backhaul and for utility and public-safety networks, occupy portions of these bands in many countries. Around 15 gigahertz, the incumbents include fixed links, space research and aeronautical systems, and the band from 15.35 to 15.4 gigahertz is reserved for radio astronomy, a passive service that observes extremely weak natural emissions and cannot tolerate interference. Below 7 gigahertz, the band from 4.4 to 4.8 or 4.9 gigahertz is used in several countries, including the United States, for military aeronautical telemetry and other government systems. Japan, by contrast, has already assigned parts of the range around 4.5 to 4.9 gigahertz to 5G, showing that the same spectrum can be treated very differently by different national administrations. The upper 6 gigahertz band from 6.425 to 7.125 gigahertz is a separate case and the subject of perhaps the most public dispute. It is used today by fixed links and fixed satellite uplinks, and in many countries it has become the domain of unlicensed Wi-Fi. The United States opened the whole band from 5.925 to 7.125 gigahertz to unlicensed use in 2020, allowing Wi-Fi 6E and Wi-Fi 7 devices to use channels up to 320 megahertz wide. Other countries, including China, have taken the opposite approach and set the upper part of the band aside for mobile networks. The international process Spectrum for mobile networks is negotiated internationally at World Radiocommunication Conferences, held every three to four years under the International Telecommunication Union. A conference does not assign spectrum to operators; it amends the Radio Regulations, the international treaty that allocates frequency bands to services and, where relevant, identifies bands for International Mobile Telecommunications, the ITU's term for cellular systems. National regulators then decide how to license the bands domestically. An identification for IMT is not mandatory, but it creates a global signal that drives equipment makers to support a band, which in turn lowers costs. The 2023 conference, WRC-23, identified the upper 6 gigahertz band for IMT in ITU Region 1, which covers Europe, Africa and the Middle East, and in a number of countries in other regions. It also set the agenda for WRC-27, scheduled for late 2027. Agenda item 1.7 of that conference asks for studies of sharing and compatibility, and of the technical conditions for IMT, in three ranges: 4.4 to 4.8 gigahertz, 7.125 to 8.4 gigahertz or parts of it, and 14.8 to 15.35 gigahertz. The language of the agenda item stresses the protection of services that already hold primary allocations in these bands and in adjacent ones, including stations operating in international waters and airspace. Table 3 summarises the candidate bands, their principal incumbents and the regional positions as they stood in late 2026. The entries simplify a complex picture, and positions may shift before the conference. Table 3. Candidate mid-band ranges for early 6G deployment. Band Main incumbents Status in late 2026 4.4 to 4.8 GHz Military aeronautical and government systems WRC-27 agenda item 1.7; already 5G in parts of Japan 6.425 to 7.125 GHz Fixed links, satellite uplinks, Wi-Fi IMT-identified in Region 1 at WRC-23; EU draft plan favours mostly mobile 7.125 to 8.4 GHz Military and government satellites, Earth exploration WRC-27 agenda item 1.7; US law excludes 7.4 to 8.4 GHz from auction 14.8 to 15.35 GHz Fixed links, space research, adjacent radio astronomy WRC-27 agenda item 1.7 Sources: ITU WRC-27 agenda item 1.7; WRC-23 outcomes; One Big Beautiful Bill Act (US, July 2025); RSPG draft 6G spectrum roadmap (February 2026). Three regional stories The United States approaches 6G spectrum through the lens of its domestic auction system and its large federal spectrum holdings. In July 2025, a budget law commonly called the One Big Beautiful Bill Act restored the Federal Communications Commission's authority to auction spectrum, which had lapsed in 2023, and extended it to 2034. The law required a pipeline of at least 800 megahertz of spectrum between 1.3 and 10.5 gigahertz to be made available for commercial use, with specific attention to the upper C-band between 3.98 and 4.2 gigahertz. It also carved out two ranges from the auction requirement: the lower 3 gigahertz band from 3.1 to 3.45 gigahertz, and the range from 7.4 to 8.4 gigahertz, both important to the Department of Defense. The law funded studies of other ranges, including 7.25 to 7.4 gigahertz, by the National Telecommunications and Information Administration. In effect, the United States has signalled that most of the 7 to 8.4 gigahertz range will not be available for mobile use domestically, while the lower edge of it and other mid-band ranges remain in play. Separately, the Federal Communications Commission has examined other upper mid-band options, including spectrum around 12.7 gigahertz. Europe has moved through its advisory Radio Spectrum Policy Group, which brings together national regulators. In February 2026 the group published a draft opinion on a 6G spectrum roadmap. Its central recommendation was to make the upper 540 megahertz of the upper 6 gigahertz band available for mobile networks as the first major 6G band, with the remaining 160 megahertz at the lower end held in reserve until after WRC-27, when the balance between Wi-Fi and mobile use can be reconsidered. The draft also identified the lower edge of the 7 gigahertz range, from 7.125 to 7.25 gigahertz, as a band to be studied for mobile use at the conference, while treating sub-terahertz frequencies as a subject for longer-term research. The opinion reflected a compromise between mobile operators, who argued that the upper 6 gigahertz band was essential for 6G capacity, and Wi-Fi advocates, who argued that home and enterprise networks carry most data traffic and need the whole band for wide channels. China has, since 2023, identified the full 6.425 to 7.125 gigahertz range for mobile networks, and Chinese vendors and operators have been among the strongest proponents of upper mid-band 6G. China's positions at WRC-27 are expected to favour broad identifications in the 7 gigahertz range. Other Asian administrations, including Korea and Japan, have pursued their own mixes of FR3 study and early sub-terahertz research. The resulting landscape is fragmented. A 6G phone sold globally may need to support the upper 6 gigahertz band in Europe and China, other parts of the 7 gigahertz range where they are released, and perhaps 4.4 to 4.8 gigahertz in Japan and elsewhere, while working with entirely different upper mid-band allocations in the United States. Equipment designers can handle a degree of fragmentation, as they do today with dozens of LTE and 5G bands, but every additional band adds filters, testing and cost. Fragmentation also has a physical dimension inside the phone. Each new band requires filters to keep the device's transmissions within the band and to reject strong signals from neighbouring services, power amplifiers and low-noise amplifiers matched to the frequency, and antennas that perform adequately in it. At 7 to 8 gigahertz, antennas are small enough that a phone can carry a modest array, perhaps four or eight elements in a module, which improves the uplink compared with today's mid-band designs. But every additional module competes for space with cameras, batteries and existing antennas in a device whose internal volume is already fully used. Device makers have therefore pressed in the standards process for a limited set of band combinations and for capabilities that can be phased in, and the agreed device bandwidth of 200 megahertz in the uplink, half the downlink figure, reflects the cost and power consumption of wideband transmit chains in handsets. Sharing instead of clearing The traditional way to make spectrum available for mobile networks was to clear it: move the incumbents to other bands or retire them, then auction the cleared range. That approach is increasingly impractical for the upper mid-band. Military satellite systems cannot easily be moved, because satellites in orbit cannot change frequency and new ones take a decade to design and launch. Radio astronomy cannot move, because the natural emissions it studies occur at fixed frequencies. The likely future is shared spectrum. Several mechanisms are available, and they differ in sophistication. Geographic exclusion zones keep mobile transmitters a safe distance from sensitive receivers, such as satellite ground stations and observatories. Power and antenna constraints limit emissions in certain directions, for example toward the geostationary orbit. Dynamic sharing, of the kind used in the American Citizens Broadband Radio Service at 3.55 to 3.7 gigahertz, allows lower-priority users to transmit when and where higher-priority users are absent, coordinated by an automated database. More advanced proposals allow mobile networks to sense incumbent activity directly. The physical properties of the upper mid-band help. Base stations with large arrays can steer energy toward users and away from the horizon or the sky, and the narrow beams reduce the spread of interference. A satellite receiver in orbit sees the aggregate of many terrestrial transmitters, so the studies under agenda item 1.7 focus heavily on how much aggregate interference a large deployment of 6G base stations would produce in the direction of satellites. The results of those studies, which are being conducted in ITU working parties through 2026 and into 2027, will set the technical conditions attached to any identification. The contest with Wi-Fi The upper 6 gigahertz debate deserves particular attention because it sets two models of wireless against each other. Mobile operators deploy licensed networks, planned and managed, with predictable quality and wide-area mobility. Wi-Fi is deployed by households and businesses, unlicensed, unmanaged and enormously popular. Studies by both camps disagree about the share of traffic each carries, but there is no doubt that a large majority of data used on smartphones flows over Wi-Fi when users are at home or at work. Wi-Fi 7, the generation of Wi-Fi standards finalised in 2024, can use channels of 320 megahertz, but only where enough contiguous spectrum is available. In the United States, with the full 1,200 megahertz of the 6 gigahertz band, several such channels fit. In a country that gives Wi-Fi only the lower 500 megahertz, at most one fits. Wi-Fi advocates argue that denying the upper band to unlicensed use will cripple the next generation of home networking. Mobile operators argue that 6G cannot deliver the city-wide capacity that governments want without it. There is no technical answer to this dispute, because it concerns which kind of network a society wants to invest in. The interesting technical point is that 6G's architecture may blur the boundary. A dense, distributed 6G network with access points inside buildings looks more like an enterprise Wi-Fi network than a traditional cellular one. Whether that convergence leads to shared bands, neutral-host networks or continued separation is one of the open questions of the next decade. What this means for 6G The spectrum process has a consequence that is easy to overlook. The first 6G specifications will be finished in early 2029, but WRC-27 will not conclude until late 2027, and national auctions or assignments typically follow conferences by two to five years. In the meantime, the first 6G networks will run on spectrum operators already hold, using the multi-radio spectrum sharing that 3GPP has made the baseline of its migration design, and on the upper 6 gigahertz band where it is available. The upper mid-band proper, from 7.125 gigahertz upward, will become available unevenly, country by country, through the early 2030s. That timetable leaves the sub-terahertz bands further out still. The ITU's WRC-19 conference identified four ranges between 275 and 450 gigahertz, totalling 137 gigahertz, for land mobile and fixed service applications, subject to protection of passive science services, and the US Federal Communications Commission opened bands above 95 gigahertz to experimental licences and some unlicensed use in 2019. Those steps keep the door open, but no mobile identification of sub-terahertz spectrum is on the WRC-27 agenda. Commercial sub-terahertz mobile networks, if they come, are a matter for the 2030s and later conferences. The upshot is that the shape of early 6G will be decided as much in the meeting rooms of regulators as in the laboratories of chip designers. The engineering of the upper mid-band is challenging but tractable. The politics of sharing it with satellites, militaries and Wi-Fi is less predictable, and its outcome will determine how much capacity the new generation actually adds in its first years. Chapter 5: Standards in Motion, from IMT-2030 to Release 21 A mobile generation becomes real when its specifications are frozen. Before that point, it is a collection of research results, prototypes and opinions. After it, chip designers can tape out silicon, equipment makers can build base stations that interoperate with phones from other companies, and operators can plan networks. The machinery that turns research into frozen specifications is slow, consensus-driven and often frustrating to outsiders, but it is the reason a phone bought in Seoul works on a network in São Paulo. Understanding how 6G is moving through that machinery explains a great deal about which ideas will appear in the first networks and which will wait. Two institutions, one process Two bodies matter most. The International Telecommunication Union, a United Nations agency, defines what qualifies as a new generation of International Mobile Telecommunications. It does this through Working Party 5D of its Radiocommunication Sector, which sets requirements and evaluation criteria and then assesses candidate technologies against them. IMT-2000 corresponded to 3G, IMT-Advanced to 4G, IMT-2020 to 5G, and IMT-2030 to 6G. The ITU does not design the technology. That work happens mainly in the Third Generation Partnership Project, a partnership of seven regional standards organisations from Europe, the United States, China, Japan, Korea and India. Its members include equipment vendors, chipmakers, operators, device makers and research institutes, numbering in the hundreds. 3GPP organises its work into releases, each a coherent set of features with agreed freeze dates. It submits its technology to the ITU as a candidate for each IMT generation, and in practice its submission has become the global standard. 3GPP's working method has three stages. Stage 1 defines service requirements: what the system must do for users. Stage 2 defines architecture: which functions exist and how they interact. Stage 3 defines detailed protocols and signalling, down to the encoding of messages. Each release passes through studies, which produce technical reports exploring options, and work items, which produce the technical specifications that equipment must follow. The final milestone is the freeze of the formal message syntax, known as the ASN.1 freeze, after which implementations can be built against a stable target. The way decisions are reached inside 3GPP shapes what comes out. Technical work is divided among working groups, each with a narrow remit: one handles the physical layer, another the radio protocols, another radio-frequency performance requirements, another the system architecture, and so on. Companies submit written contributions, often thousands for a single meeting, proposing designs and presenting simulation results. Decisions are made by consensus, which in practice means that a proposal goes forward when no significant group of companies sustains an objection. Formal votes are rare. The process favours compromise and incremental change, rewards companies that can field large delegations and run extensive simulations, and makes it difficult for any single company or country to impose its preferences. Around 3GPP, a ring of regional and national programmes shapes the agenda. The European Union funded the Hexa-X and Hexa-X-II flagship projects, which brought together vendors, operators and universities to define a European view of 6G. In North America, the Next G Alliance, organised by the standards body ATIS, published roadmaps and white papers, including work on channel measurements for sensing and for the 7 to 24 gigahertz range. China's IMT-2030 (6G) Promotion Group, established in 2019, coordinates research among Chinese operators, vendors and universities. Japan, Korea and India have their own alliances and national programmes. These bodies do not write standards, but they shape the contributions that companies bring into the standards process, and they reflect national priorities in industrial policy, security and spectrum. The geopolitical dimension has grown since the 5G era. Disputes over equipment suppliers, export controls on semiconductors and concerns about the security of network equipment have made governments more attentive to who contributes to standards and who holds the essential patents. So far, the global 3GPP process has held together, and the September 2026 decision to submit a single 6G radio technology to the ITU suggests continued commitment to one worldwide standard. That outcome is valuable and not guaranteed. A split into regional standards would raise costs for everyone and fragment the device market. What the ITU has asked for The ITU's vision for IMT-2030, set out in Recommendation ITU-R M.2160, was approved in late 2023. As described in Chapter 1, it defines six usage scenarios and fifteen capabilities. It also names four overarching aspects that should guide the design: sustainability, connecting the unconnected, ubiquitous intelligence, and security, privacy and resilience. Working Party 5D has since produced the documents that turn this vision into a test. These include the minimum technical performance requirements that a candidate must meet, the evaluation methodology that independent evaluation groups will use, and the submission template. The submission window for candidate radio interface technologies opens at the Working Party's meeting in February 2027 and closes in February 2029. Independent evaluation groups, typically formed by national bodies, universities and industry associations, will assess submissions during and after that window. The ITU expects to complete the IMT-2030 specifications around 2030. The structure of these requirements reveals the priorities of the next decade. Alongside the familiar metrics of data rate and latency, they include coverage, positioning accuracy and sensing-related capabilities. They pay explicit attention to energy efficiency. And they reflect a political dimension that was less visible in earlier generations: many administrations, especially in developing countries, pushed for the inclusion of ubiquitous connectivity to ensure that 6G does not become a technology that serves only dense, wealthy cities. Release 20: the study phase 3GPP's Release 20 has a dual role. It carries the last set of enhancements to 5G-Advanced, covering topics such as energy savings, artificial intelligence in the radio, satellite integration, coverage, mobility and sensing. And it contains the studies for 6G. The 3GPP project has described Release 20 as the final "5G-Advanced only" release; from Release 21 onward, 6G specifications will be produced alongside continued 5G evolution. The 6G studies began with service requirements. The Stage 1 study on 6G use cases and service requirements was completed in 2025, producing a draft specification of 6G system requirements. It covers artificial intelligence, integrated sensing, ubiquitous connectivity including satellites, industrial use cases and more. The architecture study, organised under the systems group, is examining whether 6G needs a new core network or an evolution of the 5G core, how artificial intelligence functions should be integrated, and how sensing should be exposed as a network service. Further studies address security, resilience, mission-critical services and application enablement. The radio studies began in mid-2025 under the radio access network groups. They started with scenarios and requirements, documented in a technical report on 6G radio scenarios and requirements, and proceeded to the physical layer, protocols and radio-frequency requirements. A distinctive feature of the 6G studies has been an emphasis on simplicity and on learning from 5G's mistakes. Delegates have repeatedly argued that 5G accumulated too many options, too many device capabilities and too many deployment variants, raising the cost of implementation and testing. The 6G study has tried to keep the number of options small. The first decisions By mid-2026 the radio study had settled several foundational choices. The downlink waveform will be cyclic-prefix orthogonal frequency-division multiplexing, as in 4G and 5G. The uplink will support both CP-OFDM and DFT-spread OFDM, and, unlike 5G, DFT-spread OFDM will support multi-layer transmission of up to two layers, allowing power-limited devices to use multiple antennas for higher uplink throughput. Uniform quadrature amplitude modulation remains the basis of the constellation design. The channel codes from 5G, low-density parity-check codes for data and polar codes for control information, will be reused with refinements aimed at simpler device implementation. The frame structure will remain slot-based with scalable numerology. The system bandwidth will range from a minimum of about 3 megahertz in narrow allocations to a maximum of 400 megahertz, with the larger figure aimed at the 7 gigahertz range. For devices in that band, the agreed capability is up to 400 megahertz in the downlink and 200 megahertz in the uplink. To some observers, these decisions are disappointing. They show that 6G's air interface will be an evolution of 5G rather than a radical departure, at least in the bands where it will first be deployed. That is a deliberate choice. Waveforms and codes are not the bottleneck for performance in mid-band spectrum; the gains come from wider bandwidth, larger antenna arrays, better uplink design, improved energy efficiency and smarter scheduling. Reusing proven components lowers the risk, cost and time of bringing 6G to market. More radical choices, such as new waveforms for sub-terahertz bands or AI-designed signal formats, remain possible in later releases. At the September 2026 radio plenary in Madrid, delegates agreed that the 6G system will be submitted to the ITU as a single radio technology, under the name "6G, Release 21 and beyond." They approved three reference device types to guide the definition of device capabilities, aimed at enhanced mobile broadband, massive Internet of Things connections and fixed wireless access, with a fourth type for extended reality or automotive uses still under discussion. They agreed that 1024-level QAM will be supported for fixed wireless access, while finding no consensus on 4096-level QAM. The migration question The most contested issue of 2026 has been how 6G will coexist with and replace 5G. It sounds like an operational detail. It is in fact a strategic choice with large consequences for cost and for the speed of adoption. The background is the history of 5G. The first 5G specifications, frozen in late 2017, defined a non-standalone mode in which 5G radios were attached to the 4G core network and relied on an LTE carrier for control. That allowed operators to launch 5G quickly, but it delayed the arrival of the full 5G architecture. Standalone 5G, with its own core network and its own control signalling, followed in 2018, yet years later many networks around the world still ran in non-standalone mode. The features that justified 5G's architecture, such as network slicing and low-latency services, were slow to reach users. Operators and vendors have described the multiplicity of 5G deployment options as a source of fragmentation, delayed rollouts and costly retrofits. For 6G, 3GPP has agreed a baseline called multi-radio access technology spectrum sharing, abbreviated MRSS. Under MRSS, 6G and 5G can operate on the same carrier at the same time, with resources shared dynamically between them. An operator can introduce 6G into its existing spectrum without first clearing 5G users out of it, much as dynamic spectrum sharing allowed 5G to share LTE carriers. The September 2026 plenary added a hard requirement that efficient sharing of at least downlink control channel resources between 5G and 6G must be supported. That addressed a concern that running both systems on a narrow carrier would waste a large share of its capacity on duplicated control signalling. The open question is whether 6G should offer anything beyond MRSS. Three additional options have been debated: dual connectivity anchored on 6G, in which a device connects to both systems with 6G in control; dual connectivity anchored on 5G, resembling the non-standalone approach of early 5G; and a dual-stack or dual-registration approach in which a device keeps separate connections to both systems. Industry reporting from the Madrid meeting indicated that a majority of operators expressing a view favoured the 6G-anchored option, but a coalition that included major vendors, a leading device maker and several large operators argued for MRSS alone, on the grounds that every additional option repeats the fragmentation of 5G. The plenary deferred the decision on an additional option to its December 2026 meeting, and endorsed a narrowed version of the dual-stack option focused on voice over 5G alongside 6G data, with a final decision for the systems group. The outcome matters for everyone who will buy 6G equipment. A single migration path keeps device and network implementations simpler and makes it more likely that the full 6G architecture, including its new services, will be available from the start. Multiple paths give operators flexibility to match their own spectrum and core network situations but risk repeating the long tail of partial deployments seen in 5G. Release 21 and the road to 2030 At its June 2026 plenaries, 3GPP approved the timeline for Release 21, the first release that will contain normative 6G specifications. Table 4 lists the main milestones of the 6G standards process as they stood in October 2026. Some intermediate dates are still subject to adjustment, and reports of the exact staging of functional freezes have varied. Table 4. Key milestones in the standardisation of 6G. Date Body Milestone Late 2023 ITU-R IMT-2030 framework (M.2160) approved Mid-2025 3GPP 6G radio studies begin in Release 20 June 2026 3GPP Release 21 timeline approved September 2026 3GPP MRSS control-sharing requirement; migration decision deferred February 2027 ITU-R IMT-2030 submission window opens March 2027 3GPP 6G work items approved; normative work begins Late 2027 ITU WRC-27 decides on mid-band identifications 2028 3GPP Functional freezes of physical layer and protocols February 2029 ITU-R IMT-2030 submission window closes March 2029 3GPP Release 21 ASN.1 freeze Around 2030 Industry First commercial 6G networks Sources: 3GPP Release 20 and Release 21 pages; ITU-R WP 5D IMT-2030 process documents; reporting on 3GPP plenaries, June and September 2026. Several things about this timeline deserve emphasis. First, it is roughly a decade after the comparable milestones for 5G, consistent with the ten-year rhythm that has held since the 1980s. Second, it leaves little slack. Industry commentators have pointed to the 2028 Los Angeles Olympics as a natural moment for pre-commercial demonstrations, but a standards-compliant commercial launch before 2030 would require an aggressive implementation schedule. Third, the scope of Release 21 is deliberately focused. It targets the bands and features needed for initial deployment: operation in existing bands through spectrum sharing, the new upper mid-band, improved massive MIMO, better uplink, energy efficiency, integration of artificial intelligence, and initial sensing capabilities. Many ideas discussed in 6G research, including sub-terahertz operation and full cell-free architecture, are candidates for later releases rather than for the first. What is left out of the first release is as informative as what is included. Satellite access, which 5G added in Release 17 through non-terrestrial network features, is expected to be part of the 6G design from the start, reflecting the ITU's emphasis on ubiquitous connectivity and the rapid growth of low-Earth-orbit constellations; how much of it lands in Release 21 rather than its successors depends on the progress of the work items. Sub-terahertz bands have no place in the first release, because no regulator has identified sub-terahertz spectrum for mobile use and the hardware is not ready for handsets. Fully cell-free operation, as described in Chapter 7, is not defined as such, although enhancements to multi-point transmission move in that direction. These are not rejections. 3GPP has always delivered generations in stages, and 5G itself gained many of its most significant features, including its first non-terrestrial and sidelink capabilities, in releases well after the first. The 6G that exists in 2035 will be considerably richer than the 6G frozen in 2029. What the process reveals Standards bodies are sometimes criticised for conservatism, and the early 6G decisions will reinforce that criticism. But the process also reveals what the industry has learned. The emphasis on fewer options, on a single migration path, on reuse of proven components and on energy efficiency reflects the experience of a generation whose deployment was slower and more expensive than its proponents hoped. It also shows where the genuine novelty of 6G lies. The waveform is familiar, but the system around it is being designed with sensing as a service, artificial intelligence as a native function, and architectural room for distributed antennas. Those are the themes of the next three chapters. Each has a place in the 6G studies, each has a firm physical basis, and each could change how wireless networks work more deeply than a new waveform ever could. Chapter 6: When the Network Learns to See Radar and radio communication grew up together. Both emerged from the same early twentieth-century physics, both were transformed by the military research programmes of the Second World War, and both rely on the same basic hardware: oscillators, amplifiers, antennas and receivers. For most of their history, however, they were built as separate systems for separate purposes. A radar transmits a signal designed to be reflected and measured. A communication system transmits a signal designed to carry information to a receiver that knows how to decode it. Integrated sensing and communication, usually abbreviated ISAC, is the proposition that a single network, using largely the same signals and the same infrastructure, can do both. The proposition has moved from academic papers to standards remarkably quickly. The IMT-2030 framework lists integrated sensing and communication as one of its six usage scenarios. 3GPP's Release 19 added channel models for sensing to its standard propagation model. Release 20 includes a study of sensing with 5G-Advanced networks focused on detecting drones, and the 6G studies treat sensing as a service the network offers, not a by-product. To understand why this happened now, and what it can realistically deliver, one has to start with how radar works. What a radar measures A radar estimates three things about a target: how far away it is, how fast it is moving toward or away from the radar, and in which direction it lies. Each measurement depends on a different property of the signal. Range comes from delay. The signal travels to the target and back at the speed of light, so the round-trip delay is twice the distance divided by the speed of light. The precision with which a radar can separate two targets at slightly different ranges, its range resolution, is set by the bandwidth of the signal: the speed of light divided by twice the bandwidth. A 100 megahertz signal resolves targets 1.5 metres apart. A 400 megahertz signal, the maximum agreed for 6G in the 7 gigahertz band, resolves them at about 37.5 centimetres. A 10 gigahertz sub-terahertz signal resolves them at 1.5 centimetres. Accuracy, the precision of the estimate for a single isolated target, can be much finer than resolution if the signal-to-noise ratio is high. Velocity comes from the Doppler shift. A target moving toward the radar compresses the reflected wave and raises its frequency; one moving away lowers it. The shift is proportional to the radial velocity divided by the wavelength, doubled for the round trip. Resolving small velocity differences requires observing the target for a long time, because the frequency resolution of any measurement is the inverse of its duration. Higher carrier frequencies help, because the same velocity produces a larger Doppler shift: at 300 gigahertz, a hand moving at 10 centimetres per second produces a round-trip shift of about 200 hertz, while at 3 gigahertz the same movement produces about 2 hertz. Direction comes from the antenna array. A wave arriving at an angle reaches different elements at slightly different times, and the pattern of phase differences reveals the angle. The angular resolution of an array is roughly the wavelength divided by the array's width. Large arrays at high frequency resolve angles finely, which is one reason the massive MIMO arrays deployed for communication are also excellent sensors. Combined, these measurements produce a picture of the environment in range, velocity and angle. A modern automotive radar at 77 gigahertz with a few gigahertz of bandwidth and a modest antenna array can detect pedestrians, cars and cyclists and track their movement. The question for ISAC is whether a communication network, which already has wide bandwidth, large arrays and dense infrastructure, can produce a similar picture. How a communication signal becomes a radar signal At first sight, communication signals look poorly suited to radar. They carry random data, which changes from one moment to the next, while radar waveforms are usually carefully designed chirps or pulses with known properties. The key insight is that a radar does not need a special waveform; it needs to know what it transmitted. A base station always knows the signal it has sent, including the data. When echoes come back, it can compare them with the original. Orthogonal frequency-division multiplexing makes the comparison particularly convenient. An OFDM signal consists of many subcarriers, each carrying a known symbol for a known duration. The echo from a target at a given range introduces a phase shift that grows linearly across subcarriers, and the echo from a moving target introduces a phase shift that grows linearly across successive symbols. By dividing each received subcarrier symbol by the transmitted one, removing the data, the receiver obtains a grid of phase shifts. A two-dimensional Fourier transform of that grid converts the linear phase patterns into peaks at the targets' ranges and velocities. This technique, studied extensively since the late 2000s, is often called OFDM radar. It works with data symbols or with reference signals, which are already transmitted at known positions. The numerology of the communication signal sets the limits of this radar. The spacing between subcarriers determines the maximum range at which a target can be measured without ambiguity: the speed of light divided by twice the subcarrier spacing. With the 30 kilohertz spacing common in 5G mid-band, that range is about 5 kilometres, comfortably beyond the reach of the echoes. With 120 kilohertz spacing, typical of millimetre-wave bands, it falls to about 1.25 kilometres. The duration of the observation, the number of symbols processed together, sets the velocity resolution, and the time between consecutive symbols used for sensing sets the maximum velocity that can be measured unambiguously. A system that uses only sparse reference signals for sensing, rather than every symbol, must choose their spacing in time and frequency carefully so that drones, cars or people fall within the measurable ranges. Much of the design work in standards consists of fitting these radar requirements into the existing structure of frames, slots and reference signals without disrupting communication. There is a catch that matters for real systems. A base station that transmits and receives at the same time and on the same frequency, as monostatic sensing requires, must hear faint echoes while its own transmitter is blasting energy into its receiver. The leakage can be billions of times stronger than the echoes. Cellular base stations today are not designed for that: in time-division duplex networks they transmit and receive at different times, and in frequency-division duplex networks on different frequencies. Monostatic sensing therefore requires either separate transmit and receive antennas with strong isolation, analogue and digital cancellation of the self-interference, or a pause in transmission while echoes return, which works only for targets far enough away that the echo arrives after the pulse ends. Each approach adds cost or reduces capacity. The alternative is to separate transmitter and receiver. In bistatic sensing, one base station transmits and another receives the reflections. In multistatic sensing, several receivers combine their observations. Devices can take part too: a phone could receive signals reflected from its surroundings, or a base station could sense using signals transmitted by phones. 3GPP's sensing channel model work defined six basic configurations: base station monostatic, base station to base station bistatic, base station to device, device to base station, device monostatic, and device to device. Bistatic and multistatic sensing avoid the self-interference problem but require precise synchronisation in time and frequency between the separated nodes, because any timing error becomes a range error. A timing offset of one nanosecond corresponds to 30 centimetres of path length. What the standards are doing The first concrete 3GPP work on sensing is modest in scope and revealing in its choice of application. The Release 20 study on integrated sensing and communication for 5G New Radio, documented in technical report TR 38.765, concentrates on detecting unmanned aerial vehicles using base stations in monostatic mode. Drones are an attractive first use case for several reasons. They fly above most clutter, so their echoes stand out against the sky. They pose real security and safety concerns around airports, prisons, stadiums and critical infrastructure, and existing detection systems are expensive and patchy. Mobile networks already cover most populated areas with base stations whose antennas point roughly toward the low sky. And drone detection does not raise the acute privacy concerns of sensing people in their homes. The study examines performance metrics such as detection probability, false alarm rate and position accuracy; how sensing measurements should be defined and reported; how the radio network and core network should exchange sensing requests and results; and whether new or existing reference signals should be used. Building on it, the 5G-Advanced programme has been working toward specifications for sensing in Release 20, while the 6G studies are treating sensing as a native service from the outset. The 6G architecture study is examining how a sensing function in the core network would receive requests from authorised applications, coordinate measurements across base stations and devices, fuse the results and expose them through standardised interfaces. In parallel, the Wi-Fi world has been developing its own sensing capabilities. The IEEE 802.11bf amendment adds sensing procedures to Wi-Fi, allowing access points and devices to measure channel changes caused by movement in a room. Wi-Fi sensing has been demonstrated for presence detection, fall detection and gesture recognition, and it shows how much information about the physical world is already encoded in ordinary wireless signals. Uses worth having The applications proposed for ISAC fall into a few groups, and they differ sharply in maturity. The first group is airspace and road monitoring. Detecting and tracking drones, monitoring traffic flows at intersections, detecting intrusions onto railway lines and measuring vehicle speeds are tasks where targets are relatively large, relatively few and in the open. Mid-band and upper mid-band networks with hundreds of megahertz of bandwidth and large arrays can plausibly perform them. Their value lies in coverage: a network that is already deployed everywhere can provide a sensing layer at low marginal cost. The second group is industrial and logistics sensing. Inside factories and warehouses, networks could track robots, forklifts and goods, detect people entering dangerous zones and monitor machinery. These environments are controlled, often private, and willing to invest in dense infrastructure. Private 5G and 6G networks in industrial sites are a natural early home for sensing. The third group is fine-grained sensing at high frequency: detecting gestures, measuring breathing and heart rate without contact, imaging objects through clothing or packaging, and building detailed three-dimensional maps of rooms. These applications require bandwidths of several gigahertz and wavelengths of millimetres, which places them in the millimetre-wave and sub-terahertz bands. They are where the terahertz research described in Chapters 2 and 3 meets sensing most directly, and they are also furthest from commercial deployment in cellular networks. The fourth group is sensing in service of communication itself. A network that knows where users and obstacles are can predict blockages, choose beams before the user moves, and hand over between access points proactively. For sub-terahertz links, where finding and keeping a narrow beam is the central difficulty, sensing may be less an added service than a prerequisite. This symbiosis is perhaps the strongest technical argument for integrating the two functions. The costs of integration Sensing and communication compete for resources. A signal optimised for communication spreads energy across the directions where users are and adapts its modulation to their channels. A signal optimised for sensing sweeps across directions where targets might be, uses a predictable structure and may need long observation times. Time, frequency, power and beam directions spent on one are not available for the other, at least not fully. Research has explored the trade-off formally, characterising the boundary of achievable combinations of communication rate and sensing accuracy, and the general finding is that substantial sensing can be obtained with modest cost to communication when the two functions share a signal intelligently, but that demanding sensing tasks require dedicated resources. Clutter is a second cost. Real environments are full of static reflectors: buildings, poles, parked cars, walls. Their echoes are much stronger than those of small moving targets such as drones or people. Separating targets from clutter relies on Doppler processing, background subtraction and tracking across time, all of which require computation and careful design. Multipath, where a target's echo reaches the receiver by several routes, can produce ghost targets. Calibration and synchronisation are a third cost. For sensing to deliver the accuracy its bandwidth allows, base station clocks and antenna arrays must be calibrated to fractions of a nanosecond and fractions of a degree. Mobile networks have historically needed timing accuracy of around a microsecond for time-division duplex operation. Sensing, like cell-free operation discussed in the next chapter, demands far tighter tolerances. Privacy, security and trust A network that can detect drones can, with different software and higher frequencies, detect people. That is the uncomfortable fact at the centre of ISAC, and it is better confronted early. Communication networks already reveal a great deal about their users through location data, but that data concerns people who carry devices and who have, at least nominally, agreed to terms of service. Sensing is different. It can detect and track people who carry no device and have agreed to nothing. Fine-grained sensing at high frequencies could reveal not only presence and movement but posture, gestures and vital signs. Even coarse sensing could reveal patterns of activity inside buildings. The governance questions are not technical, but technical design can help or hinder answers. Standards can require that sensing be performed only on explicit authorisation, that raw measurements stay within the network and that only processed, purpose-limited results be exposed, and that the granularity of exposed data be matched to the authorised purpose. The 6G work on sensing has included security and privacy studies, and the ITU's overarching emphasis on security, privacy and resilience applies. Data protection laws such as the European Union's General Data Protection Regulation will apply to any sensing that produces data relating to identifiable people, and regulators will need to decide how existing rules on surveillance, radar and spectrum use apply to networks that do both. Security has a mirror-image problem. A sensing network can be spoofed. An attacker could inject false echoes to create phantom drones or hide real ones, and a network whose beam management relies on sensing could be manipulated into poor decisions. Sensing results used for safety-critical purposes, such as traffic control or industrial safety, will need integrity protection and validation. A realistic trajectory Integrated sensing and communication is likely to arrive in stages. The first stage, already underway in 5G-Advanced, uses existing mid-band networks for coarse sensing of large targets in the open, with drone detection the leading example. The second, beginning with 6G in the upper mid-band, brings wider bandwidths, larger arrays and a native sensing service, enabling better resolution and multistatic operation across cooperating sites. The third, dependent on millimetre-wave and sub-terahertz deployment, brings fine-grained sensing in indoor and short-range settings. At every stage, the quality of sensing depends on the density and coordination of the infrastructure. A single base station sees targets from one angle and loses them behind obstacles. Many cooperating access points, observing from many angles with synchronised clocks, can triangulate, see around clutter and track targets continuously. That is exactly the architecture that the push to higher frequencies already demands for communication. The convergence is not accidental. A dense, coordinated, distributed antenna network is the best structure both for delivering high-frequency communication and for sensing the world, which is why the cell-free architecture examined next is central to the 6G story. Chapter 7: Beyond the Cell In December 1947 Douglas Ring, an engineer at Bell Laboratories, wrote an internal memorandum describing how a mobile telephone service could reuse the same frequencies in different areas by dividing a territory into small regions, each served by its own transmitter. That idea, the cell, became the organising principle of every mobile network built since. Its logic is elegant. Radio spectrum is limited, so reuse it as often as possible; signals weaken with distance, so a frequency used in one cell can be used again a few cells away; and as demand grows, split the cells into smaller ones to reuse spectrum more intensively. Analogue networks, GSM, LTE and 5G all follow the same template, with refinements in how cells share frequencies and hand users from one to the next. The cell has a built-in weakness, and it is geometric. Every cell has an edge. A user near the centre of a cell receives a strong signal from its own base station and weak interference from distant neighbours. A user near the edge receives a weaker signal from its own base station and stronger interference from the cell next door, which is transmitting on the same frequency to someone else. Users at the edge experience lower data rates, more dropped connections and more frequent handovers. In mobile networks, the gap between the best-served and worst-served users is large, and it is the worst-served who determine whether a service feels reliable. Cell-free massive MIMO is an attempt to remove the edge altogether. It replaces the picture of base stations each serving their own territory with a picture of many antennas, spread across an area, that cooperate to serve every user jointly. In a cell-free network, no user is at the edge of anything, because there are no cells. From coordination to cooperation The idea that base stations might cooperate rather than compete is not new. In the early 2000s, information theorists showed that if all the base stations in a network could share their users' data and channel knowledge, and transmit jointly as if they were a single giant antenna array with elements in many places, the capacity of the network would increase dramatically, because interference between cells would become useful signal. Work by Shlomo Shamai and Benjamin Zaidel in 2001, and by Karakayali, Foschini and Valenzuela at Bell Labs in 2006, established the principles under the names network MIMO and network coordination. The industry tried to implement a practical version of this idea in LTE under the name coordinated multipoint transmission and reception, or CoMP, standardised in Release 11 in 2012. The results were disappointing. Joint transmission across sites required the base stations to exchange large amounts of data and channel information over backhaul links that were not designed for it. Channel knowledge became outdated by the time it reached the coordinating site. Synchronisation between sites was not tight enough for coherent combining. And the frequency-division duplex systems that dominated LTE required devices to measure and report the channel from every cooperating site, which consumed uplink capacity. CoMP survived in limited forms, but the grand vision of network MIMO did not reach commercial networks. Two developments since then changed the calculation. The first was massive MIMO. In 2010 Thomas Marzetta, also at Bell Labs, showed that a base station with many more antennas than the users it serves gains a remarkable property: as the number of antennas grows, the effects of fast fading and uncorrelated noise average out, the channels of different users become nearly orthogonal, and simple linear processing achieves near-optimal performance. Marzetta's framework relied on time-division duplex operation, in which uplink and downlink share the same frequency, so the base station can learn the downlink channel from pilots sent in the uplink by exploiting the physical reciprocity of the radio channel. That removed the need for devices to report channels from every antenna. Massive MIMO became the defining technology of 5G mid-band. The second development was cheaper and denser infrastructure: centralised processing in data centres, high-capacity fibre and wireless fronthaul, and inexpensive radio units. Together, these made it plausible to distribute a massive MIMO array across a whole area rather than concentrating it on a single mast. The cell-free idea In 2017, Hien Quoc Ngo, Alexei Ashikhmin, Hong Yang, Erik Larsson and Thomas Marzetta published a paper in the IEEE Transactions on Wireless Communications titled "Cell-Free Massive MIMO Versus Small Cells." It described a network of many access points, each with one or a few antennas, scattered across an area and connected to a central processing unit. All access points served all users simultaneously on the same time and frequency resources. Each access point estimated its channels to users from uplink pilots and applied simple local processing; the central unit combined the results. The comparison in the paper was against a small-cell network with the same number of access points, in which each user was served by only its best access point. The cell-free system delivered a large improvement in the throughput that could be guaranteed to 95 percent of users, which is the measure that captures the experience of the worst-served. In the scenarios studied, the gain was several-fold, and larger when shadowing was correlated across nearby access points. The improvement came from two sources. Users who would have been at the edge of a small cell were now served by several nearby access points, so their signal improved. And the interference that would have come from neighbouring small cells was now part of a jointly designed transmission. The paper also highlighted a property that makes cell-free networks attractive beyond raw throughput: uniformity. In a cellular network, the quality of service varies sharply with location. In a cell-free network, every user tends to be close to several access points, so performance is more even across the area. For applications that need reliable service everywhere, such as industrial automation, extended reality or vehicle connectivity, uniformity is worth as much as peak speed. The scalability problem The original cell-free formulation had a serious flaw, which later work identified and fixed. If every access point serves every user, then the processing, the fronthaul traffic and the channel estimation all grow with the product of the number of access points and the number of users. In a city with thousands of access points and tens of thousands of users, that product becomes unmanageable. A network in which a new user anywhere increases the workload of every access point everywhere cannot scale. Emil Björnson and Luca Sanguinetti formalised the problem in a 2020 paper, defining a cell-free network as scalable only if the computational complexity and fronthaul load at each access point stay finite as the network grows without bound. Their solution, now widely adopted in research, is the user-centric approach. Each user is served by a cluster of nearby access points, chosen according to the strength of their channels. Each access point serves only a limited number of users. Clusters overlap, so neighbouring users share some access points but not others. There are still no cell boundaries, because each user has its own cluster centred on itself, but the burden on each access point is bounded. User-centric clusters capture most of the benefit of serving every user from every access point, because distant access points contribute little signal anyway. They make the architecture practical, and they connect it to an older idea, dynamic cooperation clusters, from the network MIMO literature. The research also clarified a spectrum of processing options. At one extreme, access points do everything locally, estimating their own channels and applying simple beamforming, and send only processed data to the central unit. This minimises fronthaul but cannot suppress interference as effectively. At the other extreme, access points forward raw received signals to the central unit, which performs joint estimation and processing across all access points serving a user. This achieves the best performance but requires high fronthaul capacity. Intermediate options pass partial information. Most practical designs will sit somewhere between, choosing the level of centralisation based on available fronthaul and the density of users. Table 5 compares the three architectural patterns discussed in this chapter. The comparison is qualitative and summarises research findings rather than measurements from deployed networks. Table 5. Three ways to organise a radio access network. Property Macro massive MIMO Small cells User-centric cell-free Antenna placement Concentrated on masts Distributed, independent Distributed, cooperating Cell edge Present, often severe Present, frequent Absent Interference handling Within-cell beamforming Frequency planning, scheduling Joint processing across access points Fronthaul demand Moderate per site Low per site High, depends on centralisation Coverage uniformity Low Moderate High Main obstacle Edge users Interference and handovers Synchronisation and fronthaul Sources: Ngo et al. (2017); Björnson and Sanguinetti (2020); Demir, Björnson and Sanguinetti (2021). Pilots, reciprocity and calibration Every multi-antenna system depends on channel knowledge, and cell-free systems are no exception. In the time-division duplex design inherited from massive MIMO, each user transmits a pilot sequence in the uplink, and access points estimate the channel from it. The number of mutually orthogonal pilot sequences is limited by the time available within each coherence interval, the period during which the channel stays roughly constant. If two users share a pilot, the access points cannot fully separate their channels, a problem known as pilot contamination. In a cellular massive MIMO system, pilot contamination arises between cells that reuse the same pilots. In a cell-free system, there are no cells, but the problem persists between users who share a pilot and are served by overlapping clusters. Pilot assignment becomes a graph-colouring problem: give the same pilot only to users who are far enough apart that their clusters do not overlap significantly. Research has produced a range of algorithms for this, and the user-centric structure makes them tractable. Downlink transmission relies on reciprocity: the uplink channel measured from pilots is assumed equal to the downlink channel. The physical propagation is reciprocal, but the transmit and receive electronics at each access point are not identical, so each access point must be calibrated to compensate for the difference. In a co-located massive MIMO array, calibration can be done internally. In a distributed system, access points separated by tens or hundreds of metres must be calibrated relative to each other, which requires over-the-air measurements between access points. Coherent joint transmission, in which several access points send the same symbol so that the signals add in phase at the user, imposes the strictest requirement. The carrier phases of the cooperating access points must be aligned to a small fraction of a cycle. At 3.5 gigahertz, a cycle lasts less than 0.3 nanoseconds; at 7 gigahertz, about 0.14 nanoseconds. Aligning separate radio units to that precision, and keeping them aligned as oscillators drift, is a demanding task. A less demanding alternative, non-coherent joint transmission, has access points send to the same user without phase alignment, which yields a power gain but forgoes the coherent combining gain. The uplink is easier, because the central unit can combine signals from several access points after the fact, compensating for their phase differences digitally. A subtler difference from co-located massive MIMO concerns channel hardening. In a large array on a single mast, all the antennas see the user through roughly the same environment, and as the number of antennas grows, the effective channel gain after beamforming becomes nearly constant. That property, called channel hardening, allows the user to decode downlink data without its own estimate of the channel, relying on the average gain instead. In a cell-free network, the user is served mostly by a handful of nearby access points, each with its own fading, and the combined channel fluctuates more. Research has shown that cell-free systems harden less, so devices benefit from downlink pilots or other mechanisms to estimate the effective channel. It is a modest cost, but it illustrates that distributing an array changes its statistics, not only its geometry. There is a corresponding benefit. Because the access points serving a user are spread out, it is unlikely that all of them are blocked or deeply faded at once. The diversity that reduces channel hardening also provides robustness. A user who walks behind a pillar loses one access point but not the others. For the reliability targets of industrial applications, that macro-diversity is worth more than the small overhead of extra pilots. Radio stripes and other physical forms How do you build a network of hundreds of cooperating access points without running a separate fibre to each one? One elegant answer, proposed by researchers at Ericsson and Linköping University and described by Giovanni Interdonato and colleagues in 2019, is the radio stripe. Antenna elements and their processing circuits are embedded in a cable or a strip, connected in series, sharing power, synchronisation and data along the same line. A stripe can be mounted along a wall, a ceiling, a lamp post or the side of a building. Processing is sequential: each unit on the stripe performs part of the computation and passes its results to the next, so the fronthaul requirement does not grow with the number of units. Radio stripes illustrate a broader point. Cell-free architecture is not a single product but a family of deployment patterns. In an office or factory, access points may be ceiling tiles connected by a shared bus. In a stadium, they may be strips along the stands. In a city, they may be small units on street furniture connected by fibre to a nearby edge data centre. The architecture adapts to the physical environment rather than imposing a uniform grid of masts. Where cell-free fits in 6G The first 6G specifications will not define a fully cell-free network in the academic sense. Release 21 will build on existing multi-transmission-point features from 5G, which allow a device to be served by more than one transmission point, and on the coherent joint transmission enhancements studied in recent 5G-Advanced releases. The 6G architecture work is designed to leave room for more distributed operation, and many companies have argued for cell-free principles in the 6G studies, but the transition will be gradual. It is likely to follow the pattern of deployment economics. Indoor and enterprise environments, where a single owner controls the space and can install dense access points connected by cable, are the natural first home for cell-free or near-cell-free systems, whether built on 6G, private networks or an evolution of Wi-Fi. Dense urban outdoor areas, where operators already deploy small cells, could adopt cooperative clusters around macro sites. Wide-area rural coverage will remain cellular for the foreseeable future, because the density of access points that cell-free operation requires is not economic where users are sparse. The link with higher frequencies is direct. At sub-terahertz frequencies, a single access point cannot reliably serve a mobile user, because hands, bodies and furniture block its narrow beam. A dense set of access points that serve each user from several directions at once, switching seamlessly as paths are blocked and unblocked, is the natural way to make such frequencies usable. The diversity that cell-free architecture provides at mid-band to improve uniformity becomes, at sub-terahertz frequencies, the precondition for coverage at all. The same is true for sensing, where multiple viewpoints overcome occlusion and improve accuracy. Cell-free architecture, then, is less a feature of 6G than a direction of travel for wireless networks in general. It asks a great deal of the infrastructure behind the antennas: fronthaul, synchronisation, processing and coordination. Whether those requirements can be met at reasonable cost is the question that determines how far and how fast the architecture spreads, and it is the subject of the next chapter. Chapter 8: Making Distributed Networks Work Architectural ideas in wireless communication tend to succeed or fail on unglamorous details. The theory of cell-free massive MIMO is mature, the simulations are compelling and the physical argument for distributed antennas at high frequencies is strong. What determines whether these ideas reach real networks is a set of practical questions. How do you move enormous volumes of radio data between hundreds of access points and a processing centre? How do you keep their clocks aligned to fractions of a nanosecond? How much energy does a dense network consume? Where does the computation happen, and how is it managed? And who pays for the fibre, the poles, the power and the ceiling space? This chapter takes those questions in turn. Fronthaul: the hidden network behind the antennas Every radio access network has two parts: the radio units that transmit and receive, and the processing that turns radio signals into data packets. The link between them is called fronthaul when the radio unit is separated from much of the processing, and backhaul when a complete base station connects to the core network. In a traditional macro site, the radio units sit on the mast and the processing sits in a cabinet at its base, connected by a few short fibres. In a distributed architecture, the radio units are scattered and the processing may be kilometres away in a data centre. The amount of data fronthaul must carry depends on where the processing is split. If the radio unit simply digitises the received waveform and sends raw samples to the processor, the data rate is roughly the sampling rate multiplied by the number of bits per sample, multiplied by the number of antenna streams. For a 400 megahertz channel, raw time-domain samples with 16-bit precision for each of the in-phase and quadrature components come to more than 10 gigabits per second for a single antenna stream. A radio unit with 64 streams would need roughly a terabit per second. That is the reason the original Common Public Radio Interface, designed for an era of narrow channels and few antennas, could not survive into the massive MIMO era unchanged. Modern designs move part of the processing into the radio unit. The split most widely used in open and virtualised radio networks, known as option 7.2x and specified by the O-RAN Alliance, places the Fourier transform and some beamforming functions in the radio unit, and sends frequency-domain samples for the subcarriers actually in use, with compression. That reduces fronthaul to a manageable level, especially when the number of spatial layers sent over fronthaul is smaller than the number of antenna elements. Cell-free architecture adds a twist. The performance advantage of a cell-free network grows with the degree of centralised processing, because joint processing across access points suppresses interference better than local processing. But centralised processing requires more fronthaul. The optimal design depends on the relative cost of fronthaul and processing. With dense fibre in a city centre, a high degree of centralisation is feasible. In a building with existing Ethernet cabling, a more local approach may be necessary. Research on cell-free systems has produced a family of processing schemes that trade fronthaul for performance along this spectrum, and practical deployments will need to choose among them. Fronthaul need not be fibre. Wireless fronthaul and backhaul, including the integrated access and backhaul feature introduced in 5G Release 16, allow a base station to use part of its own spectrum to connect onward. Sub-terahertz links, with their very high capacity over short distances, are natural candidates for wireless fronthaul between access points in places where fibre is unavailable or expensive. Commercial D-band radios already serve as short-hop backhaul in some cities. This is one of the earliest roles that sub-terahertz frequencies are likely to play in mobile networks: not connecting phones, but connecting the access points that serve them. Time and phase Mobile networks have always needed synchronisation. Time-division duplex networks, in which uplink and downlink alternate on the same frequency, require neighbouring base stations to switch between transmit and receive at the same moment, or their downlink transmissions will swamp each other's uplink receivers. The 3GPP requirement for time-division duplex cell phase synchronisation is a maximum deviation of 3 microseconds between cells, typically met by locking each base station to satellite navigation signals or to the IEEE 1588 Precision Time Protocol distributed over the transport network. Coherent cooperation demands far more. As described in the previous chapter, coherent joint transmission requires the carrier phases of cooperating access points to be aligned to a fraction of a cycle, which at 7 gigahertz is a fraction of 140 picoseconds. Multistatic sensing requires timing alignment of around a nanosecond or better, because every nanosecond of error becomes 30 centimetres of range error. These are three orders of magnitude tighter than conventional cellular synchronisation. Several approaches are being pursued. Precision timing over fibre has been demonstrated to sub-nanosecond accuracy by systems such as White Rabbit, originally developed at CERN to synchronise particle accelerator equipment and since adopted in telecommunications and finance. Distributing a common reference frequency over fibre to all access points, rather than letting each run on its own oscillator, removes most frequency drift. Over-the-air calibration, in which access points measure signals from each other and estimate their relative phase offsets, can correct residual errors. Radio stripes avoid much of the problem by carrying a shared clock along the stripe itself. The uplink, where combining happens in the processor, can tolerate larger offsets because they can be estimated and removed digitally. The difficulty grows with frequency. At sub-terahertz frequencies, a phase error of a tenth of a cycle corresponds to a timing error of a third of a picosecond at 300 gigahertz. Coherent combining across separate access points at those frequencies is a research problem, and early systems are likely to rely on selection or non-coherent combining across access points, choosing the best path rather than adding them in phase. Where the computation lives Distributed antennas push computation toward the centre. A cell-free network's processing unit must estimate channels for many users across many access points, compute beamforming weights, schedule transmissions and run the higher-layer protocols, all within the timing budgets of the radio interface. That work fits naturally into data centres located near the edge of the network, often called edge clouds, where general-purpose servers with hardware accelerators run virtualised radio software. The trend toward virtualised and open radio networks, promoted by the O-RAN Alliance and by operators seeking to reduce dependence on a few large vendors, supports this direction. Separating the radio unit, the distributed unit and the central unit, with standardised interfaces between them, allows different suppliers to provide different parts and allows processing to be pooled across many sites. Pooling has its own benefit: traffic peaks at different sites at different times, so a shared processing pool needs less total capacity than dedicated hardware at each site. The O-RAN Alliance has committed to aligning its 6G work with the 3GPP timetable. The tension is between flexibility and efficiency. General-purpose processors running radio software consume more power per unit of work than dedicated silicon. Accelerators for the most demanding functions, such as channel decoding and beamforming computation, narrow the gap. For very high-bandwidth systems, dedicated hardware in or near the radio unit will remain necessary for the lowest layers. Intelligence in the radio Artificial intelligence enters this picture in two ways. The first is as a tool for running the network. A cell-free network presents a large number of interacting decisions: which access points should serve each user, which pilot sequences to assign, how to schedule transmissions, when to put access points to sleep, how to predict and avoid blockages. Many of these decisions are combinatorial and must be made quickly, and they depend on traffic patterns and environmental features that change over time. Machine learning methods are well suited to such problems, and research has applied them to access point selection, pilot assignment, power control and beam prediction. The second is as part of the air interface itself. 3GPP studied artificial intelligence and machine learning for the radio interface in Release 18, focusing on compressing channel state information feedback, predicting the best beams and improving positioning, and moved beam management and positioning into normative work in Release 19. The 6G studies have taken this further, treating AI as a native element of the design. An important practical question is how models are trained, deployed, monitored and updated across devices and networks from different vendors, and how their behaviour is tested. A network component whose performance depends on a learned model behaves differently from one that follows a fixed algorithm, and standards must specify enough about its inputs, outputs and life cycle to make interoperability possible. There is reason for measured expectations. AI methods can improve the performance of well-defined tasks, such as beam prediction, by meaningful margins in tests. They also add complexity, data collection requirements and new failure modes. The most durable gains are likely to come in network operations, where learned models can optimise configurations across thousands of sites, rather than in replacing the core mathematics of signal processing. Surfaces that redirect waves Another idea associated with 6G is the reconfigurable intelligent surface: a panel covered with many small elements whose reflective properties can be adjusted electronically, so that the panel redirects incoming radio waves in a chosen direction. A surface on a wall could bounce a signal around a corner to a user who has no direct line of sight to an access point. Unlike an access point, such a surface needs no power amplifiers or radio chains, only control circuits, so it could in principle be cheap and energy-efficient. The European Telecommunications Standards Institute established an industry specification group on reconfigurable intelligent surfaces in 2021. The physics imposes limits. A passive surface reflects only what it receives, and the signal suffers path loss on the way to the surface and again on the way to the user. For the reflected path to be useful, the surface must be large relative to the wavelength and well placed. At sub-terahertz frequencies, where wavelengths are small, a surface of modest physical size contains many wavelengths and can provide useful gain, which makes the technology more attractive at high frequency. The control problem is also significant: configuring the surface requires knowledge of the channels to and from it, and the surface itself cannot measure them without additional circuitry. 5G Release 18 standardised a simpler relative, the network-controlled repeater, which amplifies and forwards signals under the control of the base station. Intelligent surfaces may follow in later releases if the practical benefits are demonstrated. In a dense cell-free network, they would serve as cheap supplements to access points rather than replacements. Energy and sustainability Energy consumption has become one of the defining constraints of mobile network design. The radio access network accounts for most of an operator's energy use, and energy costs have become a significant share of operating expenses. The IMT-2030 framework names sustainability as an overarching aspect, and both 5G-Advanced and 6G studies have treated energy saving as a priority. Distributed architectures cut both ways. On one hand, access points close to users need far less transmit power than a macro site to deliver the same signal strength, and a dense network can put idle access points to sleep while others cover their users. On the other hand, every access point has a baseline power consumption for its electronics, cooling and fronthaul, regardless of traffic. A network of a thousand small access points may consume more energy than a few macro sites if the baseline dominates. The key to energy-efficient dense networks is the ability to sleep deeply and wake quickly. 5G-Advanced introduced features that allow base stations to reduce transmissions of always-on signals and enter low-power states during low traffic. 6G is being designed with lean signalling from the outset, minimising the signals that must be transmitted even when no user is present. In a cell-free network, where many access points cover overlapping areas, coordinated sleep becomes particularly effective: at night, a small subset of access points can maintain coverage while the rest power down. Embodied energy, the energy used to manufacture equipment, is a further consideration. Dense networks require more units, more fibre and more installation. A full accounting of sustainability includes these, and it is one reason why the industry's interest in reusing existing sites for FR3 is as much environmental as economic. An illustrative deployment It helps to imagine how these pieces fit together in a single setting. Consider a large automated warehouse, a space where mobile robots move goods between shelving aisles and loading bays. Metal racks create deep shadows; robots must stay connected everywhere with predictable latency; and the operator wants to know where every robot and every person is at all times. A traditional design would place a few access points on the ceiling and accept coverage holes between the racks. A distributed design would instead run radio stripes along the top of each aisle, with access points every few metres, all connected to a processing unit in the building's server room. Every robot would be served by a user-centric cluster of the four or five nearest access points, so that blockage of one path by a passing forklift would be absorbed by the others. The same access points, operating with synchronised clocks, would form a multistatic sensing system that tracks robots and people from multiple angles, providing collision warnings without requiring every person to carry a tag. At night, when activity drops, most of the access points would sleep while a sparse subset maintained coverage. A few sub-terahertz links could provide high-capacity connections to fixed equipment such as inspection cameras, with their beams set up through the lower-frequency network. This is not a description of any particular installation; it is an illustration of how the technologies in this booklet combine. It also shows why enterprise settings are likely to lead adoption. The owner controls the space, the cabling and the processing, values reliability over peak speed, and can justify investment through productivity rather than through subscriptions. Who pays The final question is economic, and it may be the most decisive. Operators' returns on 5G investment have been widely described as disappointing. Revenue per user in many markets has been flat, while capital spending on spectrum and equipment was heavy. Operators are therefore cautious about 6G, and their priorities are clear in the standards process: reuse of existing sites, software upgrades where possible, a single migration path, and lower energy costs. Dense distributed networks require exactly what operators are reluctant to fund: many new sites, extensive fibre and complex installation. The likely resolution is a change in who builds the infrastructure. Several models are emerging. Neutral-host providers, including tower companies and specialist firms, build shared infrastructure in venues, campuses and city centres and lease capacity to several operators. Enterprises build private networks in their own factories, warehouses and hospitals, connecting to public networks where needed. Municipalities and utilities provide street furniture and fibre. Building owners install in-building systems as part of their property, much as they install electrical wiring. Cell-free architecture fits this world well, because it separates the access points, which can belong to a building or a neutral host, from the processing, which can serve several operators. A multi-operator cell-free network, in which shared access points cooperate to serve users from different operators, is technically conceivable, though it raises questions of spectrum sharing, security and commercial arrangements that have barely been explored. It also invites comparison with Wi-Fi, which already provides dense, low-cost indoor coverage built and owned by the people who use it. The boundary between cellular and Wi-Fi has been eroding for years, and dense 6G deployments indoors will compete directly with Wi-Fi 7 and its successors. The outcome will depend less on technical superiority than on cost, ease of deployment and the regulatory treatment of spectrum. The overall picture is of a network that is more distributed, more centralised in its processing, more dependent on precise timing and more varied in its ownership than any previous generation. None of these changes is inevitable on the 6G timetable, and each faces practical obstacles. But each follows from the physical and economic logic described in earlier chapters, and together they suggest that the most consequential change of the next decade will be in how wireless infrastructure is built, owned and operated, more than in any single number on a specification sheet. Conclusion The common way of talking about 6G puts frequency at the centre. A new generation, the story goes, conquers a new band, and the band determines what the generation can do. The evidence assembled in this booklet points to a different centre. Frequency matters, but what frequency demands matters more. Every step up the spectrum shortens links, narrows beams and makes obstacles more opaque, and the only sustainable response is a network that is denser, more coordinated and more aware of its surroundings. Sixth generation is best understood not as the generation of the terahertz band but as the generation in which the shape of the radio network begins to change. Several consequences follow from that view, and they cut against some widely held assumptions. The first 6G will look modest, and that is correct The first 6G networks, around 2030, will run mostly on existing 5G spectrum through multi-radio sharing and on new mid-band spectrum around 6 and 7 gigahertz. Their waveform will be the familiar OFDM, their codes inherited from 5G. Observers expecting a dramatic break will be disappointed. They should not be. The choices made in 3GPP's study phase reflect a correct reading of where the gains lie. In the bands that will carry most traffic for the next decade, performance comes from wider channels, larger arrays, better uplinks and lower energy use, not from exotic signal formats. A generation that delivers four times the mid-band capacity from existing sites, with lower energy per bit and a single clean migration path, would be a substantial success even if no user ever touches a terahertz link. The corollary is that the measure of early 6G should be capacity and coverage per unit of cost and energy, not peak data rates. Regulators and governments that set 6G targets in terms of headline speeds will repeat the mistake that made 5G millimetre wave look like a disappointment. The more useful targets concern the experience of the worst-served user and the cost of serving everyone. Spectrum policy is now the binding constraint For the next five years, the most important decisions affecting 6G will not be made in laboratories or standards meetings. They will be made at the World Radiocommunication Conference in late 2027 and in the national processes that follow it. The upper mid-band is crowded with satellites, military systems, scientific services and, in the upper 6 gigahertz band, with Wi-Fi. None of these incumbents can be moved quickly, and some cannot be moved at all. The practical implication is that sharing, not clearing, must become the normal mode of spectrum access for mobile networks. That requires investment in the tools of sharing: interference studies conducted openly, databases and coordination systems that work in real time, enforcement that incumbents trust, and network designs that use beam control and sensing to avoid harmful emissions. The architectural direction described in this booklet helps here. A network of many low-power access points with narrow, steerable beams creates less widespread interference than a network of high-power masts, and a network that can sense its environment can, in principle, detect and avoid incumbent receivers. Policymakers who want both strong 6G networks and protected incumbent services should favour the architectures that make sharing easier. Terahertz is a tool, not a layer The sub-terahertz bands will be used, and their capabilities are real. But the physics of absorption, blockage and power, together with the state of the hardware, place them in particular roles: short-range links of extraordinary capacity, wireless fronthaul and backhaul between access points, connections between fixed or slowly moving equipment, and fine-grained sensing. Treating terahertz as the eventual coverage layer of a national network misreads the physics. That reading has consequences for research and investment. The most valuable terahertz work in the coming decade is likely to be in packaging, power efficiency, beam management assisted by lower-frequency links and sensing, and integration with fibre-fed distributed infrastructure. The aim should be to make sub-terahertz radios cheap and reliable enough to be deployed by the thousand inside buildings and along streets, as components of a larger system, rather than to make a phone that depends on them. Distributed infrastructure changes who builds networks The strongest technical case in this booklet is for distributed, cooperating antennas. Cell-free architecture removes the cell edge, evens out service across an area, provides the path diversity that high frequencies need, and gives sensing multiple viewpoints. Its costs lie in fronthaul, synchronisation and processing, and above all in the number of access points it requires. Those costs will not be borne by mobile operators alone. The infrastructure of dense networks is more like a building's electrical wiring or a city's street lighting than like a network of towers. It will be installed by building owners, enterprises, neutral hosts and municipalities, and shared among service providers. That shift has implications well beyond engineering. It calls for standards that make shared access points and multi-operator operation practical, for planning rules that treat in-building wireless infrastructure as a utility, and for commercial models in which the owner of the ceiling and the operator of the service are different parties. Countries and cities that prepare for that model will get better 6G service sooner. Sensing needs rules before it needs scale Integrated sensing is the most novel capability of 6G and the one whose social consequences are least understood. Early uses, such as drone detection and traffic monitoring, are valuable and relatively uncontroversial. Later uses, particularly fine-grained sensing indoors at high frequencies, could detect and characterise people who carry no device and have consented to nothing. The time to set the rules is now, while the standards are being written and before networks are deployed. Sensing should be authorised for specific purposes, the data exposed should be limited to what the purpose requires, raw measurements should stay inside the network, and sensing outputs used for safety should be integrity-protected. These principles can be built into the architecture at low cost if they are designed in from the start. Retrofitting them later, after sensing services have commercial momentum, will be far harder. What to watch For readers who want to follow 6G over the next few years, a short list of indicators will reveal more than any marketing claim. The first is the outcome of 3GPP's migration decision and whether 6G keeps to a single path. The second is the result of WRC-27 on the 7 gigahertz range and the speed with which national regulators release spectrum after it. The third is the progress of Release 21 against its 2028 and 2029 freeze dates. The fourth is whether sensing services are launched with clear governance. The fifth, and most telling, is whether dense, shared, distributed infrastructure begins to appear in buildings and city centres, and who pays for it. The physics in this booklet does not change, and it points consistently in one direction: toward networks of many antennas, close to users, working together, and aware of the world through which their signals pass. How quickly the industry and its regulators follow that direction is a matter of choice. The choices being made now, in standards rooms and spectrum conferences, will determine whether the network of the 2030s is a faster version of today's or a different kind of infrastructure altogether. Hashtags: #6GNetworks #6GTechnology #NextGenerationWireless #TerahertzCommunication #TerahertzFrequencies #SubTerahertz #CellFreeArchitecture #CellFreeMassiveMIMO #MassiveMIMO #WirelessCommunication #FutureTelecommunications #IntegratedSensingAndCommunication #ISAC #SmartWirelessNetworks #Beamforming #DistributedAntennaSystems #RadioFrequencyEngineering #SpectrumManagement #FR3Spectrum #WirelessNetworkArchitecture #UltraHighSpeedConnectivity #Beyond5G #IMT2030 #3GPP #WirelessInnovation #RadioPropagation #NetworkOptimization #ArtificialIntelligenceIn6G #ReconfigurableIntelligentSurfaces #TelecommunicationsResearch

Download the Book (PDF): Introduction Walk into almost any large office building at six on a Sunday morning and listen. Somewhere above the ceiling tiles, fans are turning. In a basement plant room a chiller or boiler is warming up, following a schedule that someone typed into a controller years ago and nobody has opened since. Air is being heated in one duct and cooled in another, then mixed so that a room no one will enter today stays at 22 degrees. None of this is a malfunction in the ordinary sense. Every component is doing what it was told. The building is simply acting on what it was told years ago, not on what is happening inside it now. Commercial real estate has spent the last two decades acquiring digital tools that promise to fix this. Architects and engineers have moved from drawing lines on screens to building three-dimensional models in which a wall knows it is a wall and a duct knows what air it carries. Landlords have scattered sensors across floors to count who is present. Software companies now sell artificial intelligence that sits on top of a building's control system and adjusts setpoints every few minutes. Each of these is marketed separately, bought separately and usually run by different people. Together they make up much of what the industry calls PropTech. This booklet argues one thing. Commercial buildings waste energy mainly because they are operated without information: they run on assumptions fixed at design time about who will be inside, how the equipment behaves and what the controls are doing. Building Information Modeling, occupancy sensing and AI-driven HVAC control produce large and lasting savings only when they form a single, maintained chain of information that runs from the design model through construction and handover into the control loop itself, and back again through measurement. Any one of the three, bought on its own, tends to disappoint. Joined, they can change how a building behaves. Why the chain matters The argument rests on a simple observation about where the money and the carbon are. The International Energy Agency estimates that operating buildings accounts for roughly 30 percent of global final energy consumption and a little over a quarter of energy-related carbon dioxide emissions. In United States commercial buildings, the Energy Information Administration's 2018 Commercial Buildings Energy Consumption Survey found space heating to be the single largest end use, at about a third of consumption, with ventilation and cooling adding substantially more. Heating, cooling and moving air are the bulk of what a commercial building does with energy, and those are exactly the systems that depend most on knowing what is going on. Lighting is comparatively easy. A light fitting is on or off, bright or dim, and an occupancy sensor in the ceiling can make that decision locally with little harm if it gets it wrong. Heating, ventilation and air conditioning, collectively HVAC, is different. It is slow, coupled and invisible. A chilled-water plant serves dozens of air-handling units, each of which serves dozens of zones; a decision made at one level changes the conditions under which every other level operates. Comfort complaints arrive hours after the cause. Faults hide: a stuck damper or a leaking valve may waste energy for years while every room stays within its temperature band, because some other part of the system quietly compensates. This is why the energy problem in commercial buildings is, at bottom, an information problem. The engineering to heat and cool efficiently has existed for decades. What is usually missing is accurate, current knowledge of three things: what the building physically contains, who is using it and when, and how the systems are actually behaving. BIM addresses the first, occupancy sensing the second, and AI-driven control the third, with each relying on the other two. What the evidence actually shows The claims made for smart buildings are often large, and some of them are true. A field trial run by Pacific Northwest National Laboratory fitted advanced controls to 66 rooftop air-conditioning units in eight commercial buildings and recorded an average 57 percent reduction in their heating and cooling electricity. Google reported in 2016 that a DeepMind system cut the energy used for cooling in its data centres by 40 percent. A simulation study from Lawrence Berkeley National Laboratory estimated that ASHRAE's published high-performance control sequences would save an average of 31 percent of HVAC energy compared with typical practice. Set beside these are the portfolio numbers. When Lawrence Berkeley National Laboratory tracked nearly 6,000 buildings belonging to 96 organisations that had installed energy analytics, the median savings after two years were 9 percent for fault detection software and 4 percent for simpler energy information systems. Both figures are worth having, and both paid back within one or two years, but neither is drastic. The gap between those sets of numbers is the subject of this booklet. Large savings appear where the baseline was very poor and where the intervention reached the actual control logic with accurate information about the building and its occupants. Small savings appear where software watched a building it did not understand and produced alerts that nobody acted on. The difference is rarely the cleverness of the algorithm. It is whether the information chain was complete. How the argument proceeds The first chapter explains why buildings waste energy in the first place, drawing on the well-documented gap between how buildings are designed to perform and how they actually perform. The second describes the shift from two-dimensional drawings to parametric models: what changes when a building is described as a structured database of objects rather than a set of lines. The third examines interoperability, the long struggle to let different software and different firms share one model without losing meaning, and the standards that have emerged from that struggle. The fourth chapter is about handover, the moment when a building passes from those who made it to those who run it. It is where most of the information created during design and construction is lost, and it is where the connection between the model and the control system has to be made. The fifth turns to occupancy: the sensors that count people, how accurate they are, what they cost in privacy, and why an emptier post-pandemic office has made them more valuable. The sixth shows that much of what is sold as intelligent control is in fact good conventional control done properly, and that this is where the most reliable savings live. The seventh examines the genuinely new techniques, from model predictive control to reinforcement learning to cloud-based supervisory systems, and the conditions under which they work. The eighth deals with proof: how savings are measured, why they fade, and how regulation in New York and the European Union is turning measured performance into a legal obligation. The booklet is written for owners, asset managers, developers, facility managers, policy officials and anyone else who has to decide whether to pay for these technologies and how to tell whether they worked. It assumes no engineering background, but it does not simplify the mechanisms away. Where a number is given, it comes from a named source; where the evidence is thin, it says so. One further point of orientation. The technologies discussed here are often presented as a sequence, as if a building first gets a model, then sensors, then AI. In practice the sequence runs the other way for most of the existing stock. The vast majority of the commercial buildings that will be operating in 2050 already exist, and almost none of them were designed in BIM. For them, the question is how to assemble enough of the information chain, after the fact, to make intelligent operation possible. The answer turns out to depend less on how sophisticated the eventual control algorithm is than on how disciplined the organisation is about knowing what it owns. Chapter 1: The Blind Building Every commercial building is designed twice. The first design is the one the architect and engineers produce: a set of assumptions about how many people will occupy each floor, when they will arrive and leave, how much heat their computers will throw off, how much sun will come through the glass and how the mechanical systems will respond. The second design is the one that emerges over the following years as the building is fitted out, leased, re-leased, partitioned, patched and adjusted by people who have never seen the first. The building that consumes energy is the second one. The building that was modelled, specified and certified is the first. The distance between the two is known in the industry as the performance gap, and it is large. In 2016 Innovate UK, the British government's innovation agency, published a summary of its Building Performance Evaluation programme, which had studied a set of recently completed non-domestic buildings in detail. Their actual carbon emissions averaged around 3.8 times the design estimates. Only one building in the sample came in close to its prediction. Electricity use across the sample ranged from 28 to 367 kilowatt-hours per square metre per year, a thirteen-fold spread among buildings that were all new and all designed to modern standards. The investigators identified a recurring set of causes: building management systems that were poorly integrated, controls too complicated for the people operating them, and systems that, when confused, defaulted to running flat out. That last observation captures something important. A building that does not know what is happening inside it tends to fail in the expensive direction. When in doubt, the controls ventilate more, heat more, cool more and run longer, because the penalty for discomfort is an angry tenant and the penalty for waste is a utility bill that nobody in particular is responsible for. Where the energy goes To understand why information matters so much, it helps to follow the energy through a typical large office building with a central plant and variable air volume distribution, the most common arrangement in North American and many European commercial buildings. Outdoor air is drawn into an air-handling unit, mixed with air returning from the occupied spaces, filtered, and passed over a cooling coil fed with chilled water from a central chiller. In many systems the air is cooled to around 13 degrees Celsius regardless of season, because cold air carries more cooling per unit of fan energy and because cooling the air also dehumidifies it. That cold air is pushed by a supply fan through ductwork to terminal boxes above the ceilings of individual zones. Each terminal box has a damper that opens or closes to let in more or less air, and in perimeter zones and many interior ones it also has a reheat coil, fed with hot water from a boiler or with electricity, that warms the air if the zone is too cool. Four things in this arrangement drive most of the waste. The first is ventilation for people who are not there. Ventilation codes, most prominently ASHRAE Standard 62.1 in the United States and its equivalents elsewhere, set minimum outdoor air rates that combine an allowance per person with an allowance per unit floor area. The per-person allowance is calculated at design occupancy. If a floor designed for 200 people holds 60, the building still heats or cools outdoor air for 200 unless something tells it otherwise. In a cold climate, conditioning outdoor air is one of the largest single loads a building carries. The second is simultaneous heating and cooling. Because the central system delivers air that has already been cooled, any zone that needs less cooling than the minimum airflow delivers must reheat that air. The terminal box cannot close its damper fully, because it must still deliver enough air for ventilation and for the diffusers to mix the room properly. Minimum airflow settings are often set conservatively high during commissioning and never revisited, so a large fraction of interior zones may spend much of the year receiving cold air and immediately warming it back up. Every unit of energy spent on that cycle is spent twice. The third is operating hours. A building's schedule is usually a coarse rule: on at six in the morning on weekdays, off at seven in the evening, with a holiday calendar that may or may not be maintained. Tenants request after-hours operation and the overrides are never removed. One late-working team on one floor may keep a whole air-handling unit, serving ten floors, running until midnight. The fourth is faults. Sensors drift out of calibration. Dampers stick. Valves leak through. Economisers, the dampers that are supposed to use cool outdoor air for free cooling when conditions allow, are a notorious example: they fail in place, and the building simply buys mechanical cooling instead. Because HVAC systems are designed with margin, one component's failure is usually absorbed by another working harder. Occupants notice nothing. The energy meter records the cost, but nobody reads the meter at the level of detail needed to see it. None of these four problems requires new physics to solve. Each requires the building to know something it currently does not: how many people are in a zone, what the terminal box minimum actually needs to be, whether anyone is still present at eight in the evening, and whether the economiser damper is actually where the controller thinks it is. Design assumptions as fixed beliefs The deeper reason buildings behave this way lies in how they are delivered. A commercial building is designed by one set of firms, built by another, often fitted out by a third, and operated by a fourth. Each hands a package of information to the next, and each package is translated, abridged and partly lost in the handing. Consider what an HVAC engineer actually decides. To size the equipment, the engineer calculates peak loads: the hottest afternoon, the coldest morning, the full design occupancy. Equipment is selected to meet those peaks with a safety margin. The control sequences, the written logic describing how the equipment should operate, are then specified, usually in a few pages of prose in the mechanical specification. Those sequences are interpreted by a controls contractor who programs them into a building automation system, often by adapting a standard library of logic the contractor has used on previous jobs. The result is commissioned, meaning checked to see whether it works, during a compressed period at the end of construction when the building is empty, the schedule is behind and the main goal is to obtain an occupancy certificate. At each step, assumptions harden into fixed settings. A design occupancy of one person per ten square metres becomes a minimum outdoor airflow programmed into a controller. A guess about afternoon solar gain becomes a reheat valve limit. A peak-day calculation becomes a supply air temperature that never resets. None of these settings is labelled as an assumption. In the controller they are simply numbers, and the operating staff who inherit the building have no record of why any of them was chosen. This is what is meant here by a blind building. It is not a building without sensors; most modern commercial buildings have hundreds or thousands of measurement points. It is a building whose operating logic is fixed to a picture of reality that was never accurate and becomes less accurate every year. The sensors that exist mostly serve local loops, each keeping one temperature or one pressure at its setpoint. They are not used to ask whether the setpoint itself still makes sense. The cost of not knowing The clearest evidence that information alone has value comes from commissioning research. Commissioning, in its fuller sense, is the systematic process of verifying that a building's systems are designed, installed and operated to meet the owner's requirements. Retro-commissioning applies the same process to existing buildings, usually by investigating how the systems are actually running and correcting what has drifted. In 2009 Evan Mills and colleagues at Lawrence Berkeley National Laboratory assembled the largest dataset then available on commissioning outcomes: 643 buildings covering about 99 million square feet in 26 American states. For existing buildings, the median whole-building energy saving was 16 percent, at a median cost of 30 cents per square foot, giving a median simple payback of 1.1 years. For new construction, where commissioning was integrated into delivery, median savings were 13 percent with a payback of 4.2 years. Comprehensive commissioning produced nearly twice the savings of limited, single-system efforts. Those numbers deserve to be read carefully, because they are the backbone of the argument in this booklet. Retro-commissioning does not install new equipment. It does not change the building envelope. It sends skilled people into a building to find out what is actually happening and correct the settings and faults they discover. A median saving of 16 percent from that activity alone is a measure of how much energy commercial buildings waste through ignorance of their own condition. The research also found that savings generally persisted for at least several years, but other studies have documented the familiar pattern of gradual decay: settings drift back, overrides accumulate, new faults appear, and the building slowly returns toward its blind state. Commissioning is a snapshot. The building keeps changing after the commissioning agent leaves. The economic promise of digital technology in commercial buildings is, in its most defensible form, the promise to make commissioning continuous: to build into the building a permanent capacity to know its own physical configuration, its own occupancy and its own behaviour, and to adjust accordingly. BIM supplies the physical configuration. Occupancy sensing supplies the human use. Analytics and AI control supply the behaviour and the adjustment. Seen this way, none of the three is a novelty. Each is an attempt to automate part of what a very good commissioning engineer does by hand. If the waste is this large, it is fair to ask why the people who run buildings tolerate it. The answer lies less in negligence than in how responsibility and feedback are arranged. Start with metering. Most commercial buildings have one or a few utility meters measuring total electricity and total gas or district heat. The bill arrives monthly, aggregated across every system and every tenant. A chiller running all weekend, a set of terminal boxes reheating air all winter, or a stuck economiser damper each adds a few percent to a bill that already varies by more than that with the weather. Without submetering at the level of individual systems, and without a model of what consumption should be given the weather and occupancy, there is no way to see the waste in the bill. It is lost in the noise. Next, consider how operators are judged. A facilities team is measured by the absence of complaints and the absence of failures. A tenant who is too warm calls; a tenant whose zone is being simultaneously heated and cooled at great expense but stays comfortable does not. Every incentive in the operator's day pushes toward settings that prevent complaints, which usually means more airflow, longer hours and tighter temperature bands. Energy waste is silent; discomfort is loud. Maintenance contracts reinforce the pattern. Many are organised around planned preventive tasks, such as changing filters, greasing bearings and inspecting belts, and around reactive tickets raised by complaints or alarms. Neither category includes "find out whether the control sequences still make sense." Retro-commissioning, which is the activity that would catch the problems described above, is usually a separate project bought every few years, if at all. Finally, the building automation system itself is rarely designed to reveal waste. Its graphics typically show each piece of equipment on its own screen, with current values and alarms. An operator can see that an air handler is running and that its supply temperature is at setpoint. It is much harder to see, from those screens, that forty terminal boxes on that air handler have their reheat valves open at the same time as the chilled-water valve is open, because the information is scattered across forty other screens. The system displays its parts. It does not explain itself. The result is a stable equilibrium of waste. Nobody is acting irrationally, and each party is behaving sensibly within its own view. What is missing is a view of the whole: a way to see the building as a connected system, compare what it is doing with what it should be doing, and attribute the difference to specific causes. That is an information problem, and it is the one that the technologies in this booklet, properly connected, can address. Why the problem has become urgent For much of the twentieth century there was little pressure to close the performance gap. Energy was cheap relative to rents and salaries, building owners passed utility costs through to tenants, and nobody outside the building measured what it consumed. Three changes have altered that. The first is regulation that targets measured operation rather than design intent. New York City's Local Law 97, whose first compliance period began in January 2024, caps the greenhouse gas emissions of most buildings over 25,000 square feet and charges owners $268 for every tonne of carbon dioxide equivalent above the limit. The caps tighten sharply in 2030. The European Union's recast Energy Performance of Buildings Directive, adopted in 2024, requires member states to set minimum energy performance standards for the worst-performing non-residential buildings and extends mandatory building automation and control systems to smaller buildings than before. Both regimes judge buildings by what they actually emit or consume, which means the performance gap is no longer an embarrassment but a liability. The second is the change in how offices are used. Since 2020, many offices have been substantially emptier than before, and their occupancy has become more variable from day to day. Kastle Systems, which tracks access-card activity across a large sample of American office buildings, reported in December 2025 that average weekly occupancy had reached about 56 percent of its pre-pandemic baseline, a post-pandemic high, with peaks on Tuesdays and troughs on Fridays. A building programmed to ventilate and condition every floor at design occupancy from Monday to Friday is now wrong most of the time, and wrong in a way that wastes energy. The third is electrification and the grid. As heating moves from gas boilers to heat pumps, and as grids absorb more variable wind and solar generation, the timing of a building's electricity demand starts to matter as much as its total. A building that can shift its cooling or heating by an hour or two, pre-cooling before a peak or coasting through one, has value to the grid that a building on a fixed schedule does not. Doing that safely requires knowing how the building's thermal mass will respond, who will be inside, and what the equipment can deliver. It is impossible without information. If buildings waste energy because they are blind, then the measure of any digital technology is how much sight it restores and whether that sight reaches the controls. This is a demanding test. A beautiful BIM model that ends up as a file on a project server after handover restores nothing. An occupancy dashboard that facilities staff look at once a month changes nothing in the plant room. An AI system that recommends setpoint changes to an operator who lacks the time or authority to make them saves nothing. The rest of this booklet follows the information chain from its start in the design model to its end in a measured, verified reduction in energy use. Each chapter examines one link: what it is, what it can do, where it usually breaks and what it takes to keep it intact. The test throughout is the same. Does this technology help the building know what it is, who is in it and what it is doing, and does that knowledge change what the equipment actually does? Chapter 2: From Drawings to Models For most of the history of architecture, a building existed before construction only as a set of drawings. A plan showed the building cut horizontally and viewed from above; a section showed it cut vertically; an elevation showed one face. A detail showed a corner or a junction at large scale. A schedule listed the doors, the windows or the finishes. Each drawing was a separate artefact, and the building itself existed only in the heads of the people who could read them together and assemble a three-dimensional object out of two-dimensional evidence. That arrangement had strengths. Drawings are compact, legally well understood and readable by every trade. But it had one structural weakness that no amount of skill could overcome: the drawings were not connected to one another. If an architect moved a wall on the plan, nothing forced the section, the elevation, the door schedule and the structural engineer's framing plan to change. Coordination depended on people noticing. On a large project, with thousands of drawings produced by dozens of firms, people did not always notice, and the discrepancies surfaced on site as clashes, rework and claims. Computer-aided drafting, which spread through the profession in the 1980s and 1990s, made drawing faster but did not change this weakness. A line in an early CAD file was still just a line, with a start point, an end point, a layer and perhaps a colour. The software did not know that two parallel lines represented a wall, still less that the wall had a fire rating or a thermal resistance, or that a door was hung in it. CAD digitised the drawing board. It did not digitise the building. What a model knows Building Information Modeling, in the sense that matters here, replaces the set of drawings with a single structured description of the building from which drawings can be generated. The core unit is not a line but an object. A wall object has a type, a location, a height, a thickness, a set of material layers, and relationships: it is hosted on a floor level, bounds certain rooms, and contains certain doors and windows. A duct object has a cross-section, a length, an insulation specification, a system to which it belongs, and connections to the fittings at each end. A room or space object has a name, a floor area, a volume, an intended use and, in a well-developed model, a design occupancy and design temperature range. From this description, a plan is a horizontal slice through the model at a chosen height, rendered according to drawing conventions. A section is a vertical slice. A door schedule is a query listing every door object and its properties. Because all these views are generated from one underlying description, changing the model changes them all. Move the wall, and the plan, the section, the elevation, the room areas and the door schedule update together. The idea is older than the software that made it practical. In 1975 Charles Eastman, then at Carnegie Mellon University, published an article in the AIA Journal titled "The Use of Computers Instead of Drawings in Building Design," describing a prototype called the Building Description System. Its premise was exactly the one above: describe the building once, as a database of elements with attributes and relationships, and derive drawings from it. Eastman pointed out that drawings were redundant descriptions of the same object and that keeping redundant descriptions consistent was a large and avoidable source of cost. It took roughly a quarter of a century for commercial software and desktop computing power to catch up with that argument. Graphisoft's ArchiCAD, launched in the 1980s under the label of a "virtual building," was an early commercial example. Revit, developed by a start-up founded in 1997 and acquired by Autodesk in 2002, made the object-based approach dominant in North American and British practice over the following decade. What parametric means The word most often attached to BIM is parametric, and it is worth being precise about it, because it is the feature that makes a model behave differently from a drawing. A parametric object is defined by rules rather than by fixed geometry. A window family in a modelling tool is not a drawing of one particular window but a template with parameters: width, height, sill height, frame depth, glazing type. Change the width parameter and the geometry regenerates. The window also carries rules about its relationship to its host: it must sit within a wall, it cuts an opening of its own size, it moves when the wall moves. Parameters can also be linked to one another and to the wider model. A floor-to-floor height can be defined once at the level of the building, so that changing it moves every floor, stair, riser and façade panel above. A structural column can be constrained to a grid line, so that moving the grid moves the column. A duct's size can be calculated from the airflow it carries and the design velocity, so that changing the room's cooling load propagates through the airflow calculation to the duct dimension and then to the ceiling void depth required. This is where the model becomes useful for energy. Because rooms, walls, windows and systems are objects with properties, the model contains, in principle, much of the input that an energy simulation needs: the geometry of each thermal zone, the construction of each surface, the orientation and area of glazing, the intended use and occupancy of each space, and the systems serving each zone. In the drawing-based world, an energy modeller had to rebuild all of this by hand from the drawings, a process that took weeks and was often done once, late in design, as a compliance exercise. In a well-structured BIM workflow, a simplified analytical model can be extracted from the design model early and repeatedly, allowing the design team to test alternatives such as a different glazing ratio, a deeper overhang or a change in orientation while those decisions are still cheap to change. The phrase "in principle" matters. Architects model for drawings and coordination, not for simulation. Spaces may be missing or drawn inconsistently, walls may overlap, and thermal properties may not be filled in. Converting a design model into a clean analytical model still requires care, and many energy modellers continue to rebuild geometry in dedicated tools because they trust it more. The capability is real but has to be planned for from the start, which is a recurring theme in this booklet: information is only useful downstream if someone upstream decides to create it properly. Coordination, clashes and the end of the light table The first commercial benefit of BIM, and still the one that sells it to contractors, is spatial coordination. On a drawing-based project, the architectural, structural, mechanical, electrical and plumbing drawings were overlaid, sometimes literally on a light table, by coordinators trying to find places where a beam ran through a duct or a pipe passed through a column. On a model-based project, the separate discipline models can be federated into a combined model and checked automatically for geometric intersections. Software reports every place where two objects occupy the same space or come closer than a defined clearance. Clash detection sounds trivial and is not. A large hospital or laboratory may have tens of thousands of potential clashes in early coordination, most of them minor but some capable of stopping work for days if discovered on site. Resolving them in the model, in coordination meetings before fabrication, has changed how mechanical and electrical contractors work. Many now fabricate ductwork and pipework off site directly from the coordinated model, confident that it will fit. For the purposes of this booklet, the key consequence is less obvious. Clash coordination forces the mechanical systems to be modelled accurately, in three dimensions, with real sizes and real routes. The duct that appears in the coordinated model is very nearly the duct that gets installed. That model therefore contains a detailed record of the HVAC distribution system: which air-handling unit serves which zones, which terminal boxes sit where, which valves control which coils. This is exactly the information that an operator, an analytics platform or an AI controller later needs and usually cannot find. Because a model grows over the course of a project, the industry has developed ways to say how much any part of it can be relied on. In North America, the BIMForum Level of Development specification describes stages from a conceptual placeholder through generic and specific representations to an as-installed record. An air-handling unit might begin as a box of approximate size, become a box with the correct capacity and connection points, become a specific manufacturer's model with exact dimensions, and finally be verified against what was actually installed. The specification exists because users of a model need to know whether a number in it is a guess or a fact. The industry also speaks of additional dimensions. Linking model objects to the construction schedule, so that the sequence of work can be visualised and checked, is often called 4D. Linking them to cost data, so that quantities and estimates update as the design changes, is called 5D. Some practitioners extend the numbering to sustainability and facility management. The labels are marketing as much as engineering, but the underlying idea is sound: once a building is described as a database of objects, any other information that can be attached to those objects becomes available for analysis. For operation, the dimension that matters most is not a number at all but identity. If each piece of equipment in the model carries a unique, stable identifier, then that identifier can be carried through procurement, installation, commissioning and into the maintenance management system and the building automation system. A chiller then means the same thing to the designer who specified it, the contractor who installed it, the commissioning agent who tested it, the maintenance software that schedules its service and the controller that runs it. Without stable identity, each of those systems maintains its own description of the chiller, and the descriptions drift apart. Existing buildings and the limits of models Most of what has been said so far applies to new construction. But new buildings add only a small fraction to the commercial stock each year, and most of the buildings that need their energy use reduced in the next two decades were drawn on paper or in early CAD. For those buildings, the question is whether a model can be created after the fact, and whether it is worth doing. The technology for capturing existing buildings has improved enormously. Terrestrial laser scanners and, increasingly, mobile and handheld scanners record millions of points per second, producing a point cloud: a dense set of measured positions on every visible surface. Photogrammetry derives similar information from overlapping photographs. Converting a point cloud into an object model, a process usually called scan-to-BIM, remains partly manual. Software can recognise planar surfaces and suggest walls, floors and columns, and is getting better at recognising pipes and ducts, but identifying equipment, linking it to systems and recording what is hidden above ceilings or inside walls still needs a person. For energy purposes, a full geometric model of an existing building is often unnecessary. What an analytics or control system needs is a structured description of the HVAC system: what equipment exists, what it serves, how it is connected and which sensors and actuators belong to which piece of equipment. That description can be built from the building automation system's own point list, the mechanical drawings, a site survey and the maintenance records, without modelling every wall in three dimensions. Chapter 4 examines how that lighter-weight, system-centred model is built and why it is becoming the real bridge between BIM and smart HVAC. Between the full geometric model and the point list lies a range of choices, and choosing well is a matter of asking what decisions the information will support. A portfolio owner planning deep retrofits across fifty buildings may want enough geometry in each to run an energy simulation of envelope and plant upgrades. An owner whose immediate aim is better control may need only the systems and spaces. An owner planning to re-lease and refit floors may want accurate floor plates and ceiling voids. Each purpose implies a different scope of capture, and the most common waste in existing-building BIM projects is to commission a detailed geometric model for its own sake and then find that it lacks the system relationships that the operations team actually needed. Enthusiasm for BIM sometimes obscures its limits, and an owner deciding what to ask for needs to understand them. A model is not automatically accurate. It is a set of claims made by the people who built it, at a particular stage, for a particular purpose. An architect's model at the end of design development may show ductwork as generic placeholders, sized roughly and routed by eye. A mechanical subcontractor's fabrication model may be precise for the ductwork and approximate for everything around it. Treating any model as a reliable description of the building without knowing its level of development, its author and its date is as risky as treating an outdated drawing as the truth. A model does not settle legal responsibility. In many jurisdictions and on many projects, the contract documents remain the drawings and specifications, not the model, and the model is provided for information only. That arrangement protects designers from liability for errors in data they did not intend others to rely on, but it also means that the information most useful to operators carries no warranty. Some public clients and large owners have begun to make the model a contractual deliverable with defined content and acceptance criteria. Where they have not, the operator inherits a model whose reliability nobody has promised. A model costs money to create and more to maintain. Smaller design firms and contractors often model only what their own work requires, and they price additional information requirements accordingly. For a modest building, a full model carrying every property an operator might want can cost more than the operator is willing to pay. The right response is not to model everything but to decide in advance which information will actually be used, and to model that well. Most importantly, a model describes what the building is, not what it does. Even a perfect record model contains no information about how the building is being operated, how many people are inside or how the equipment is performing. It is the static half of the description the building needs. The dynamic half comes from sensors and controls, and the value of the model depends on whether it is connected to them. The move from drawings to models is sometimes described as a change in representation, as if the industry had simply upgraded its pictures. It is more fundamental than that. A drawing is a message from one person to another about a building. A model is a shared, queryable record of the building itself. The first can only be read. The second can be computed on. That difference is what makes the rest of the information chain possible. An energy simulation can only test design options quickly if the geometry is computable. An analytics platform can only diagnose a fault in a terminal box if it knows which air handler feeds that box and which zone it serves. An AI controller can only reason about how a change in supply air temperature will affect each zone if it knows the system topology. None of these requires a perfect three-dimensional model. All of them require the building to be described as connected objects with stable identities and meaningful properties. The irony of the BIM era is that the industry learned to build such descriptions extremely well for the purpose of designing and constructing buildings and then, at the moment the building became occupied and began consuming energy, largely threw them away. Understanding why that happens requires a look at how models are shared between firms and between software, which is the subject of the next chapter. Chapter 3: The Shared Model A building is not designed by one firm using one piece of software. Even a modest office block involves an architect, a structural engineer, a mechanical and electrical engineer, often a façade consultant, a fire engineer, an energy modeller, a cost consultant, a general contractor and a long chain of specialist subcontractors and suppliers. Each uses tools suited to its discipline. The structural engineer runs analysis software that thinks in members, loads and connections. The mechanical engineer uses tools that think in airflows, pressure drops and heat loads. The fabricator of the ductwork uses software that thinks in sheet-metal gauges and seam types. Each tool has its own internal description of the building, optimised for its own purpose. The promise of BIM depends on these descriptions being able to talk to one another without losing meaning. A model in which the architect's room knows its occupancy but the mechanical engineer's zone does not, or in which the air handler's identity changes between the design model and the contractor's fabrication model, delivers little more than a well-coordinated set of drawings. Interoperability, the ability of different systems to exchange information and use it, is therefore not a technical side issue. It decides whether the information chain holds. The price of not sharing The clearest estimate of what poor interoperability costs remains a 2004 study commissioned by the United States National Institute of Standards and Technology and carried out by RTI International, led by Michael Gallaher with Alan O'Connor, John Dettbarn and Linda Gilday. Using 2002 data, it estimated that inadequate interoperability cost the American capital facilities industry about 15.8 billion dollars a year. The study counted the costs of manually re-entering data, checking and reconciling inconsistent information, maintaining redundant systems, waiting for information that existed elsewhere and searching for documents. Its authors considered the figure a lower bound. The striking finding was not the total but where it fell. Roughly two-thirds of the cost was borne by owners and operators rather than by designers or builders, and most of it was incurred during the operations and maintenance phase, after construction was complete. Facility staff spent time searching for equipment information, verifying what was actually installed, recreating as-built records and dealing with the consequences of not knowing what they owned. The people who created the information during design and construction were not the people who paid for its loss. That asymmetry explains much of the history that followed. Designers and contractors adopted BIM because it helped them coordinate and build. Owners, who stood to gain most from good information during operation, were slower to understand what to ask for, and the industry's contracts rarely required information to be delivered in a form operators could use. The McKinsey Global Institute's 2017 report on construction productivity, which found that labour productivity in construction had grown by only about 1 percent a year globally over the previous two decades, compared with 2.8 percent for the world economy as a whole, identified fragmented information and poor digital adoption as among the reasons. The problem was not a lack of software. It was a lack of agreement on how information should flow between parties. Open standards for models and process The industry's main response has been to develop open, vendor-neutral standards for describing buildings and the processes for exchanging information about them. The most important are summarised in Table 1, which also shows that they cover different stages of a building's life and different kinds of information. Table 1. Principal open standards for building and systems information. Standard Steward What it describes Main stage of use IFC (ISO 16739-1) buildingSMART International Building objects, properties and relationships Design, construction, handover ISO 19650 series ISO, from UK BS 1192 and PAS 1192 How information is specified, produced and exchanged Whole life cycle COBie Originated with US Army Corps of Engineers Spaces, equipment, warranties, maintenance data in tabular form Handover to operations Brick Schema Open consortium, academic origin Equipment, sensors, control points and their relationships Operations and analytics Project Haystack Open community Tags describing control points and equipment Operations and analytics ASHRAE 223P ASHRAE (in development) Semantic model of building automation and control systems Design through operations The Industry Foundation Classes, known as IFC, form the central standard for exchanging building models between software. IFC began in the mid-1990s with an industry consortium that later became buildingSMART International, and it is now an international standard: IFC4 was published as ISO 16739 in 2013, and IFC 4.3, which extends the schema to infrastructure such as roads, railways and bridges, was published as ISO 16739-1 in 2024. IFC defines a large vocabulary of object types, such as walls, slabs, spaces, ducts, pumps, chillers and sensors, together with the properties they can carry and the relationships they can have: containment, connection, assignment to systems, assignment to zones. In practice IFC exchange works well for geometry and for common properties, and less well for complex parametric behaviour and discipline-specific detail. A model exported to IFC from one tool and imported to another usually arrives as a faithful but static description: the objects and their properties are there, but the rules that made them parametric in the native tool are not. For the purposes of operation, this is often acceptable. An operator does not need the parametric rules that generated the ductwork; it needs to know what was installed and how it is connected. But it means IFC is better understood as a format for handing over a snapshot than as a live link between design tools. The more serious limitation is that IFC files are only as good as the information put into them. IFC can represent which terminal boxes are served by which air handler, but only if the mechanical modeller assigned them to a system in the native tool and the export was configured to carry that assignment. On many projects these relationships are never modelled, because nothing in the design or construction process needed them. The standard can carry the information; the process does not always create it. This is why the more consequential development of the last decade concerns process rather than file format. The ISO 19650 series, whose first parts were published in December 2018, grew out of British standards and specifications developed for the United Kingdom government's BIM programme. From April 2016 the UK required what it called BIM Level 2 on centrally procured public projects, and it needed a framework for defining what information a client should ask for and how suppliers should deliver it. ISO 19650 sets out that framework. It distinguishes the organisation's overall information needs, the information required for a particular asset, and the information required for a particular project, and requires each to be specified in advance. It defines a common data environment, a managed repository through which information passes from work in progress to shared, published and archived states, with defined checks at each transition. It assigns responsibility for producing each piece of information and sets out how the client accepts it. The significance for energy is subtle but real. Under ISO 19650, an owner who wants the delivered model to support analytics and control must say so in its asset information requirements before design starts: which equipment must be modelled, which properties each item must carry, which relationships must be recorded, and in what format the information must be delivered at handover. If the owner says nothing, the process will faithfully deliver whatever the design and construction teams needed for their own purposes, which rarely includes the system topology an HVAC controller depends on. The standard does not make anyone ask the right questions, but it gives them a place to ask and a mechanism to enforce the answers. A further step in the same direction is buildingSMART's Information Delivery Specification, a machine-readable way of stating which objects a model must contain and which properties each must carry. Instead of an information requirement written in prose and checked by someone reading through a model, an owner can publish a specification that software checks automatically: every air-handling unit must have a manufacturer, a model number, a design airflow and an assignment to a system; every space must have a use and a design occupancy. The difference is the one between a contract clause that is hoped to be met and one that is tested. For an owner who wants operational information to survive handover, automated checking at each project milestone is the most practical way to make sure it is actually being created, rather than discovering its absence after practical completion. The operational vocabularies COBie, the Construction Operations Building information exchange, is the most widely known attempt to get design and construction information into a form that operators can use. Developed around 2007 by Bill East and colleagues at the United States Army Corps of Engineers' research and development centre, it organises handover information into a set of related spreadsheets listing facilities, floors, spaces, zones, equipment types, individual components, systems, spares, warranties, maintenance jobs and documents. Its design is deliberately unglamorous. Spreadsheets can be opened by anyone and imported into most computerised maintenance management systems, and that accessibility was the point. COBie captures equipment, but it is designed around maintenance rather than control. It records that an air-handling unit exists, where it is, who made it, when its warranty expires and what service it needs. It does not easily record which temperature sensor in the building automation system reads the unit's supply air, which output command drives its cooling valve, or which terminal boxes it feeds in a form an analytics engine can follow. Maintenance asks what the asset is and how to look after it. Control asks how the asset is connected and how it behaves. The vocabularies that address the control side emerged from a different community. The building automation industry has long had a communication standard, BACnet, published by ASHRAE as Standard 135 and first issued in 1995, which allows controllers from different manufacturers to exchange values. But BACnet says how to read a value, not what the value means. A typical large building may expose tens of thousands of BACnet points, each with a name chosen by whichever technician programmed it: something like "AHU3_SAT" or "FL12-VAV-07-DA-T." An experienced engineer can often guess what these mean. Software cannot, and every building uses different conventions. Project Haystack and Brick Schema both tackle this problem by giving points and equipment machine-readable meaning. Haystack, an open community effort, attaches standard tags to each point, such as "discharge," "air," "temp" and "sensor," so that software can find all discharge air temperature sensors without parsing the naming convention. Brick, first described in a 2016 paper at the ACM BuildSys conference by Bharathan Balaji and colleagues from several universities and IBM, goes further. It defines a formal ontology of equipment, points and locations, and the relationships between them: this sensor is a point of this air handler, which feeds these terminal boxes, which serve these rooms. Relationships matter because they let software reason. An application written once, say to detect simultaneous heating and cooling, can query any Brick-described building for every air handler, the terminal boxes it feeds and the heating valves in those boxes, and run its logic without being rewritten for each site. ASHRAE's proposed Standard 223P aims to bring this approach into the formal standards process, defining a semantic model for building automation and control data that can be created at design time and carried through to operation, with ties to both Brick and IFC. Whether it becomes widely adopted remains to be seen, but its direction is clear: the description of how systems are connected should be created once, by those who design them, and delivered as a machine-readable asset rather than reconstructed later by an analytics vendor at the owner's expense. It helps to follow a single piece of equipment through these vocabularies, because the differences between them become concrete. Take an air-handling unit on the roof of a ten-storey office building, serving floors one to five. In the mechanical engineer's design model, it is an object of a particular family, with parameters for design airflow, cooling and heating capacity, external static pressure and electrical supply. It is connected by modelled ductwork to forty terminal boxes on those five floors, and it belongs to a system named something like "Supply Air System 1." If the model has been built carefully, each terminal box is also assigned to the space it serves. Exported to IFC, the unit becomes an instance of a unitary equipment class, with its properties carried as property sets and its connections preserved through ports and distribution systems, provided the export was configured properly. An IFC viewer can show the unit, list its properties and, if the system relationships were carried, highlight every terminal box it feeds. What has gone is the native tool's ability to recalculate sizes; what remains is a faithful snapshot. In the COBie handover, the unit appears as a row in the component sheet, with a name, a type, a space where it sits, a serial number, an installation date, and warranty and maintenance details linked from other sheets. The system sheet may list "Supply Air System 1" and the components belonging to it. This is what a maintenance team needs to schedule filter changes and track warranty claims. But the row says nothing about which building automation points belong to the unit. In a Brick model, the unit is an entity of the air-handling unit class. It has points: a supply air temperature sensor, a supply air temperature setpoint, a supply fan speed command, a chilled-water valve command, an outdoor air damper command, a mixed air temperature sensor, each linked to the actual identifier in the building automation system. It feeds forty terminal box entities, each of which has its own points and feeds a zone, which contains rooms. An analytics application can ask, in one query, for every terminal box fed by this unit whose reheat valve is open while the unit's cooling valve is also open. That is the question at the heart of the simultaneous heating and cooling problem described in Chapter 1, and none of the other descriptions can answer it. The four descriptions are not rivals. Each serves a different purpose, and a mature information chain uses all of them, linked by a common identifier for the unit. The failure mode is not choosing the wrong one. It is creating only the first, for the purposes of design, and expecting the others to appear by themselves. Shared meaning, not shared files The lesson of the interoperability story is that exchanging files is the easy part. Agreeing on meaning, and on who is responsible for creating it, is the hard part. A building's information can be technically interoperable, flowing cleanly from one tool to another in an open format, and still be useless for operation because nobody recorded the relationships that operation depends on. For an owner, this has a direct practical consequence. The value of BIM to building performance is decided largely in the contract, before a line is drawn. An owner who wants a building that can later be monitored, diagnosed and optimised by software must specify, in the language of ISO 19650 and in vocabularies such as COBie and Brick, what information it requires about its HVAC systems and how that information must be structured at handover. It must then check that it receives it. Most owners do not yet do this, which is why the most common experience of smart building projects in existing buildings is a long and expensive period in which an analytics vendor sends engineers to reverse-engineer what the building contains. That reverse engineering takes place at the point where the information chain is most fragile: the handover from construction to operation. Chapter 4: Handover, Where Information Is Lost At the end of construction there is a period, usually a few weeks and rarely as long as anyone planned, in which a building passes from the people who built it to the people who will run it. Keys are handed over. Operating and maintenance manuals arrive, often as thousands of scanned PDF pages organised by subcontractor rather than by system. Warranties begin. The contractor's site team disperses to the next project, taking with it the detailed knowledge of what was actually installed, which compromises were made on site and which settings were adjusted to get the building through its final inspections. This moment is the weakest link in the information chain. Everything the design and construction teams knew about the building has, at least in principle, been recorded somewhere. Very little of it arrives in a form that the operations team can use, and almost none of it arrives connected to the building automation system that will run the HVAC plant for the next twenty or thirty years. What the operator actually receives To see the problem concretely, consider what an operator needs to answer a simple question: why is the third-floor conference room too warm on summer afternoons? The operator needs to know which terminal box serves the room, which air-handling unit feeds that box, what the box's airflow limits are set to, what the air handler's supply temperature is doing, whether the room's temperature sensor is correctly located and calibrated, and how the room's actual occupancy and heat gains compare with the design. In a well-run building with a good engineer who has worked there for years, much of this lives in that engineer's head. In most buildings, it must be pieced together from mechanical drawings that may not reflect late changes, a building automation system whose point names may not match the drawing tags, a maintenance system that records equipment by a third set of identifiers, and a walk up a ladder to look above the ceiling. The design model, if one exists, contains most of the physical answer: the box, its location, the duct connecting it to the air handler, the design airflow. But it is in a different system from the building automation software, using different identifiers, and nobody has linked them. The building automation system contains the live values: the box's damper position, its airflow reading, its setpoints. But it has no structured idea of where the box is, what it serves or what it was designed to do. The two halves of the answer exist. They are simply not joined. The industry's standard response is the record model, sometimes called the as-built model: a version of the BIM updated at the end of construction to reflect what was actually installed. In theory this gives the operator an accurate physical description to start from. In practice record models vary widely in quality. Updating a model to match site reality is laborious, the contractor's incentive to do it well is limited once practical completion has been certified, and owners rarely check the result. Equipment may be modelled correctly in geometry but carry design-stage data rather than the make, model and serial number actually installed. System assignments may be missing. Late changes made by subcontractors may never reach the model at all. There is also a deeper mismatch of purpose. A record model is organised the way designers think about a building: by discipline, by floor and by element type. The operator thinks about it as a set of systems that interact. The question the operator asks is rarely "what is on level three" and usually "what does this air handler serve and what is it doing". A record model can answer the first question well and the second only if system relationships were deliberately modelled and maintained. Commissioning as the bridge The one process at handover that is specifically designed to connect intended behaviour with actual behaviour is commissioning. As Chapter 1 noted, its results are impressive: a median 13 percent energy saving in new buildings, according to Lawrence Berkeley National Laboratory's 2009 study of more than 600 projects. Commissioning is also the activity in which the physical building, the design intent and the control system all have to be looked at together. The commissioning agent works from the owner's project requirements and the designer's basis of design, checks that each sensor reads correctly and each actuator moves as commanded, and tests the sequences of operation against the specification. This makes commissioning the natural moment to build the bridge between the model and the control system. When a commissioning agent verifies that the temperature sensor labelled "FL3-VAV-12-ZN-T" in the building automation system is in fact the sensor in the third-floor conference room, served by terminal box 3-12 on air-handling unit 2, that is precisely the piece of information an analytics platform will later need. In most projects it is recorded on a checklist, signed and filed in a binder, and never entered into any system where software could use it. The improvement is not technically difficult. If the owner's information requirements specify that the handover must include a machine-readable model of the HVAC systems, with every control point linked to the equipment it belongs to and every piece of equipment linked to the spaces it serves, using a vocabulary such as Brick, then the commissioning process becomes the point at which that model is verified. The controls contractor generates the point list; the designer's model supplies the equipment and topology; the commissioning agent confirms that each link is correct in the field. The output is a semantic model of the building's systems that reflects reality on the day of handover. A model of this kind should greatly shorten the onboarding of any analytics or control software, since the mapping work it replaces is the slowest part of most deployments, though published, independent measurements of the effect are still scarce. The commissioning standards already point in this direction. ASHRAE Standard 202 and its companion guideline describe a commissioning process that begins at project inception with the owner's project requirements and ends with a systems manual and training for operating staff. The systems manual is meant to explain how the building's systems work, what their design intent is and how they should be operated. On most projects it is a document, written for people. Turning its core content, the equipment, the relationships, the sequences and the verified setpoints, into structured data written for software is the natural next step, and it requires no new process, only a decision about the form of the deliverable. Mapping the existing building For the large majority of buildings, there is no design model and no handover to improve. The building has been running for years, its documentation is incomplete, and its building automation system has been modified by several generations of technicians. Bringing such a building into the information chain means creating the system model after the fact. The usual starting point is the building automation system's own point list, which can be exported from most systems. A typical list contains thousands of entries, each with a name, a type, a unit, a present value and sometimes a description. The task is to work out which points belong to which equipment, what each point measures or commands, and how the equipment is connected. Engineers have done this by hand for decades. It is tedious and error-prone, and for a large portfolio it is a major cost. This is one of the places where machine learning has had a quiet but genuine impact. Researchers and vendors have developed methods that infer point types from names, units and value patterns: a point whose name contains "SAT" and whose value hovers around 13 degrees is probably a supply air temperature. More sophisticated approaches look at how values move together over time. If a terminal box's damper opens shortly after a particular zone's temperature rises, the two points probably belong to the same control loop; if a group of terminal boxes all respond when a particular air handler's supply temperature changes, they are probably fed by it. These methods do not replace engineering judgement, but they can produce a draft model that an engineer corrects rather than creates from scratch, which changes the economics of bringing large portfolios online. Physical surveys still matter. Point lists say nothing about equipment that has no controls, about which ducts actually connect which boxes, or about damage and modifications that were never documented. Laser scanning and photography, discussed in Chapter 2, help where geometry matters. For the purposes of control, a disciplined walkdown with the mechanical drawings, verifying equipment tags and connections, often delivers most of the value at a fraction of the cost of a full geometric model. To make the process concrete, consider how a disciplined team would bring a typical existing building into the information chain: a twenty-storey office tower built in the late 1980s, with a central chilled-water plant, gas boilers, four large air handlers and several hundred variable air volume terminal boxes, whose building automation system has been upgraded piecemeal twice. The first step is an inventory of what exists in digital form. The team exports the point list from the building automation system and, in a building of this kind, might find around 9,000 points, named according to at least three different conventions reflecting the original installer and the two later upgrades. It gathers whatever drawings exist, which turn out to be a mix of original paper drawings, scanned to PDF, and CAD files from a refurbishment of six floors a decade ago. It extracts the equipment register from the maintenance management system, which lists the major plant accurately but records terminal boxes only as a count per floor. The second step is to build a draft system model from the point list. Automated classification assigns most points to a type, such as zone temperature, damper command or airflow, with a confidence score. Points with low confidence are flagged for review. The team groups points into equipment by naming patterns and confirms the groupings by checking that values behave consistently. It then infers which terminal boxes belong to which air handler, first from the drawings and naming, then by examining the data: when one air handler's supply temperature changed during a past maintenance event, which boxes' discharge temperatures moved with it? The third step is field verification. A technician walks the building with the draft model, checking a sample of terminal boxes on each floor: is the box where the model says it is, does its tag match, does the sensor in the room correspond to the point the model links it to? Discrepancies are common. On a typical floor, a few boxes may have been relabelled during a fit-out, a zone sensor may have been moved behind a new partition, and one box may have been added without ever being integrated into the controls. Each discovery is corrected in the model. The fourth step is to record spaces and occupancy assumptions. Each zone is linked to the rooms it serves, with their use and floor area, so that ventilation and comfort requirements can be associated with the right equipment. Where occupancy sensors exist, they are linked to the same rooms. The output is a semantic model of the building's HVAC systems in an open vocabulary, owned by the building's owner, with every point mapped and every relationship verified to an agreed sampling standard. With it, a fault detection rule library can be applied in days rather than weeks, Guideline 36-style sequences can be specified against real equipment, and any supervisory control vendor can be given the model rather than being paid to recreate it. None of this is technically remarkable. The difficulty is organisational: deciding that the model is worth creating as an asset in its own right, assigning someone to own it, and paying for it separately from any analytics product, so that it is not locked inside one vendor's platform. At this point the term digital twin usually enters the conversation, and it needs to be handled carefully, because it is used to mean almost anything. In its strict sense, borrowed from aerospace and manufacturing, a digital twin is a computational representation of a specific physical asset that is kept synchronised with that asset through measured data and that can be used to predict its behaviour. Three elements are essential: a structured description of the specific asset, a live data connection, and some ability to simulate or predict. A three-dimensional model on a screen with live sensor values floating over it has the first two but not the third. A generic energy simulation of a building type has the third but not the first two. For HVAC operation, the useful twin is usually far less visually impressive than the term suggests. It consists of a semantic model of the building's systems, linked to the live data stream from the building automation system and to a set of models, physical, statistical or both, that can predict how zones will respond to changes in conditions and control actions. It may never be rendered in three dimensions at all. Its value lies in letting software ask and answer questions such as: if the supply air temperature on this air handler is raised by one degree, which zones will lose comfort first, and how much fan and reheat energy will change? A twin of this kind depends on the information chain described so far. Without the semantic model, the software cannot know which zones a change affects. Without live data, it cannot know the current state. Without the physical description from BIM or survey, its predictive models have to learn everything from data, which takes longer and generalises less well. The phrase "digital twin" is not a product that can be bought. It is the name for what results when the earlier links in the chain have been built and connected. The ownership question deserves emphasis because it is where many otherwise sound projects go wrong. When an analytics or AI vendor builds the system model as part of onboarding its own product, the model typically lives inside that vendor's platform, in a proprietary form, and is not delivered to the owner. The owner has paid for it, through set-up fees or a longer subscription, but cannot take it to a competitor, use it for a different application or hand it to a new controls contractor. Contracts for any work that creates or updates a system model should state that the model belongs to the owner, must be kept current, and must be delivered in an open vocabulary on request and at the end of the engagement. This is a small clause with large consequences for the owner's freedom to choose technology later. Keeping the model alive The final difficulty at handover is that it is not a single moment. Buildings change continuously. Tenants fit out floors and add partitions. Equipment is replaced, often with units of different capacity or from a different manufacturer. Controls contractors add points, rename them and modify sequences. A model that was accurate on the day of handover may be badly wrong five years later. Owners who succeed in keeping their information chain intact treat the system model as an operational asset with a named owner, not as a construction deliverable. Every change to the HVAC systems or the building automation system, whether it is a tenant fit-out, an equipment replacement or a sequence change, is required to update the model, and contracts with fit-out contractors and controls vendors say so. Some organisations check consistency automatically, comparing the live point list against the semantic model and flagging new, missing or renamed points. This discipline is unglamorous and rarely appears in PropTech marketing. It is nonetheless the single most important determinant of whether the analytics and AI control discussed in later chapters keep working. A sophisticated control algorithm running on an outdated model of the building is operating blind in a new way: confident, fast and wrong. The physical description of the building and its systems, maintained over time, is the first link in the chain. The second is knowledge of the people who use it. Chapter 5: Counting People A commercial HVAC system exists for the people inside the building. It heats, cools and ventilates to keep them comfortable and to dilute the carbon dioxide, moisture and pollutants they and their activities produce. Yet in most buildings the control system has no direct knowledge of whether anyone is there. It knows the time of day, the day of the week, and the temperatures and pressures its sensors report. Occupancy enters only indirectly, as a schedule someone programmed and as heat that eventually shows up as a warmer room. This chapter is about the second link in the information chain: measuring how many people are in each part of a building, and when. It is the link that has changed most since 2020, both because sensors have become cheaper and because occupancy itself has become far less predictable. Why occupancy is the missing variable Recall the four sources of waste from Chapter 1: ventilation for absent people, simultaneous heating and cooling, excessive operating hours, and faults. Occupancy information addresses the first and third directly, and the second indirectly. Ventilation is the clearest case. Outdoor air requirements in standards such as ASHRAE 62.1 are built from a per-person component and a per-area component. The per-area part covers emissions from the building and its furnishings and must be supplied whenever the space is in use. The per-person part covers what occupants themselves emit and, in principle, scales with how many are present. If a control system knows the actual number of people in a zone, it can reduce the per-person portion of outdoor air accordingly. In a cold or hot, humid climate, where every cubic metre of outdoor air must be heated or cooled and dehumidified, that reduction has real value. Operating hours are the second case. A zone that is empty does not need to be held at occupied setpoints. Most buildings already have unoccupied modes for nights and weekends; occupancy sensing allows the same logic to apply to individual floors, wings or rooms during the day. Recent editions of ASHRAE's ventilation and energy standards recognise an "occupied standby" mode for certain space types, in which a room within an occupied building that is temporarily empty, detected by an occupancy sensor, can have its ventilation reduced or stopped and its temperature setpoints widened. A conference room booked for an hour and used for twenty minutes, or a floor of a hybrid-work office that sits empty on Fridays, is exactly where this matters. Occupancy also helps with simultaneous heating and cooling. Terminal box minimum airflows are often set to provide design-occupancy ventilation at all times. If the system knows a zone is lightly occupied or empty, it can lower that minimum, reducing the amount of cold air that has to be reheated. Before 2020, the pattern of occupancy in most offices was reasonably regular, if lower than design. Workplace studies regularly found desks unused for much of the working day, because of meetings, travel, leave and part-time work, but the building filled and emptied on a predictable weekday rhythm. A fixed schedule was wasteful, but not wildly so. Hybrid working broke that rhythm. Kastle Systems' access-card data, the most widely cited public series in the United States, showed average weekly office occupancy across its sample at about 56 percent of the pre-pandemic baseline in December 2025, with Tuesday peaks of around 66 percent and the best-occupied premium buildings considerably higher. These figures measure entries relative to a past baseline rather than seats filled, so they should not be read as the fraction of desks in use. What they show clearly is that attendance now varies sharply by day of the week, by building and by tenant, and that the Monday-to-Friday design assumption is wrong on most days for most buildings. This has two consequences. The first is that the energy waste from running at design occupancy has grown. The second is that owners and tenants now have a separate, commercial reason to measure occupancy: to decide how much space they need, how to lay it out and whether to consolidate. Many organisations have installed occupancy sensing for space planning, and the same data can feed HVAC control at little additional cost if the systems are connected. This is one of the clearest examples of the argument of this booklet. The information already exists in many buildings; what is missing is the link between the workplace analytics platform and the building automation system. How occupancy is measured There is no single occupancy sensor. Several technologies are in use, and they differ in what they detect, how accurately, at what cost and with what implications for privacy. Table 2 compares the main options, and the paragraphs that follow it explain the trade-offs. Table 2. Common occupancy-sensing technologies and their trade-offs. Technology What it detects Main strength Main weakness Privacy exposure Passive infrared (PIR) Motion of warm bodies Cheap, already common in lighting Misses still occupants; presence only Low Carbon dioxide sensor Accumulated exhaled CO2 Directly tied to ventilation need Lags occupancy; affected by airflow Low Thermopile or low-resolution thermal array Heat signatures in a field of view Counts people, detects still occupants Limited coverage per sensor Low to moderate Time-of-flight or doorway counter People crossing a threshold Accurate zone counts Errors accumulate if counts drift Low to moderate Wi-Fi or Bluetooth device data Connected phones and laptops Uses existing network infrastructure Devices are not people; coarse location Moderate to high Camera with on-device analytics People in an image High accuracy and detail Cost, acceptance, data protection High Passive infrared sensors are the default. They detect changes in the infrared radiation reaching them, which in practice means moving warm bodies. They are inexpensive and already installed in many buildings for lighting control. Their weakness is that they detect motion, not presence. A person typing quietly at a desk may fall below the detection threshold, which is why lights in offices with PIR control sometimes go out on people who are working. They also say nothing about how many people are present, only whether anyone has recently moved. Carbon dioxide sensors measure the gas that occupants exhale. Indoor carbon dioxide concentration rises above the outdoor level, currently a little over 420 parts per million, in proportion to the number of people and inversely to the ventilation rate. Because carbon dioxide is directly related to the per-person ventilation requirement, it has long been used for demand-controlled ventilation, in which outdoor air is increased as indoor concentration rises. Its weaknesses are lag and ambiguity. A room fills with people well before its carbon dioxide level rises, and the level depends on the ventilation rate as well as on occupancy, so a well-ventilated crowded room and a poorly ventilated nearly empty room can read the same. Cheaper sensors also drift and need recalibration. Thermal array sensors, which use a small grid of thermopile elements to produce a very low-resolution heat image, can count people within their field of view and detect people who are sitting still, without capturing anything that resembles a recognisable image. Doorway counters, using time-of-flight or similar depth sensing, count people entering and leaving a zone. They can be very accurate for individual crossings, but small errors accumulate over a day, so they need periodic resetting, typically when a zone is known to be empty overnight. Wi-Fi and Bluetooth data infer occupancy from the number of devices connected to access points in each area. This is attractive because the infrastructure already exists, but devices are not people: one person may carry a phone, a laptop and a watch, and another may carry nothing that connects. Location resolution is coarse, typically a floor or a part of one. Cameras with on-device analytics are the most accurate and the most contentious. Modern units can count people and discard the images on the device, transmitting only numbers, but tenants and employees understandably react differently to a camera than to a heat sensor. Accuracy, and how much is enough The right question is not which sensor is most accurate but how accurate the occupancy signal needs to be for the control decision it feeds. Different decisions need different information. Deciding whether a zone can enter standby needs only reliable presence detection, with a sensible delay so that a briefly empty room is not shut down. Setting ventilation proportionally to the number of occupants needs a count, but errors of a few people in a large zone matter little. Predicting the next morning's arrival pattern so that the plant can start up at the right time needs historical patterns more than real-time precision. A control strategy that asks for more precision than it needs pays in sensor cost and in complexity. A worked example shows why counts matter for ventilation. ASHRAE 62.1's ventilation rate procedure for ordinary office space combines a people component of 2.5 litres per second per person with an area component of 0.3 litres per second per square metre. Take a 1,000-square-metre open-plan floor designed for 100 people. At design occupancy, the people component is 250 litres per second and the area component 300, for a total of 550 litres per second of outdoor air at the breathing zone, before adjustments for how efficiently air is distributed. Now suppose only 40 people are present, as is common on a Monday or Friday in a hybrid office. The people component falls to 100 litres per second and the total to 400, a reduction of more than a quarter in outdoor air that must be heated, cooled or dehumidified. If the building has no count, it ventilates for 100 people every day. The same arithmetic shows the limits. The area component does not change with occupancy, so even an empty but occupied-mode floor still needs 300 litres per second; only switching the zone to standby, where codes allow, removes it. And the saving depends on climate. In a mild season, when outdoor air is close to indoor conditions, conditioning it costs little and reducing it saves little; it may even be better to bring in more for free cooling. The control logic must know not just how many people are present but when reducing outdoor air actually saves energy. The most direct evidence on what occupancy-based control achieves in practice comes from controlled experiments rather than vendor claims. In a study published in Applied Energy in 2022, Meng Kong, Bing Dong, Rongpeng Zhang and Zheng O'Neill integrated three different occupancy-sensing technologies into the real-time HVAC control of commercial building spaces and compared them side by side against conventional control. Weekly average HVAC energy savings ranged from 17 to 24 percent, while daily savings varied with the accuracy of each sensor and with weather. Occupants' satisfaction with thermal comfort and perceived air quality stayed above 80 percent. The result is useful for two reasons. It shows that the savings are substantial and do not require sacrificing comfort, and it shows that sensor accuracy directly affects how much is saved. A related experiment by the United States General Services Administration took a different approach, asking occupants rather than sensing them. In 2015 its Green Proving Ground programme tested a system that allowed occupants in a federal building in Phoenix to request warmer or cooler air from their phones, with the system learning their preferences over time. Requests for temperature changes to maintenance staff fell by 59 percent and 83 percent of occupants reported being more satisfied. Modelled savings were large, but the evaluators were candid that the subscription cost could exceed the energy savings in many buildings. The case illustrates a pattern that recurs throughout this field: knowing what occupants want and where they are is valuable, but the business case depends on the cost of obtaining that knowledge relative to the energy and comfort it buys. Sensors are not the only source of occupancy information, and for some decisions they are not the best one. Room booking systems record when meeting rooms are expected to be used, by how many people, often days in advance. Access control systems record entries to the building and, in some cases, to individual floors. Workplace scheduling tools in hybrid offices record which teams plan to come in on which days. Each of these is a forecast rather than a measurement, and each is imperfect: meetings are booked and not held, people badge in and leave, plans change. But a forecast is exactly what a controller needs to decide when to start pre-conditioning a floor, and combining a forecast from bookings with a measurement from sensors lets the system both plan ahead and correct itself. A conference room booked for ten people at two o'clock can be pre-cooled from half past one; if the sensor shows nobody has arrived by a quarter past two, the room can be released to standby and the booking flagged as a no-show. Forecasting from history is equally useful. A building that has recorded occupancy floor by floor for a few months can learn that one tenant fills its floors early on Tuesdays and Wednesdays and barely appears on Fridays, while another arrives late and stays late. Start-up times, ventilation plans and even which air handlers run at all on a given day can then be matched to expected use. This is one of the simplest and most reliable uses of occupancy data, and it does not require any sensor more accurate than a well-placed counter. Privacy and trust Occupancy sensing raises legitimate concerns about surveillance, and the way an organisation addresses them decides whether the system is accepted. In the European Union, the General Data Protection Regulation applies whenever data can identify an individual, which can include device identifiers and sometimes data detailed enough to track a person's movements across a building. Even where the law does not apply, employees and tenants may object to being monitored, and their objection can end a project. Several principles reduce the risk. Collect the least detailed data that serves the control purpose: counts rather than identities, zones rather than desks, and aggregated rather than individual records. Process data on the device where possible, so that images or device identifiers never leave the sensor. Separate occupancy data used for HVAC control from any data used for workplace analytics or security, with different access rules. Tell occupants what is collected and why. These measures are not only ethical safeguards; they make the system easier to deploy and maintain. For HVAC purposes, the privacy-preserving options are usually sufficient. A ventilation controller needs to know that a zone contains about twelve people, not who they are. A plant start-up algorithm needs to know that a floor typically fills by half past nine on Tuesdays, not which employee arrives first. The engineering need and the privacy interest point in the same direction more often than vendors of detailed tracking systems suggest. Having occupancy data is not the same as using it. In many buildings, occupancy sensors installed for lighting control or workplace analytics are not connected to the HVAC system at all. Where they are, the connection is often limited to a single binary signal for a room. Making occupancy useful for HVAC requires the semantic model discussed in Chapter 4: the control system must know which sensors cover which spaces, which spaces belong to which HVAC zones, and which terminal boxes and air handlers serve those zones. A count of twelve people in a room is useless if the controller does not know which box to adjust. Occupancy is therefore the second link in the chain, and it depends on the first. The physical and system model tells the controller where the people are in relation to the equipment; the occupancy sensors tell it how many there are. The third link, examined in the next two chapters, is the control logic that turns that knowledge into action, and the first thing to say about it is that most of its value comes not from intelligence but from doing well what conventional control was always supposed to do. Chapter 6: Control Before Intelligence Before asking what artificial intelligence can do for a building's HVAC system, it is worth asking what ordinary control is supposed to do and how often it does it. The answer to the second question is: much less often than it should. A large share of the savings attributed to smart buildings comes from correcting control logic that was poorly specified, badly programmed or allowed to decay. That is not a criticism of smart building technology. It is a statement about where the reliable savings lie, and about why the information chain matters even before any machine learning is involved. A building automation system is built from layers of feedback loops. At the bottom, a local loop holds one variable at a setpoint by adjusting one actuator. A terminal box modulates its damper to hold the zone at its temperature setpoint. An air-handling unit modulates its chilled-water valve to hold the air leaving the cooling coil at its supply air temperature setpoint. The supply fan's variable-speed drive adjusts fan speed to hold a static pressure in the duct. Each of these loops is usually a proportional-integral controller, a simple algorithm, decades old, that compares the measured value with the setpoint and adjusts the output in proportion to the error and to the accumulated error over time. Local loops are well understood and generally work, provided they are tuned and their sensors are accurate. The energy performance of a building is decided mainly one level up, by supervisory logic that chooses the setpoints the local loops pursue. What supply air temperature should the air handler deliver? What duct static pressure should the fan maintain? What chilled-water temperature should the plant produce? When should the economiser bring in outdoor air for free cooling? When should the building start up in the morning, and when should each zone's minimum airflow apply? These choices are written in the sequences of operation, the narrative description of control logic in the mechanical specification. A typical sequence says, for example, that the supply fan shall maintain duct static pressure at a setpoint, and that the setpoint shall be adjusted according to some rule. Historically the rule was often absent, or written so loosely that each controls contractor implemented it differently. The result was fixed setpoints chosen to satisfy the worst case: a static pressure high enough to serve the most remote box on the hottest day, a supply air temperature cold enough to cool the hottest zone. The fans then spent most of the year pushing more air at higher pressure than any zone needed, and the terminal boxes throttled it back and reheated it. Resets: asking the zones what they need The central technique of good supervisory control is the reset: adjusting a system-level setpoint based on what the zones it serves actually require. Two examples show how it works and why it saves so much. Consider duct static pressure. The supply fan must create enough pressure for every terminal box to obtain the airflow it needs. Only a few boxes, at any moment, are close to fully open and struggling to get enough air. If the fan maintains a fixed, high pressure, every other box must throttle its damper to waste the excess pressure. A static pressure reset lowers the setpoint gradually until some boxes start to report that they cannot get enough air, then raises it slightly. The fan runs at the lowest pressure that satisfies the most demanding zones. Because fan power rises steeply with pressure and flow, a modest reduction in pressure produces a substantial reduction in fan energy. The method most widely recommended for this is known as trim and respond. At fixed intervals, the controller trims the setpoint down by a small amount. Each zone that is starved of air, for example a box whose damper has been more than 95 percent open for a sustained period, generates a request. If the number of requests exceeds a small threshold, the controller responds by raising the setpoint by a larger amount in proportion to the requests. The setpoint therefore drifts down continuously until the zones push back, then settles at the lowest acceptable level. A small number of rogue zones, such as a box with a failed sensor that always reports being starved, can be excluded by ignoring a set number of requests. Supply air temperature reset works the same way in the other direction. In mild weather, if no zone needs the coldest air, the air handler can deliver warmer air. That saves chiller energy and, crucially, reduces reheat, because zones that would otherwise receive cold air and warm it up now receive air that needs less warming. The trade-off is fan energy, because warmer air carries less cooling and more of it must be moved, and the best sequences balance these effects according to outdoor conditions. A simple illustration shows how the logic behaves over a morning. Suppose an air handler starts the day with its static pressure setpoint at a deliberately high 300 pascals. Every two minutes the controller trims it by 10 pascals. For the first hour, with the building cool and lightly loaded, no zone is starved, so the setpoint drifts steadily down, past 250, past 200. At 150 pascals, as the sun reaches the east façade, two east-facing boxes open fully and begin generating requests. If the controller is set to ignore two requests, nothing happens yet. At 140 pascals a third box joins them, and the controller responds by raising the setpoint by an amount proportional to the excess requests, say 15 pascals. The setpoint then hunts in a narrow band, just high enough to satisfy the most demanding boxes, and it rises and falls through the day as the sun moves and the loads shift. The fan spends most of its hours well below the fixed setpoint that a conventional design would have chosen, and because fan power falls steeply with pressure and flow, the energy saved is disproportionately large. These numbers are illustrative, but the pattern is the one described in Guideline 36. Neither technique requires any intelligence beyond counting requests and adjusting a number. Both require information: the controller must be able to read the damper position, airflow and temperature of every zone served by the air handler, and it must know which zones those are. This is the system topology from Chapter 4 again. A building whose automation system does not know which boxes belong to which air handler cannot implement a trim-and-respond reset properly, however modern its controllers. Guideline 36 and the standardisation of good practice For decades, good sequences existed in the heads and files of a minority of skilled engineers and controls programmers. Most buildings received something worse, because each project specified and programmed its sequences from scratch. ASHRAE Guideline 36, first published in 2018 and revised since, was a deliberate attempt to change that. It provides complete, detailed, tested sequences of operation for the most common HVAC system types, including multiple-zone variable air volume air handlers, their terminal boxes, and the plants that serve them, written precisely enough to be programmed consistently and verified objectively. Guideline 36 includes trim-and-respond resets for static pressure and supply air temperature, ventilation logic that calculates outdoor air requirements zone by zone, terminal box logic that allows minimum airflows to be set much lower than traditional practice, economiser control with proper safeguards, and built-in fault detection rules that flag common failures. It also includes logic for the occupied-standby mode discussed in Chapter 5, using occupancy sensors to reduce ventilation and widen setpoints in temporarily empty spaces. Publishing good sequences does not guarantee they will be implemented faithfully. A sequence written in prose must still be translated by a controls programmer into the proprietary language of a particular manufacturer's controllers, and that translation is where errors enter: a reset implemented with the wrong sign, a request threshold misread, a time delay omitted. Lawrence Berkeley National Laboratory's OpenBuildingControl project has addressed this by expressing Guideline 36 sequences in a formal, machine-readable control description language that can be simulated before installation, translated automatically into vendor programming where tools allow, and used to verify, from trend data, that the installed system behaves as specified. The point is the same one made throughout this booklet: control logic is information, and it is lost or corrupted at each handover unless it is captured in a form that can be checked. How much does this matter? In a simulation study published in 2022 in the Journal of Building Performance Simulation, Kun Zhang, David Blum, Hwakong Cheng, Gwelen Paliaga, Michael Wetter and Jessica Granderson of Lawrence Berkeley National Laboratory and Taylor Engineers estimated the savings from Guideline 36 sequences for multiple-zone variable air volume systems compared with baseline sequences representing typical current practice. Across the climates and conditions modelled, the savings varied widely but averaged 31 percent of HVAC energy. Simulation results should be treated with caution, and real buildings rarely match models exactly, but the direction is clear: simply standardising good conventional control can save a large share of HVAC energy in the system type that dominates large commercial buildings. Large buildings with central plants are only part of the commercial stock. A great many smaller buildings, such as retail stores, restaurants, schools and low-rise offices, are served by packaged rooftop units, each a self-contained box with a compressor, fans and controls, serving one zone or a few. These units have traditionally been controlled very simply: a thermostat calls for heating or cooling, and the unit runs its fan at full speed while ventilating at a fixed outdoor air fraction. The field trial by Pacific Northwest National Laboratory mentioned in the introduction tested what happened when such units were retrofitted with controls borrowed from larger systems: variable-speed supply fans that slowed down when full airflow was not needed, demand-controlled ventilation that adjusted outdoor air to carbon dioxide levels, and better integration of the economiser with mechanical cooling. Sixty-six units in eight buildings across Washington, Ohio, California and Pennsylvania, including shopping centres, grocery stores, big-box stores and a medical clinic, were monitored over about a year from 2012. The average reduction in the units' heating, ventilating and cooling electricity was 57 percent, with individual buildings ranging from 20 to 90 percent. The authors estimated savings of about 1,500 dollars per unit per year at national average electricity prices and a typical payback of around three years. That result is often quoted as evidence for smart HVAC, and in a sense it is. But it is worth noticing what made it possible. The baseline was a fan running at constant full speed whenever the unit was on, and ventilation set without regard to occupancy. Most of the saving came from fan speed, because fan power falls roughly with the cube of speed. The intervention was neither artificial intelligence nor a cloud platform. It was a variable-speed drive, a carbon dioxide sensor and better control logic. The size of the saving was a measure of how wasteful the baseline was. The economiser deserves a separate mention, because it combines large potential savings with an unusually high failure rate. An economiser uses outdoor air for cooling whenever the outdoor air is cooler than the return air, opening its dampers wide and reducing or eliminating the need for mechanical cooling. In temperate climates this can supply much of the cooling needed in spring and autumn. But the dampers, linkages and actuators are mechanically exposed, the outdoor air sensors drift, and the control logic that decides when to economise and how to coordinate it with mechanical cooling is easy to get wrong. A failed economiser is invisible to occupants, because mechanical cooling simply takes over. Field studies of rooftop units have repeatedly found high rates of economiser faults, and the PNNL trial described below included better economiser integration among its measures. Guideline 36 specifies the economiser logic in detail precisely because so many installed systems get it wrong. Fault detection: finding what has broken Even well-designed sequences degrade when equipment fails. Sensors drift, actuators stick, valves leak and overrides are left in place. Because the HVAC system compensates, faults can persist for years. The second pillar of good conventional control is therefore systematic fault detection and diagnostics: software that continuously monitors building automation data, applies rules that encode engineering knowledge, and flags conditions that indicate a fault. The rules are often simple. If the outdoor air damper is commanded to minimum but the mixed air temperature is close to the outdoor temperature, the damper is probably stuck open. If the heating and cooling valves on the same air handler are both open at the same time, something is fighting itself. If a zone's airflow reading is well below its setpoint while its damper is fully open, the box is starved or its sensor is wrong. If a supply fan is running at three in the morning on a Sunday with no override recorded, something is wrong with the schedule. Guideline 36 includes a standard set of such rules for air handlers, and commercial analytics platforms offer libraries of hundreds. What does fault detection achieve in practice? The best evidence comes from the Smart Energy Analytics Campaign, a programme run by the United States Department of Energy and Lawrence Berkeley National Laboratory that tracked the experience of organisations installing energy management and information systems. By 2019, the campaign had data from 96 organisations covering nearly 6,000 buildings and 518 million square feet. Two years after installation, the median energy saving among users of fault detection and diagnostics software was 9 percent, and among users of simpler energy information systems, which visualise meter data without diagnosing faults, 4 percent. Both delivered typical simple paybacks of one to two years. Nine percent is far below the headline figures quoted for individual projects, and that is the point. Fault detection software finds problems; it does not fix them. Every alert must be investigated by someone with the time, skill and authority to act, and in many organisations the backlog of unaddressed alerts grows faster than the maintenance team can clear it. The portfolios that achieved the largest savings in the campaign were those that built processes around the software: dedicated analysts, clear routines for triaging and assigning faults, and follow-up to confirm the fix. The analytics were necessary but not sufficient. What conventional control teaches Three lessons from this chapter shape everything that follows. First, a large share of available HVAC savings can be captured with well-understood, conventional techniques: resets, proper minimum airflows, demand-controlled ventilation, variable-speed fans, sensible schedules and systematic fault detection. These techniques are mature, their behaviour is predictable, and they are increasingly standardised. Any intelligent control system that is sold on a 20 or 30 percent saving should be asked how much of that saving would come from simply implementing these basics. Second, even conventional good practice depends on information that many buildings lack. A trim-and-respond reset requires knowing which zones belong to which air handler. Demand-controlled ventilation requires carbon dioxide or occupancy measurement in the right places. Fault detection requires knowing what each point measures. The semantic model and the occupancy data from earlier chapters are not optional extras for AI; they are prerequisites for doing the basics well. Third, savings from software depend on the organisation around the software. The gap between the 57 percent achieved in a carefully run field trial and the 9 percent median in a large portfolio is partly a difference in baselines, but it is also a difference in how faithfully the findings were acted on. Technology that relies on people to act on its recommendations will save what those people have time to do. The last lesson leads directly to the appeal of the systems discussed in the next chapter. If people are the bottleneck, why not let the software act directly on the controls? That is the promise of autonomous AI-driven HVAC, and it changes both what can be achieved and what can go wrong. Chapter 7: Learning Machines in the Plant Room The conventional control described in the previous chapter reacts. It measures what is happening now and adjusts setpoints according to rules written in advance. A trim-and-respond reset does not know that the afternoon will be hot, that a large meeting is booked in the fourth-floor conference room at two o'clock, or that electricity will be expensive between four and seven. It finds out when the zones start complaining. The techniques grouped under the label of AI-driven HVAC differ in one essential respect: they anticipate. They use a model of how the building behaves, whether derived from physics, learned from data or both, to predict the consequences of possible control actions over the coming hours, and choose the actions that achieve the best outcome against a defined objective. That ability to look ahead is what allows them, in the right conditions, to save energy beyond what good conventional control achieves, and to do things conventional control cannot do at all, such as shift load in time to suit the electricity grid. Model predictive control The oldest and best-established anticipatory technique is model predictive control, developed in the process industries from the 1970s onwards. The idea is straightforward. At each control step, perhaps every fifteen minutes, the controller takes a model of the system, the current measured state, and forecasts of the disturbances that will affect it over a horizon, typically the next day: weather, solar radiation, occupancy, electricity prices. It then solves an optimisation problem to find the sequence of control actions over the horizon that minimises energy or cost while keeping every zone within its comfort limits and every piece of equipment within its operating limits. It applies only the first action in that sequence, waits for the next step, measures again and repeats the whole calculation with updated information. Applied to a building, this lets the controller exploit the building's thermal mass. On a hot day, it may pre-cool the structure in the early morning, when outdoor air is cool, chillers run efficiently and electricity is cheap, and then let the temperature drift upward within the comfort band through the afternoon peak. On a winter morning, it may start heating each zone at exactly the time needed to reach setpoint as occupants arrive, given the overnight temperature and the zone's measured heat-up rate, rather than at a fixed hour. If occupancy forecasts say a floor will be lightly used, it can plan for that floor to run at relaxed setpoints. The research evidence for model predictive control in buildings is substantial. The most instructive single study is a long-running Swiss project in which researchers from ETH Zurich and partners implemented model predictive control in an occupied office building and ran it for seven months. Their results, published in IEEE Transactions on Control Systems Technology in 2016 under the title "Model Predictive Climate Control of a Swiss Office Building: Implementation, Results, and Cost–Benefit Analysis," showed improved energy performance and comfort compared with the building's existing rule-based control. But the authors also concluded that the initial investment required, chiefly the engineering effort to build and maintain an accurate model of the specific building, was likely too high to justify deployment in everyday projects on the basis of operating cost savings alone. That conclusion is central to understanding the field. Model predictive control works when its model is good, and building a good model of a specific building has historically been expensive, slow and dependent on scarce expertise. The model needs the building's thermal zones, their construction, the equipment serving them and how it is connected, and parameters calibrated against measured data. In other words, it needs exactly the information chain described in the first five chapters. Where that chain exists, much of the cost of building the model disappears. Where it does not, each deployment begins with weeks of engineering to reconstruct what the building is. The obvious response to the cost of physics-based models is to learn the model from data instead. A building automation system that has been logging values for a year contains a great deal of information about how the building responds to weather, occupancy and control actions. Statistical and machine-learning methods can fit models that predict, for example, a zone's temperature fifteen minutes ahead from its current temperature, the outdoor temperature, solar radiation, the time of day and the supply airflow. These data-driven models have strengths and weaknesses that mirror those of physics-based models. They are cheap to build if the data exist, and they automatically capture the peculiarities of the specific building, such as the zone that overheats because of a server cupboard, or the floor whose sensors read high. But they learn only from the conditions they have seen. A building whose controls have always held a fixed supply air temperature provides little information about what would happen if that temperature were changed, so a purely data-driven model may predict poorly in exactly the regions where an optimiser wants to go. They also struggle with conditions that occur rarely, such as an extreme heat wave or an unusually crowded day. The approach most practitioners now favour is a hybrid, often called grey-box modelling. A simplified physical structure, typically representing each zone as a small network of thermal resistances and capacitances, analogous to an electrical circuit, is defined from what is known about the building. Its parameters are then estimated from measured data. The physical structure ensures that predictions obey basic laws, so that adding heat raises temperature and insulation slows heat loss, even in conditions the data do not cover. The data calibrate the structure to the real building. Here again the information chain pays off: the more accurately the zones and systems are described in advance, the less the data have to discover and the more reliable the model becomes. Reinforcement learning and the data centre case Reinforcement learning takes a different approach. Instead of building an explicit model and optimising over it, a reinforcement learning agent learns a control policy directly, by taking actions, observing outcomes and gradually improving its choices to maximise a cumulative reward, such as negative energy cost minus a penalty for discomfort. The technique gained wide attention after its successes in games and robotics, and it has been tested extensively for building control. The appeal is that, in principle, reinforcement learning needs no hand-built model and can discover strategies that engineers would not think of. The difficulty is that learning by trial and error in a real building is slow, risky and uncomfortable for occupants. In a 2020 review in Applied Energy, Zhe Wang and Tianzhen Hong of Lawrence Berkeley National Laboratory examined the literature on reinforcement learning for building controls and found that the field was still largely in the research stage, with only about 11 percent of studies involving real buildings. They identified three main obstacles: training is time-consuming and data-hungry, controllers need to be made far more safe and robust before they can be trusted in occupied buildings, and policies learned in one building transfer poorly to others. The standard workaround is to train the agent in simulation, using a detailed model of the building, and then deploy it in the real building with further fine-tuning. That approach reintroduces the need for an accurate model, which is what reinforcement learning was supposed to avoid. To make comparisons between methods fair, the research community has built shared test environments. The best known, BOPTEST, developed under the International Building Performance Simulation Association's Project 1, provides standardised building emulators against which any control algorithm, whether rule-based, predictive or learned, can be tested on the same cases with the same performance measures. Its existence reflects a recognition that claims about AI control need common benchmarks to be taken seriously. The most famous example of AI-driven cooling did not involve an office building, but it is worth examining because it is frequently cited and its lessons transfer. In 2016 Google and its DeepMind research unit reported that a machine-learning system had reduced the energy used for cooling in its data centres by 40 percent. That system made recommendations that human operators then implemented. In 2018 the companies announced a step further: an autonomous system that took snapshots of thousands of sensors every five minutes, used deep neural networks to predict how different combinations of actions would affect future energy consumption, and sent the selected actions directly to the data centre's control system. According to the companies' account, the system's performance improved steadily as it learned, and over nine months its delivered savings rose from about 12 percent to about 30 percent. The safety design is as instructive as the savings. The system considered a large number of possible actions but only implemented those for which its confidence was high. Every action was checked against constraints specified by human operators, and the local control systems verified instructions before applying them. Operators could take back control at any time, and the system was designed to revert to conventional rules if anything unexpected happened. The lessons for commercial buildings are mixed. A large data centre is an unusually favourable environment for AI control: its loads are large and continuous, its instrumentation is extensive and well maintained, its systems are documented and its operators are highly skilled. The cooling plant is the core of the business, and the organisation has data scientists on staff. A typical office building has none of these advantages. But the architecture, a learning layer that proposes actions, a constraint layer that checks them, and a conventional control layer that executes them and can always take over, has become the template for commercial products. Supervisory AI and grid flexibility The form of AI-driven HVAC most commonly sold to commercial building owners today is a supervisory layer, usually delivered as a service. A small computer is installed in the building and connected to the existing building automation system, typically over BACnet. It reads the available points, sends data to cloud-based models, and writes back adjusted setpoints, such as zone temperature setpoints, supply air temperatures, static pressure setpoints and start times, at intervals of a few minutes. The existing controllers continue to run the local loops. If the supervisory layer loses its connection or is switched off, the building reverts to its native sequences. The attraction of this architecture is that it avoids replacing the existing control system, which is costly and disruptive. It can be installed quickly and sold on a subscription with a savings guarantee. Vendors commonly advertise double-digit percentage reductions in HVAC energy. One well-documented example is 45 Broadway, a 32-storey office building in Lower Manhattan built in 1983. In a case study published by the vendor BrainBox AI with the building's owner, the system was reported to have reduced HVAC-related electricity consumption by 15.8 percent over eleven months in 2023, saving about 235,000 kilowatt-hours and 43,000 dollars. The deployment was phased floor by floor with the building's chief engineer, and the case study notes the care required around computer rooms that needed condenser water at all times. Figures from vendor case studies are useful but not independent. They depend on how the baseline was defined, how weather and occupancy were adjusted for, and which buildings were chosen for publication. Chapter 8 examines how such claims should be tested. But even taken at face value, the reported savings sit in a recognisable range: larger than the median from fault detection alone, comparable to good conventional control properly implemented, and far below the headline figures from single-site field trials with very poor baselines. It is also worth asking what the AI is actually doing in these deployments. Much of the value of a supervisory system often comes from implementing, automatically and continuously, the kinds of resets, schedule optimisation and setpoint relaxation described in the previous chapter, with predictive models used to fine-tune timing and magnitude. That is a real service, because most buildings do not have those sequences properly implemented and the supervisory layer delivers them without reprogramming the native controllers. But it means that an owner comparing a supervisory AI service against an upgrade to Guideline 36 sequences is comparing two routes to many of the same savings, with different costs, risks and degrees of ongoing dependence on a vendor. The fastest-growing reason to want predictive control has less to do with a building's own consumption than with the electricity system around it. As heating shifts from gas boilers to electric heat pumps and as grids take on more wind and solar generation, the cost and carbon intensity of electricity increasingly vary hour by hour. A kilowatt-hour consumed at midday on a sunny spring day may be cheap and nearly carbon-free; the same kilowatt-hour consumed at six on a winter evening may be expensive and generated by gas. Utilities and grid operators have begun to pay buildings to shift or shed demand at critical times, and tariffs with strongly time-varying prices are spreading. A commercial building is well suited to providing this flexibility because its structure stores heat. Concrete floors, walls and furnishings absorb and release thermal energy slowly. A building that is pre-cooled by a degree or two in the hours before an afternoon peak can then let its temperature drift back up during the peak, reducing chiller load when electricity is most valuable, without occupants noticing much difference. In winter, a building heated by heat pumps can be warmed slightly ahead of an evening peak and then allowed to coast. The United States Department of Energy has promoted this idea under the name of grid-interactive efficient buildings, and pilot programmes have demonstrated it in practice. Doing this well is a prediction problem. The controller must know how quickly each zone will warm or cool, how much load can be shifted without breaching comfort, when people will be present, and what prices or signals the grid will send. Fixed rules handle this poorly, because the right strategy depends on the day's weather, occupancy and prices together. Model predictive control handles it naturally, because it optimises over a forecast horizon. This is the area where anticipatory control has its clearest advantage over even the best reactive sequences, and it is likely to become more valuable as electrification proceeds. It also sharpens the dependence on information. A building that promises a grid operator it will shed 200 kilowatts for two hours must be confident that it can do so without overheating its tenants. That confidence comes from an accurate model of the building's thermal behaviour, reliable knowledge of who will be inside and verified performance from previous events. A building that does not know itself cannot make credible promises to the grid. Risks of letting software act Moving from software that recommends to software that acts changes the risks. Three deserve attention. The first is comfort and safety. An optimiser that is rewarded for saving energy will push setpoints toward the edges of whatever comfort band it is given. If the band is wrong, or if the model's predictions are wrong for an unusual zone, occupants will be uncomfortable before anyone notices. Some spaces, such as laboratories, data rooms, healthcare areas and spaces with strict humidity requirements, should be excluded or tightly constrained. Good deployments start with conservative limits, widen them gradually as confidence grows, and monitor complaints as closely as energy. The second is cybersecurity. A device that connects the building automation network to the internet and can write setpoints is an attack path. Building automation protocols were designed for closed networks and many older installations have weak or no authentication. Any supervisory system should be deployed with network segmentation, encrypted outbound-only connections where possible, strict limits on which points it can write, and logging of every command it issues. The third is dependence. When the AI layer is delivered as a subscription, the savings stop when the subscription stops, and the knowledge of how the building is being optimised sits with the vendor. Owners should insist that the semantic model of their building, the configuration of the optimisation and the logged data remain theirs and can be exported in open formats. This is the information chain once more: an owner that owns its information can change vendors; one that does not is locked in. Putting the evidence together, AI-driven control adds value beyond good conventional control in three main situations. The first is where predictable variation can be exploited: thermal mass, weather forecasts, occupancy patterns and time-varying electricity prices give a predictive controller room to shift and shape loads in ways a reactive controller cannot. The second is where the building is complex enough that the interactions between subsystems, such as chillers, cooling towers, pumps, air handlers and terminal boxes, offer optimisation opportunities that fixed rules miss. The third is where the grid increasingly rewards flexibility, as heating electrifies and the value of shifting demand by an hour or two rises. In each case, the intelligence is only as good as the information it works with. A predictive controller without an accurate system model predicts poorly. Without occupancy data, it plans for the wrong people. Without proper measurement afterwards, nobody can tell whether it worked. That final link, proof, is the subject of the last chapter. Chapter 8: Proving the Savings Every technology discussed in this booklet is sold on a promise of reduced energy use. Very few buyers are in a position to check whether the promise was kept. The difficulty is not dishonesty, although that exists, but a basic problem of evidence: energy savings are the difference between what a building consumed and what it would have consumed without the intervention, and the second quantity can never be observed. The building in the counterfactual world, with the same weather, the same occupancy and the same tenants, but without the new controls, does not exist. Everything depends on how it is estimated. This is the last link in the information chain, and it closes the loop. Measurement tells the owner whether the model, the sensors and the controls are doing what was claimed. It also feeds back into operation: a building whose savings are measured continuously can see when they start to fade and act before the losses accumulate. And increasingly, measurement has legal consequences, because regulators are beginning to judge buildings by what they actually use. Suppose an office building used 4.0 million kilowatt-hours of electricity last year and 3.5 million this year, after a supervisory AI system was installed. Did the system save 12.5 percent? Perhaps. But last summer may have been hotter than this one. Occupancy may have fallen as a tenant moved to hybrid working. A floor may have been vacant for six months. The chiller may have been replaced. Any of these could account for some or all of the difference, or could mask savings larger than the raw figures suggest. The discipline that deals with this problem is called measurement and verification. Its central tool is the adjusted baseline: a model, fitted to data from before the intervention, that predicts consumption from the factors that drive it, such as outdoor temperature, occupancy, operating hours and floor area in use. After the intervention, the baseline model is fed the actual conditions of the reporting period to predict what the building would have consumed had nothing changed. Savings are the difference between that prediction and what was actually measured. The most widely used framework is the International Performance Measurement and Verification Protocol, maintained by the Efficiency Valuation Organization. It defines four approaches, which differ in what is measured and where the boundary of the analysis is drawn. Table 3 summarises them. Table 3. The four measurement and verification options of the International Performance Measurement and Verification Protocol. Option Boundary What is measured Typical use A Retrofit isolation Key parameters only; others estimated Simple equipment changes, such as a lighting retrofit B Retrofit isolation All relevant parameters, often continuously A single system such as a chiller plant or fan C Whole facility Utility meters for the whole building Multiple interacting measures, controls programmes D Whole facility or system Calibrated computer simulation New buildings or where no baseline data exist Controls interventions usually fall under Option C, because they affect many systems at once and their effects interact. Changing supply air temperature changes fan energy, chiller energy and reheat energy together; isolating any one would miss the net effect. Option C has a well-known weakness: the whole-building meter records everything, including tenant equipment, lifts, kitchens and any other change in the building, so the savings must be large enough to stand out from that background noise. ASHRAE Guideline 14, the main American technical guidance on measuring savings, sets statistical criteria for how well a baseline model must fit before it can be relied on, and generally advises that whole-building approaches are suitable only when expected savings are a reasonably large fraction of total consumption. The availability of interval meter data, with readings every fifteen minutes or hour rather than once a month, has changed this field substantially. Baseline models fitted to hourly data, using techniques that range from piecewise regression on temperature to machine-learning models, can detect smaller savings with more confidence and in less time. Lawrence Berkeley National Laboratory and others have tested such automated methods against large datasets and made open-source tools available. A disciplined owner can now verify the savings claimed by a controls vendor with data the owner already has, provided the baseline period was recorded and the non-routine changes during the reporting period are documented. A worked example shows how the method changes the answer. Return to the office building that used 4.0 million kilowatt-hours last year and 3.5 million this year. Suppose the owner had fitted a baseline model to two years of hourly data from before the AI system was installed, relating consumption to outdoor temperature, hour of day, day of week and the number of floors leased. Feeding this year's actual weather and leasing into that model predicts 3.8 million kilowatt-hours: this year was milder and one floor was vacant for part of it, so consumption would have fallen anyway. The adjusted saving is therefore 0.3 million kilowatt-hours, about 8 percent, not the 12.5 percent the raw comparison suggested. If the vendor's guarantee was 10 percent, the difference determines who owes whom money. These numbers are hypothetical, but the shape of the problem is typical. Two further complications arise in practice. The first is non-routine events: changes during the reporting period that the baseline model does not capture, such as a new tenant installing a server room, a floor converted to a laboratory, or a major equipment replacement. These must be identified and adjusted for separately, and disputes over them are where many performance contracts go wrong. The second is attribution. If an owner installs occupancy sensors in March, upgrades sequences in June and adds a supervisory AI service in September, a whole-building analysis can measure the combined effect but cannot easily say which measure delivered what. Vendors naturally prefer to claim the whole. Owners who want to know what each step contributed need to stage their interventions, measure between them or meter the affected systems separately. What the evidence adds up to With these methods in mind, it is possible to read the evidence collected in earlier chapters more critically. Table 4 brings the main figures together. They are not directly comparable, because they cover different building types, interventions, baselines and methods, and the point of placing them side by side is precisely to show that difference. Table 4. Selected evidence on HVAC and controls savings in commercial buildings. Source Intervention Scope Reported result Type of evidence Mills, LBNL (2009) Commissioning of existing buildings 643 buildings in total 16% median whole-building Multi-site compilation Smart Energy Analytics Campaign, DOE and LBNL (2019) Fault detection and diagnostics software Nearly 6,000 buildings in total 9% median after two years Portfolio data PNNL field trial (2014) Advanced rooftop unit controls 66 units, 8 buildings 57% average of unit HVAC electricity Field measurement Zhang et al., LBNL (2022) ASHRAE Guideline 36 sequences Simulated VAV systems 31% average HVAC energy Simulation Kong et al., Applied Energy (2022) Occupancy-based HVAC control Side-by-side test spaces 17% to 24% weekly HVAC energy Controlled experiment BrainBox AI case study (2023) Supervisory AI control One 32-storey office 15.8% HVAC electricity Vendor-reported The pattern is consistent with the argument of this booklet. Large percentages come from single interventions measured against poor baselines, such as constant-speed rooftop fans, or from simulation against typical practice. Portfolio medians are smaller, because real portfolios include buildings that were already reasonably well run, interventions that were partly implemented, and findings that were never acted on. Savings from occupancy-based control and supervisory AI fall in between. Read together, the figures suggest that a well-run programme combining a reliable system model, occupancy sensing, good sequences and supervisory optimisation can reasonably target HVAC savings in the range of a fifth to a third in a typical existing office building, and much more in a badly run one. They do not support claims of drastic savings in buildings that were already well operated. Why savings fade Measured savings tend to decay. Lawrence Berkeley National Laboratory's commissioning research found that savings often persisted for several years, but other studies of retro-commissioning have documented a gradual return toward the earlier baseline. The mechanisms are familiar. An operator responding to a complaint widens a minimum airflow and never narrows it again. A controls technician fixes a fault by placing a point in manual override. A tenant fit-out adds new terminal boxes programmed with default settings. A sensor drifts and the reset logic that relies on it quietly stops working. None of these is dramatic, and each erodes savings a little. Continuous measurement is the only reliable defence. A building whose consumption is compared each week against its adjusted baseline, and whose controls are checked automatically against the intended sequences, can see degradation as it happens. This is the strongest argument for treating analytics, fault detection and measurement as permanent operating functions rather than one-off projects. It is also why autonomous supervisory systems have an advantage over recommendation-only software: they continually reassert the optimised settings, so a manual override is noticed and reversed rather than becoming permanent. But that advantage holds only if the supervisory system's own model of the building is kept current, which brings the argument back to Chapter 4. Measurement also guards against a subtler problem: optimising the measure rather than the outcome. A control system rewarded for reducing HVAC electricity can achieve its target by letting zones drift to the edge of comfort, by reducing ventilation below what occupants need, or by shifting load to systems that are not metered in the evaluation, such as a gas-fired boiler or a tenant's own cooling units. None of these is a genuine saving. A credible measurement programme therefore tracks comfort and indoor air quality alongside energy: zone temperatures against their bands, complaint logs, carbon dioxide levels in representative spaces and ventilation rates against requirements. Reporting energy alone invites gaming, intentional or not. The same discipline protects owners against the opposite risk, that of being blamed for failures that are not theirs. When an AI system is accused of causing discomfort, a record of what it commanded, what the zones experienced and what the occupancy was at the time allows the question to be settled with evidence rather than anecdote. In buildings where autonomous control is new, that evidence often decides whether the technology survives its first difficult summer. The operators who trust such systems are usually those who can see exactly what the system did and why. Continuous measurement also changes the relationship with tenants. A tenant that can see its own floor's consumption, adjusted for weather and occupancy, alongside comfort data, is in a position to participate: to agree wider temperature bands in exchange for lower service charges, to notify the building of planned low-occupancy days, or to accept pre-cooling during grid events. Without data, these conversations rest on trust; with it, they rest on shared evidence. Finally, measurement connects the individual building to its market. In the United States, many cities and several states require large buildings to report annual energy use, usually through the Environmental Protection Agency's ENERGY STAR Portfolio Manager, which normalises consumption for weather, building type and use and produces a score comparing the building with similar ones nationally. New York City's benchmarking law, Local Law 84, has required this of large buildings for more than a decade, and the reported scores are public. Benchmarking does not prove that any particular intervention worked, because it compares a building with its peers rather than with its own counterfactual. But it does make persistent underperformance visible to tenants, lenders and buyers, and it supplies the measured baseline on which the next generation of performance standards, including New York's, has been built. Regulation turns performance into liability For most of the history of commercial buildings, measured energy performance had few legal consequences. Building codes regulated design, energy certificates were usually based on calculated rather than measured performance, and the gap between them was nobody's legal problem. That is changing, and the change alters the economics of everything in this booklet. New York City's Local Law 97 is the most prominent example. It sets annual greenhouse gas emissions limits, based on measured energy use, for most buildings over 25,000 square feet, about 50,000 buildings in total. The first compliance period began in 2024; the limits tighten substantially in 2030, with the law aiming for a 40 percent reduction in emissions from the covered buildings by that year and 80 percent by 2050. Owners pay 268 dollars per tonne of carbon dioxide equivalent above their limit. City data reported in 2024 suggested that most covered buildings were on track for the first-period limits, but that a much smaller share already met the 2030 limits. For a large office building that exceeds its 2030 cap by a few thousand tonnes, the annual penalty runs to hundreds of thousands of dollars. Measured operational savings now have a direct, calculable value beyond the utility bill. The European Union's recast Energy Performance of Buildings Directive, adopted in 2024 and due to be transposed into national law by May 2026, works through several mechanisms at once. It requires member states to set minimum energy performance standards so that the worst-performing 16 percent of non-residential buildings are renovated by 2030 and the worst 26 percent by 2033. It requires new buildings to be zero-emission from 2030, and new public buildings from 2028. And it extends the requirement for building automation and control systems in non-residential buildings: what had applied to buildings with heating or air-conditioning systems above 290 kilowatts is to apply to systems above 70 kilowatts by the end of the decade. Those systems must be able to monitor, analyse and adjust energy use continuously and to benchmark the building's efficiency. Regulations of this kind change the question owners ask. Under a design-based regime, the question is whether the building was designed to comply. Under a performance-based regime, the question is whether the building actually performs, year after year, and whether the owner can prove it. The information chain, from model through sensors and control to measurement, becomes part of compliance rather than an optional efficiency investment. Who pays and who benefits The final obstacle to proving and capturing savings is commercial rather than technical. In much commercial real estate, the landlord pays for the building systems while tenants pay for the energy, either directly or through service charges. The landlord who invests in better controls may see no return; the tenant who would benefit has no control over the systems. This split incentive has slowed investment in building efficiency for decades. Performance-based regulation partly resolves it by placing the obligation, and the penalty, on the owner. Other mechanisms help. Green lease clauses can share both the costs and the savings of efficiency measures between landlord and tenant, and oblige tenants to share energy data. Energy performance contracts, in which a specialist company finances the improvements and is paid from measured savings, depend entirely on credible measurement and verification. Subscription-based AI control services, which often guarantee a minimum saving, are a new variant of the same idea. In each case the contract is only as good as the method used to measure the savings, and the owner should insist on a documented baseline, an agreed adjustment model, access to the underlying data and the right to independent verification. Measurement is therefore not an afterthought. It is the means by which a building learns whether the rest of the information chain is working, the means by which investment is justified and contracts are enforced, and increasingly the means by which a building demonstrates to a regulator that it complies with the law. Without it, the most sophisticated model and the most intelligent controller remain claims. Conclusion If commercial buildings waste energy because they are operated without information, then the order in which owners spend money on digital technology matters more than the choice of any particular product. That is the practical consequence of the argument made here, and it runs against the way PropTech is usually sold. The usual sales sequence starts at the visible end. A vendor demonstrates a dashboard, a three-dimensional twin with live values floating over it, or an AI engine that promises a fifth off the energy bill without capital works. The owner buys the visible layer and then discovers, during a long and expensive onboarding, that the building's systems must first be mapped, its point names decoded, its sensors checked and its sequences understood before the product can do anything. The cost of that discovery is often borne by the vendor in the first building and by the owner in every building after it, hidden inside subscription fees. Worse, the resulting system model usually belongs to the vendor, so the work is repeated when the owner changes supplier. Build the chain from the bottom A better sequence starts at the other end. The first investment is in knowing what the building is: a maintained, machine-readable model of its HVAC systems, with every piece of equipment identified, every control point linked to its equipment and every piece of equipment linked to the spaces it serves. For new buildings, this means specifying it in the owner's information requirements before design begins, under ISO 19650, and verifying it during commissioning. For existing buildings, it means a mapping exercise that combines the building automation system's point list, the drawings, the maintenance records and a site survey, assisted where useful by machine-learning tools that infer point types and relationships from data. The output should be expressed in an open vocabulary such as Brick or Haystack and owned outright by the owner. The second investment is in doing conventional control properly. With a system model in hand, the building's sequences can be compared against good practice, of which ASHRAE Guideline 36 is now the clearest codification, and the gaps closed: static pressure and supply air temperature resets, sensible minimum airflows, proper economiser logic, demand-controlled ventilation, schedules that reflect actual use, and systematic fault detection with a working process for acting on what it finds. The evidence reviewed here suggests that this step alone captures a large share of the achievable savings in most existing buildings, and it does so with techniques whose behaviour is predictable and whose maintenance does not depend on a single vendor. The third investment is in occupancy information, chosen to match the control decisions it will feed. Presence detection for standby modes, counts for ventilation, and historical patterns for start-up optimisation each have different requirements. Where workplace sensors already exist for space planning, connecting them to the HVAC controls is often the cheapest saving available. Privacy-preserving technologies are usually sufficient, and choosing them makes the system easier to deploy and keep. Only then does predictive or AI-driven supervisory control earn its place. On top of an accurate system model, good native sequences and reliable occupancy data, a predictive layer can exploit thermal mass, forecasts and time-varying prices to capture savings and grid flexibility that reactive control cannot. Without those foundations, much of what it does is to compensate, at subscription prices, for missing basics. Running alongside all four steps is measurement: an adjusted baseline established before any intervention, continuous comparison of actual consumption against it, and documentation of the changes that would otherwise confuse the comparison. Measurement is what tells an owner which step is delivering and when savings begin to fade. What owners should ask for This sequence implies a short list of demands that any owner can make of designers, contractors and technology vendors. From designers and contractors on new projects, owners should require that the handover include a machine-readable system model in an open format, verified during commissioning, alongside the record model and maintenance data. This is a contractual requirement that costs little if it is specified at the start and a great deal if it is reconstructed later. From controls contractors, owners should require sequences based on published high-performance practice, point naming that follows an agreed convention, and the right to see and change the programming of the systems they own. From analytics and AI vendors, owners should require that the system model, configuration and data remain the owner's property and be exportable in open formats; that every command the system issues be logged; that savings be measured against a documented baseline using a recognised protocol; and that network access be limited and secured. A vendor that resists these conditions is selling dependency rather than performance. From their own organisations, owners should require that someone is accountable for the system model and keeps it current through fit-outs, equipment replacements and control changes. This role does not yet appear on most organisation charts. It should. What changes for the people who run buildings The information chain also changes the work of building operators. The traditional chief engineer carried the building's information in memory: which valve leaked, which zone always complained, which override had been left in because of a tenant who left years ago. That knowledge was valuable and fragile, and it walked out of the door when the engineer retired. A building with a maintained system model, continuous fault detection and measured performance needs operators who can work with data as well as with equipment: who can interpret a fault report, judge whether an optimiser's proposed setpoint is sensible for a particular zone, and recognise when the model no longer matches the building. The skills are not exotic, but they are different, and the industry has been slow to train for them. Technology that removes the need for skilled operators is not on offer. What is on offer is technology that makes skilled operators far more effective, and that preserves their knowledge in a form that outlasts their tenure. What policy should do For policymakers, the argument suggests two things. The first is that performance-based regulation, of the kind New York and the European Union have adopted, does more to close the gap between designed and actual performance than any design-stage requirement, because it makes the gap the owner's problem. It also creates demand for exactly the information chain described here, since an owner facing a penalty for measured emissions needs to know what the building is doing and why. The second is that public procurement can set the standard for information handover. Governments are among the largest commissioners of buildings. If public clients consistently require machine-readable system models, verified at commissioning and expressed in open vocabularies, the supply chain will learn to produce them, and the cost will fall for everyone. The United Kingdom's use of central procurement to establish BIM practice in the 2010s showed that this approach can move an industry. The same lever could now be applied to the operational information that BIM handovers still leave out. The building that knows itself Commercial buildings will not become efficient because they acquire intelligence. They will become efficient when they know what they are, who is in them and what they are doing, and when that knowledge reaches the equipment that heats, cools and ventilates them. Building Information Modeling, occupancy sensing and AI-driven control are each a partial means to that end. None of them is sufficient alone, and the most sophisticated of them is the most dependent on the others. The engineering problem of heating and cooling a building efficiently was largely solved decades ago. What remains is an information problem: creating the right information during design, preserving it through handover, keeping it current through years of change, and connecting it to the controls. That problem is less glamorous than the technologies marketed to solve it, and solving it depends more on contracts, process and organisational discipline than on algorithms. It is also where the savings are. Hashtags: #PropTech #BuildingInformationModeling #BIM #SmartHVAC #SmartBuildings #BuildingAutomation #BuildingManagementSystems #BMS #DigitalTwins #IntelligentBuildings #AIInBuildings #HVACOptimization #EnergyEfficiency #BuildingEnergyManagement #SustainableBuildings #GreenBuildingTechnology #OccupancySensing #SmartSensors #DemandControlledVentilation #PredictiveMaintenance #FaultDetectionAndDiagnostics #ModelPredictiveControl #ReinforcementLearning #BuildingPerformance #EnergyAnalytics #BuildingInteroperability #FacilityManagement #SmartBuildingTechnology #CarbonEmissionReduction #CommercialRealEstateTechnology

Download the Book (PDF): Introduction Stand at the edge of a cornfield in early July and the crop looks uniform. The rows run green to the horizon, the canopy is closing, and nothing in the color or posture of the plants suggests that one corner of the field has been short of nitrogen for two weeks, that a strip along the drainage tile is waterlogged, or that a fungal infection has taken hold in a low spot where morning dew lingers. The human eye is a remarkable instrument, but it evolved to find ripe fruit and hidden predators, not to measure the internal state of a leaf. By the time a farmer walking the field can see yellowing, wilting or lesions, the plant has usually been under stress for days, sometimes weeks, and some of the yield has already been lost. A small aircraft flying overhead at 100 meters, carrying a camera that records light the eye cannot see, can often tell a different story. Healthy leaves reflect near-infrared light strongly and absorb red light hungrily. Stressed leaves do both a little less well, and they do it before they change visibly. Turn that difference into a number for every few centimeters of the field and a map appears: a picture not of how the crop looks but of how it is functioning. Pair that map with a second aircraft that can carry forty, seventy or a hundred liters of liquid and release it at a controlled rate over a precisely known patch of ground, and the possibility emerges of treating only the plants that need treatment, in the amount they need, at the moment they need it. That is the promise of precision agriculture drones, and it is a real one. It is also widely misunderstood. Much of what is written about agricultural drones falls into two camps. One treats the aircraft as the innovation and talks about flight times, tank sizes and autonomy. The other treats the colored map as the innovation and implies that a field of red and green pixels is, by itself, a diagnosis. Both miss the point. The argument of this book The controlling idea of this book is simple to state and demanding to practice. The value of an agricultural drone does not lie in the aircraft or in the image. It lies in closing a loop: measuring light reflected from the crop, converting that measurement into a trustworthy signal, diagnosing what the signal actually means on the ground, turning the diagnosis into a prescription, and applying that prescription with a machine whose dose and placement are known. Each link in the loop is governed by physics, biology or law, and each link has limits. The environmental and economic benefits that people attribute to drones, less chemical, less runoff, better yields, appear only when every link holds. A beautiful vegetation map attached to a careless spray job delivers nothing. A superbly calibrated spray drone driven by a map nobody checked delivers precisely the wrong thing. Seen this way, the subject of the book becomes clearer. It is not a buyer's guide, though it discusses real equipment. It is not a remote sensing textbook, though it explains the physics that remote sensing rests on. It is an account of how the loop works, where it breaks, and what it takes to close it well. How the book is built The chapters follow the loop in order. The first chapter explains why leaves reflect light the way they do: what chlorophyll absorbs, why the internal structure of a leaf scatters near-infrared light so efficiently, what the red edge is, and why water and temperature carry information of their own. Everything else in the book depends on these mechanisms, and a reader who understands them will not be fooled by a map. The second chapter follows light into the camera. It covers the multispectral sensors actually flown on farms today, how raw pixel values become reflectance through radiometric calibration, why the sun, the clouds and the time of day matter so much, and how dozens or hundreds of overlapping images are stitched into a single map. Most of the bad data in agricultural remote sensing is produced here, quietly, by people who skipped a step. The third chapter is about reading the result. The Normalized Difference Vegetation Index, NDVI, is the most widely used number in agricultural remote sensing, and it deserves both its fame and a great deal of skepticism. The chapter explains what NDVI measures, why it saturates in dense canopies, how soil and shadow distort it, and which other indices, built on the red edge or adjusted for soil, do a better job in particular situations. The fourth chapter takes on the most compelling claim in the field: that drones can detect crop stress before it is visible. Sometimes they can, and the chapter explains the mechanisms, from leaf temperature rising when stomata close to subtle shifts in the red edge when chlorophyll begins to decline. It also explains why early detection is not the same thing as early diagnosis, and why the farmer's boots and the agronomist's tissue test remain part of the loop. The fifth chapter turns maps into decisions. It explains management zones, prescription maps and the logic of variable rate application, including how sensor-based nitrogen recommendations work and why reference strips matter. A worked example follows the arithmetic of a prescription through to the quantity of product saved or redistributed. The sixth chapter is about the spray drone as a machine. It explains rotor downwash, swath width, droplet size, rotary atomizers and hydraulic nozzles, low carrier volumes, battery logistics and the real work rate of a drone in the field, with an honest comparison against ground sprayers and crewed aircraft. The seventh chapter is the environmental ledger. It covers the physics of drift and the pathways of runoff, the evidence from field trials on how much drone sprays drift, and the real source of chemical savings: not the aircraft itself but the shift from broadcast application to targeted and spot application. It also addresses the risks that targeted spraying introduces. The eighth chapter deals with rules, markets and money. It describes how the United States, the European Union, Canada and China regulate drone spraying, how trade and security policy reshaped the American market in 2025, and what the economics of ownership and contracting look like for a working farm. The conclusion draws out what follows from the whole argument: what farmers, advisers, regulators and manufacturers should do differently if the loop, rather than the aircraft, is the unit of value. A note on scope Precision agriculture is a large field. It includes yield monitors, soil sampling grids, satellite imagery, autonomous tractors and camera-guided ground sprayers. This book refers to all of these where they illuminate the drone story, and satellites in particular appear often because they are the drone's most important competitor and complement. But the focus throughout is on the two things drones distinctly do: imaging crops from low altitude with multispectral and thermal sensors, and applying liquids and granules from the air in a controlled way. The examples come mostly from row crops, orchards, vineyards and rice, where most of the commercial activity and most of the research has taken place. Specific products, regulations and market figures are given as they stood in late 2026. They will change. The physics of a leaf, the logic of calibration and the discipline of the loop will not. Chapter 1: Light and Leaves Every vegetation map ever produced by a drone, a satellite or a handheld sensor rests on a handful of physical facts about how a green leaf handles sunlight. These facts are not difficult, but they are frequently skipped, and skipping them is how people come to believe that a red patch on an NDVI map means "disease" or that a green patch means "healthy." A leaf is an optical object with a structure, a chemistry and a temperature, and each of these leaves its own signature in the light that comes back to the sensor. Understanding those signatures is the first link in the loop. What a leaf does to sunlight Sunlight arriving at the top of a crop canopy covers a broad spectrum. The portion that matters most for agricultural sensing runs from about 400 nanometers, deep violet, through the visible colors to about 700 nanometers, deep red, and then on into the near-infrared up to roughly 1,300 nanometers, beyond which lie the shortwave infrared bands and, much further out, the thermal infrared at around 8,000 to 14,000 nanometers. When this light reaches a leaf, three things can happen to any given photon: it can be absorbed, reflected or transmitted through the leaf to the leaves and soil below. The proportions differ enormously by wavelength, and those differences are the raw material of remote sensing. In the visible range, a healthy leaf is a very efficient absorber. Chlorophyll a and chlorophyll b, the pigments that drive photosynthesis, absorb strongly in the blue around 430 to 450 nanometers and in the red around 640 to 680 nanometers. Carotenoids add further absorption in the blue and blue-green. What chlorophyll absorbs less efficiently is green light around 550 nanometers, and that is why leaves look green: a modest fraction of green light, typically on the order of 10 to 20 percent, is reflected, while red reflectance from a dense healthy leaf is often only a few percent. The eye sees the green peak; it has no way of registering how deep the red trough is. In the near-infrared, the story is completely different. Chlorophyll does not absorb near-infrared light, and neither does much else in a living leaf. A plant has no use for these photons, which carry too little energy per photon to drive photosynthesis but would heat the leaf if absorbed. Instead, near-infrared light is scattered. Inside a typical broadleaf, below the upper epidermis, lie the palisade cells and then the spongy mesophyll, a loose arrangement of cells separated by air spaces. Every time light crosses a boundary between a hydrated cell wall and an air space, it changes refractive index, from roughly 1.4 to 1.0, and part of it is bent or reflected. After many such encounters, a large share of the near-infrared light emerges back out of the leaf. A single healthy leaf may reflect 40 to 50 percent of incident near-infrared light and transmit most of the rest. The jump from low red reflectance to high near-infrared reflectance happens over a narrow span of wavelengths, roughly 680 to 750 nanometers. This steep rise is called the red edge, and it is one of the most information-rich features in the plant spectrum. Its position and steepness depend on how much chlorophyll the leaf holds. A leaf rich in chlorophyll absorbs further into the long-wavelength red, pushing the inflection point of the edge toward longer wavelengths. As chlorophyll declines, from nitrogen shortage, senescence or disease, the absorption feature narrows and the red edge shifts toward shorter wavelengths, sometimes by 10 or 20 nanometers. That shift is small, invisible to the eye, and measurable with the right instrument. Beyond about 1,300 nanometers, in the shortwave infrared, water dominates. Liquid water in leaf tissue absorbs strongly at around 1,450 and 1,940 nanometers and more weakly at 970 and 1,200 nanometers. A leaf that is losing water reflects more in these bands. Most drone cameras used on farms do not record shortwave infrared, because the detectors are expensive and heavy, but the water bands explain why some satellite missions and research instruments carry them, and why water stress is often detected on drones by a different route: temperature. From a leaf to a canopy A sensor flying over a field does not see a single leaf. It sees a canopy: many layers of leaves at different angles, stems, shadows, flowers or tassels, patches of soil between rows, crop residue, weeds. The reflectance recorded in each pixel is a blend of all of these, weighted by how much of the pixel each occupies and how brightly each is lit. The canopy amplifies the near-infrared signal. Because a leaf transmits much of the near-infrared light it does not reflect, that light passes to the leaf below, which again reflects and transmits a large share, and so on. Some of the light reflected from lower layers passes back up through the upper leaves and out of the canopy. The more leaf layers there are, the more near-infrared light returns, up to a limit. Red light behaves in the opposite way: the top leaves absorb most of it, so adding more leaves underneath changes red reflectance very little once the first layer or two are in place. This asymmetry is crucial. Near-infrared reflectance of the canopy keeps climbing as leaf area increases, while red reflectance bottoms out early. Agronomists measure leaf area with the leaf area index, LAI: the total one-sided area of leaves per unit of ground area. An LAI of 1 means there is, on average, one square meter of leaf above each square meter of soil; a mature maize or wheat crop commonly reaches an LAI of 3 to 6. At low LAI, early in the season, much of what the sensor sees is soil, and the canopy signal is diluted. At high LAI, the canopy is optically thick and further increases in leaf area make progressively less difference to the measured signal. This is the physical root of the saturation problem that limits NDVI, which the third chapter takes up in detail. Soil itself has a characteristic signature. Bare mineral soil reflects more and more as wavelength increases through the visible and near-infrared, without the sharp red trough or near-infrared plateau of vegetation. Wet soil is darker than dry soil across the spectrum. Organic-rich soil is darker than sandy soil. Crop residue left on the surface after harvest often looks spectrally similar to dry, bright soil. These differences matter because, early in the season and in row crops with wide spacing, soil contributes a large share of every pixel. A map of a young crop that looks patchy may be showing differences in soil moisture or color rather than differences in the plants. A small calculation shows how strongly mixing shapes the measurement. Take a young crop whose leaves, seen alone, reflect 4 percent of red light and 45 percent of near-infrared, over a dry soil reflecting 20 percent of red and 28 percent of near-infrared. If the canopy covers 60 percent of a pixel, the pixel's red reflectance is a weighted average, 0.6 times 4 plus 0.4 times 20, or 10.4 percent; its near-infrared is 0.6 times 45 plus 0.4 times 28, or 38.2 percent. If the soil is wet and darker, reflecting say 10 percent of red and 15 percent of near-infrared, the same crop gives a pixel with 6.4 percent red and 33 percent near-infrared. The plants are identical, but the pixel's red reflectance has fallen by more than a third and its near-infrared by about a seventh, simply because it rained. Any index built from these two bands will move accordingly. The lesson is not that early-season imagery is useless, but that it measures the crop and the soil together, and the two can only be separated by thinking about both. Geometry matters too. Leaves are not flat mirrors, and canopies are not flat surfaces. The amount of light reflected toward the sensor depends on the angle of the sun and the angle of view, a relationship that remote sensing scientists describe with the bidirectional reflectance distribution function. In practical terms, a canopy viewed with the sun behind the sensor looks brighter, because the camera sees the sunlit faces of leaves and few shadows; the bright region opposite the sun is called the hotspot. Viewed toward the sun, the same canopy looks darker, with more shadow in each pixel. A drone camera with a wide field of view records both conditions in a single frame, which is one reason why vegetation maps can show bands of brightness that follow the flight lines rather than anything in the field. Early signals: pigments and heat Chlorophyll is the dominant pigment, but not the only one, and some of the most useful early warning signals come from the others. Carotenoids, including the xanthophylls, play a protective role. When a leaf absorbs more light than its photosynthetic machinery can use, as happens under drought, heat or high light, the excess energy can damage the photosynthetic apparatus. Plants manage this by converting one xanthophyll, violaxanthin, into another, zeaxanthin, which helps dissipate excess energy as heat. This xanthophyll cycle operates within minutes and causes a small change in reflectance near 531 nanometers. In 1992, John Gamon and colleagues proposed the Photochemical Reflectance Index, which compares reflectance at 531 nanometers with a reference band at 570 nanometers to track this change. Because it responds to the efficiency with which a leaf is using light rather than to the amount of chlorophyll, it can in principle signal stress before any pigment is lost. In practice, the signal is small, it is easily confounded by canopy structure and illumination, and it needs narrow spectral bands that most commercial drone cameras do not provide. It remains largely a research tool, but it illustrates a general principle: the earliest signals of stress are physiological and transient, and they are subtle. Anthocyanins, the red and purple pigments, accumulate in some species under nitrogen or phosphorus deficiency, cold or other stresses. Phosphorus-deficient maize seedlings, for example, often show purpling. These pigments absorb in the green, around 550 nanometers, and alter the shape of the visible spectrum in ways that some indices try to exploit. Finally, there is fluorescence. A small fraction of the light absorbed by chlorophyll, on the order of one to two percent, is re-emitted as red and far-red light, peaking around 685 and 740 nanometers. This solar-induced fluorescence is tightly linked to photosynthetic activity. It can be measured from satellites and specialized airborne instruments by exploiting narrow dark lines in the solar spectrum, but it is too faint to separate from reflected light with ordinary multispectral cameras. It belongs, for now, in the research literature rather than in the farm toolkit. The last physical mechanism is not about reflection at all. It is about temperature. A transpiring leaf cools itself in the same way a sweating person does. Water evaporates from the moist surfaces inside the leaf and escapes through the stomata, the adjustable pores on the leaf surface, and each gram of water evaporated carries away roughly 2,400 joules of heat. A well-watered crop under strong sun can therefore run several degrees cooler than the surrounding air. When a plant runs short of water, it closes its stomata to conserve what it has. Transpiration falls, evaporative cooling falls, and leaf temperature rises, often within hours. Stomatal closure is also triggered by root damage, vascular diseases that block water transport, and some pests. In each case the leaf may still be green, still full of chlorophyll, and still reflecting near-infrared light normally, yet it is warmer than it should be. A thermal camera, recording emitted radiation in the 8,000 to 14,000 nanometer range, can see that warmth. Turning leaf temperature into a stress indicator requires context, because temperature also depends on air temperature, humidity, wind and sunlight. In the early 1980s, Sherwood Idso, Ray Jackson and colleagues at the United States Department of Agriculture developed the Crop Water Stress Index, which places the measured canopy temperature on a scale between two reference points: the temperature a fully transpiring crop would have under the same conditions, and the temperature of a crop that is not transpiring at all. A value near zero means no water stress; a value near one means transpiration has effectively stopped. Modern drone practice often approximates the reference points with wet and dry artificial surfaces placed in the field, or by using the coolest and warmest crop pixels in the image, but the logic is the same. Thermal sensing has distinctive limits. Thermal cameras that drones can carry have much lower resolution than visible or multispectral cameras, typically 640 by 512 pixels or fewer. Their absolute accuracy is often quoted at around plus or minus two degrees Celsius, or a few percent of reading, which is coarse compared with the differences being measured. And soil, especially dry sunlit soil, can be far hotter than any leaf, so pixels that mix soil and canopy can produce misleading temperatures unless the soil is masked out. Thermal data is powerful for irrigation management and for detecting conditions that disrupt water flow through the plant, but it demands careful handling. What the physics implies Several practical conclusions follow from these mechanisms, and they shape everything that comes later in the book. First, the strongest and most robust signal of green vegetation is the contrast between red absorption and near-infrared scattering. This is why the overwhelming majority of agricultural indices use red and near-infrared bands, and why a camera with only these two bands can already produce useful maps. Second, that contrast measures mainly how much green, light-absorbing leaf is present: leaf area combined with chlorophyll content. It does not, by itself, distinguish between causes. A patch of crop with low chlorophyll because of nitrogen deficiency, a patch with low leaf area because of compacted soil, a patch thinned by poor emergence and a patch damaged by hail can all produce similar values. The signal is a symptom. Third, the red edge carries information about chlorophyll concentration that remains sensitive after red absorption has saturated. This is why cameras with a red-edge band have become standard for serious agronomic work, and why indices using the red edge are often preferred in dense canopies. Fourth, the earliest physiological responses to stress, stomatal closure and changes in the xanthophyll cycle, show up in temperature and in narrow-band reflectance features before they show up in the broad red and near-infrared contrast. Detecting stress before it becomes visible therefore depends on choosing the right sensor for the right stress, and on understanding that early signals are smaller and noisier than late ones. Fifth, every measurement is a mixture of leaf, canopy, soil, shadow and geometry. The same field, flown at a different hour, under different cloud, or at a different growth stage, will produce different numbers even if nothing about the crop has changed. Making those numbers comparable is the work of calibration, and it is the subject of the next chapter. Chapter 2: From Photons to Maps A multispectral camera does not measure reflectance. It measures how many electrons accumulated in each pixel of a silicon detector during a fraction of a second, through a filter, behind a lens, under whatever light happened to be falling on the field at that moment. Converting those counts into a number that describes the crop, and that can be compared with another number taken next week or in the next field, is a chain of corrections. Each step is routine; each is easily skipped; and skipping any of them produces maps that look authoritative and mean little. This chapter follows light from the field to the finished map. The cameras farms actually fly Agricultural multispectral cameras are built around a simple design choice: instead of recording a continuous spectrum, they record a few carefully chosen bands, each through its own filter and usually its own lens and sensor. This keeps them light, cheap and fast enough for a small drone, while capturing the spectral features that matter most for vegetation. Two cameras illustrate the current mainstream. The DJI Mavic 3 Multispectral, a compact integrated drone, carries four multispectral bands, green at 560 nanometers, red at 650, red edge at 730 and near-infrared at 860, each recorded on a 5-megapixel sensor, alongside an ordinary color camera. It has an onboard sunlight sensor that records incoming light for each image, and an RTK positioning module for centimeter-level location. The manufacturer quotes coverage of up to 200 hectares in a single flight of about 43 minutes. The MicaSense RedEdge-P, a separate camera that mounts on a larger drone, records five bands, blue at 475 nanometers, green at 560, red at 668, red edge at 717 and near-infrared at 842, plus a higher-resolution panchromatic band used to sharpen the others. Its five spectral bands give a ground pixel of about 7.7 centimeters at 120 meters altitude; the panchromatic band halves that. The choice of band centers and widths is not cosmetic. A red band centered near 668 nanometers with a 14-nanometer width sits squarely in the chlorophyll absorption trough. A wider or shifted band would mix in light from the steep red edge and blunt the contrast. The red-edge band sits on the slope itself, so it responds to the edge's position. Different manufacturers place it differently, 717 in one case, 730 in another, which means red-edge indices computed from different cameras are not directly interchangeable. The same is true, to a lesser degree, for the red and near-infrared bands. An NDVI value from one camera is not the same quantity as an NDVI value from another, and switching instruments mid-season, or comparing maps from different service providers, introduces differences that have nothing to do with the crop. There are cheaper options. Some growers fit ordinary consumer cameras with modified filters so that one channel records near-infrared light. These can show relative differences within a single image, which is sometimes enough to spot a planter skip or a drainage problem, but the bands are broad and overlapping, and the results cannot be calibrated reliably. At the other end, hyperspectral cameras record hundreds of narrow, contiguous bands and can resolve features such as the 531-nanometer xanthophyll signal, but they are expensive, heavy and generate enormous data volumes. They are research instruments that occasionally move into high-value specialty crops. It is worth placing drones beside satellites, because the comparison determines when a drone is worth flying at all. Table 1 compares representative platforms on the attributes that matter for crop monitoring. Table 1. Representative imaging platforms for crop monitoring (manufacturer and agency specifications). Platform Spectral bands relevant to crops Typical ground pixel Revisit Main limitation DJI Mavic 3 Multispectral (drone) Green, red, red edge, NIR A few cm, set by altitude On demand Area per day; operator time MicaSense RedEdge-P (drone payload) Blue, green, red, red edge, NIR, pan 7.7 cm at 120 m (2 cm sharpened at 60 m) On demand Needs larger drone; data volume Sentinel-2 (ESA satellites) 13 bands incl. three red edge 10 m (visible, NIR); 20 m (red edge) About 5 days Clouds; field-scale only Landsat 8 and 9 (USGS/NASA) Visible, NIR, SWIR, thermal 30 m; thermal 100 m About 8 days combined Coarse for within-field work PlanetScope (commercial) Up to 8 bands incl. red edge About 3 m Near daily Cost; cross-sensor consistency The table explains the division of labor that has emerged on well-run farms. Satellites, especially the free Sentinel-2 data, provide frequent, cheap, consistent views of whole fields at a resolution adequate for spotting zones of tens of meters. They fail under cloud, which can blank out critical weeks in humid regions, and they cannot see individual plants, rows or small patches. Drones fill the gaps: they fly beneath clouds, on the day they are needed, at a resolution that resolves rows, plants and weeds. They pay for that with limited coverage per hour and with all the calibration work that a satellite agency does centrally and a drone operator must do personally. Calibration: turning counts into reflectance The quantity that describes the crop is reflectance: the fraction of incoming light, in a given band, that the surface reflects. A camera records something proportional to reflected radiance, which depends on both the surface and the incoming light. If a cloud passes over the sun, the camera's counts drop, though the crop has not changed. Calibration is the process of dividing out the light. The first step is internal. Each camera has its own sensor response, lens vignetting that darkens the corners of each frame, dark current that adds counts even with no light, and exposure settings that change from image to image. Manufacturers supply models for these effects, and processing software applies them automatically when the metadata is intact. This step converts raw counts into a quantity proportional to radiance. The second step deals with illumination. There are two common methods, and good practice uses both. The first is a calibrated reflectance panel: a flat target of known, stable reflectance in each band, photographed on the ground immediately before and after the flight. The software compares the panel's known reflectance with what the camera recorded and computes a conversion for each band. The second is a downwelling light sensor, a small upward-facing sensor on top of the drone that records incoming light for every image. If a cloud dims the sun halfway through a flight, the light sensor registers the change, and the software scales each image accordingly. Each method has weaknesses. The panel captures conditions at the moment it is photographed, not twenty minutes later. It must be clean, held level, not shaded by the operator, and photographed at the right distance. A panel that has been scuffed, left in the sun for two seasons or splashed with mud no longer has its certified reflectance. The light sensor, for its part, tilts with the aircraft as it turns and banks, and on partly cloudy days it measures light from a patch of sky that may not match the light falling on the part of the field being imaged. Under rapidly changing broken cloud, neither method fully rescues the data, and the honest answer is often to fly another day. Timing matters as much as equipment. Near solar noon, the sun is high, shadows are short and illumination changes slowly. Early morning and late afternoon flights produce long shadows within the canopy, stronger directional effects and faster changes in light. A common rule of thumb in agronomic practice is to fly within about two hours of solar noon, on clear or uniformly overcast days, with consistent timing across repeated flights. Uniform overcast is better than scattered cumulus, because the light is diffuse and stable, even though it is dimmer. A useful test of a calibration workflow is to look at areas that should not change: a gravel road, a concrete pad, a patch of bare soil that stayed dry. If the reflectance of these features jumps between flights, the crop values will have jumped too, and any apparent change in the crop must be treated with suspicion. Flight planning and the geometry of capture The other half of producing a map is geometry. A drone flying at 120 meters with a camera of typical field of view photographs a footprint perhaps 80 to 100 meters across. To build a seamless map of a 50-hectare field, it must take hundreds of images in parallel flight lines, and they must overlap. Overlap serves two purposes. It allows the stitching software to find the same features in neighboring images and work out exactly where the camera was and which way it was pointing for each one. And it means each point on the ground is seen in several images, so the software can choose the view closest to vertical, reducing directional effects. For crop canopies, which are visually repetitive and move in the wind, matching features is harder than for buildings or roads. Agricultural mapping therefore commonly uses forward overlap of 75 to 85 percent and side overlap of 70 to 80 percent, higher than would be needed over a city. Ground sample distance, the size of each pixel on the ground, is set by altitude, sensor pixel size and focal length. Halving the altitude halves the pixel size and quadruples the number of images needed to cover the same area, along with flight time and processing time. The right resolution depends on the question. Mapping nitrogen variability across a cornfield needs pixels of perhaps 10 to 50 centimeters; finer detail adds noise from individual leaves and shadows rather than useful information. Counting plants or distinguishing small weeds from crop seedlings needs pixels of one or two centimeters. Flying lower than the question requires is one of the most common and costly inefficiencies in drone work. Flight speed interacts with exposure. A drone moving at 10 meters per second with an exposure of one-five-hundredth of a second moves two centimeters during the exposure, enough to blur a two-centimeter pixel. Under dim light, cameras lengthen exposure, and blur increases. Some cameras use global shutters, which capture the whole frame at once; others use rolling shutters, which read out line by line and can distort images of a moving scene. Multispectral cameras with separate lenses for each band also have parallax: each band views the field from a slightly different position, and the software must align them precisely, or the bands will be offset and every computed index will be wrong along edges. Stitching, orthomosaics and their artifacts The processing software, whether a commercial package such as Pix4Dfields, DJI Terra or Agisoft Metashape, or a cloud service, carries out a sequence of steps. It identifies matching features across images, solves for each camera position, builds a three-dimensional model of the surface, projects each image onto that surface so that every pixel lands in its correct map location, and blends the overlapping images into a single seamless mosaic called an orthomosaic. It then applies the radiometric calibration and computes indices band by band. Each step can leave fingerprints in the final map. Where feature matching fails, often in uniform canopy or along field edges, the mosaic may show smeared regions or misplaced patches. Where blending switches from one image to another, a seam can appear, especially if the light changed between the two images. Where directional reflectance effects are strong, the map can show stripes parallel to the flight lines, alternately brighter and darker. Where the wind moved leaves between images, fine detail blurs. None of these are crop signals, but all can look like crop signals to an untrained reader. Positioning accuracy matters most when maps are compared over time or used to direct a sprayer. A drone using ordinary satellite positioning may place its map with errors of a meter or more. That is fine for spotting a 30-meter patch of stress but useless for directing spot treatment of individual weeds or for comparing two flights pixel by pixel. RTK positioning, which corrects satellite signals using a nearby base station or a correction network, brings errors down to a few centimeters. Ground control points, marked targets surveyed into the field, do the same job when RTK is unavailable. The interaction of these choices is easiest to see in a single example. Consider a rectangular 60-hectare maize field, 1,000 meters long and 600 meters wide, to be mapped at mid-season for nitrogen variability with a five-band camera that gives a ground pixel of about 7.7 centimeters at 120 meters. The question is zone-scale: differences across tens of meters. Pixels of 8 centimeters are already far finer than the question needs, so there is no reason to fly lower than 120 meters, which in most jurisdictions is also the default ceiling for small drones. Suppose the camera's footprint at that height is about 90 meters across the flight direction and 75 meters along it. With 75 percent side overlap, adjacent flight lines are spaced a quarter of 90 meters apart, about 22.5 meters. Covering a 600-meter width therefore takes about 27 lines, each 1,000 meters long, for some 27 kilometers of flying, plus turns. With 80 percent forward overlap, the camera must fire every 15 meters along each line, about 67 captures per line or roughly 1,800 capture events in total, each producing one image per band, or around 9,000 to 11,000 individual image files once the panchromatic band is included. At 10 meters per second, the 27 kilometers of lines take about 45 minutes, plus turning time. On a mid-size drone that may mean two battery changes, each a moment when light can change between the first and second halves of the field. The operator photographs the reflectance panel before takeoff, after landing, and ideally at each battery swap. Processing several gigabytes of images on a capable workstation or in the cloud takes hours rather than minutes. Now consider the cost of flying lower to obtain finer pixels. At 60 meters, the footprint halves in each direction, the number of flight lines roughly doubles, the spacing between captures halves, and the total number of images roughly quadruples. Flight time doubles to about an hour and a half or more, and processing time grows faster still. For a nitrogen question, none of that extra detail adds information; individual leaves and shadows simply introduce more variability that the analyst must average away. For a weed-mapping question in young soybeans, by contrast, even 60 meters may be too high, and the analyst would fly a few test patches at 20 to 30 meters rather than the whole field. The example shows why experienced operators begin flight planning with the question, not with the camera's best resolution. It also shows why consistent capture conditions are hard to achieve over large areas: the longer a mission runs, the more the sun moves and the more chance clouds have to drift across, and every battery swap is a seam in time as well as in space. The checklist that emerges is not glamorous, but it is the difference between data and decoration. A sensor appropriate to the question, with known band positions. Radiometric calibration with a clean panel and a working light sensor. Flights near solar noon under stable light, at consistent times across the season. Altitude and overlap chosen for the resolution the question needs, not the finest the camera can deliver. Accurate positioning when maps will be compared or used for application. And a visual check of the result for seams, stripes and smears before anyone draws conclusions from it. With a map that meets those conditions, the next step is to read it, and that means confronting the strengths and limits of NDVI. Chapter 3: Reading the Map The Normalized Difference Vegetation Index is the most familiar number in agricultural remote sensing. It appears on farm management apps, in satellite dashboards, in drone service reports and in thousands of research papers. It is genuinely useful. It is also routinely asked to do things it cannot do. Reading an NDVI map well means knowing exactly what the number measures, how it behaves across the life of a crop, how it is distorted, and when a different index is the better tool. What NDVI is and where it came from NDVI is calculated from two reflectance values, near-infrared and red, as their difference divided by their sum. Written out, it is near-infrared minus red, over near-infrared plus red. Because reflectance values are fractions between zero and one, NDVI ranges from minus one to plus one. Water, which reflects more red than near-infrared, gives negative values. Bare soil typically gives values around 0.1 to 0.2. Sparse or stressed vegetation falls in the 0.2 to 0.5 range, and dense, healthy green canopies commonly reach 0.8 to 0.9. A simple calculation shows why. Suppose a healthy maize canopy reflects 45 percent of near-infrared light and 5 percent of red. The difference is 0.40, the sum is 0.50, and NDVI is 0.80. Now suppose a patch of bare, dry soil reflects 25 percent of near-infrared and 18 percent of red. The difference is 0.07, the sum is 0.43, and NDVI is about 0.16. A pixel that is half canopy and half soil will fall somewhere between, though not exactly at the midpoint, because NDVI is a ratio rather than a simple average. The index was developed in the early 1970s by John Rouse and colleagues at Texas A&M University, working with data from the first Landsat satellite, then called ERTS-1, to monitor vegetation across the Great Plains. The normalization, dividing the difference by the sum, was the key design decision. It partly cancels out the effect of overall brightness. If thin haze or a slight change in sun angle dims both bands by the same fraction, the ratio is unchanged. That robustness is why the index spread from satellites to aircraft to drones to handheld sensors and became a common language. Physically, NDVI responds to the amount of green, photosynthetically active leaf in the pixel: some combination of how much leaf area there is, how much chlorophyll the leaves hold and how much of the pixel the canopy covers. In the early season, it is dominated by canopy cover, essentially the fraction of the ground hidden by green leaves. As the canopy closes, it increasingly reflects leaf area and chlorophyll. Because these quantities are all linked to the crop's capacity to capture light and grow, NDVI often correlates with biomass and, at certain growth stages, with final yield. What NDVI does not track is cause. A low value in one part of a field says that there is less green leaf there than elsewhere. It does not say why. The possible explanations include nutrient deficiency, water stress, waterlogging, soil compaction, poor seed placement, low plant population, disease, insect damage, herbicide injury, hail, a different hybrid planted in that strip, or simply that the crop there was sown a week later. Each of these calls for a completely different response, and several call for no response at all. An NDVI map is a map of where to look, not a map of what to do. The relationship with yield deserves a word of its own, because it is the most commonly overstated. NDVI correlates with final yield most strongly at particular growth stages, typically when the canopy is approaching full cover but before it saturates, and in seasons and fields where the factors limiting yield are already expressed in the canopy by that time. If a drought arrives during grain fill, after the imagery was taken, the mid-season map will have overestimated yield in exactly the areas that suffer most. If a disease strikes late, the same. Correlations reported from one field and season, sometimes strong, often weaken when applied to the next. An NDVI map is therefore best treated as a description of the canopy at a moment, from which yield potential can be estimated with an uncertainty that should be stated, rather than as a forecast. There is also a practical objection worth answering: if NDVI has so many limitations, why does anyone still use it? The answer is that its limitations are well understood, its two bands are available on almost every sensor, from cheap drones to satellites with half a century of archives, and for the most common early-season questions, where is the stand poor, where is the canopy lagging, which parts of the field differ from the rest, it works well. It is a good general-purpose screen. The mistake is not to use it but to use it beyond the conditions in which it is reliable. Treating the map as a guide to where to look, rather than as an instruction about what to do, is the single most important habit in drone agronomy. Many disappointing experiences with drone imagery come from treating a vegetation index map as a prescription rather than as a scouting guide. Saturation and the index family The most serious technical limitation of NDVI is saturation. As a canopy becomes denser, red reflectance falls quickly toward a floor of a few percent, because the upper leaves absorb almost all of it. Near-infrared reflectance keeps rising, but because NDVI divides the difference by the sum, further increases in near-infrared move the index less and less. Once leaf area index exceeds roughly 3, NDVI changes very little even as biomass continues to accumulate. The arithmetic makes this vivid. Take a canopy with red reflectance of 4 percent and near-infrared of 40 percent: NDVI is 0.36 over 0.44, about 0.82. Now increase near-infrared by a quarter to 50 percent while red stays at 4 percent: NDVI becomes 0.46 over 0.54, about 0.85. A large difference in canopy reflectance has produced a change of 0.03 in the index, smaller than the variation that calibration errors can introduce from flight to flight. The consequence is that in the mid to late season of high-yielding crops, exactly when nitrogen management and disease decisions are often made, NDVI maps of the healthy majority of a field can look uniformly dark green. Real differences in nitrogen status, which matter for yield, are compressed into a narrow band of index values. Drone operators sometimes compensate by stretching the color scale so that the narrow range fills the full palette, which makes small differences visible but also amplifies noise and calibration artifacts. A map stretched in this way can look dramatic while showing differences of a few hundredths of an index unit. Remote sensing scientists have developed dozens of alternative indices, each designed to address a particular weakness of NDVI. A handful have earned a place in practical agronomy. Table 2 summarizes them. Table 2. Vegetation indices commonly used in drone agronomy (formulas as published by their originators). Index Bands used Designed to Best used for Main caution NDVI Red, NIR Measure green leaf amount Early to mid season; canopy cover; biomass zones Saturates above LAI of about 3 NDRE Red edge, NIR Track chlorophyll in dense canopy Mid to late season nitrogen status Values depend on red-edge band position GNDVI Green, NIR Increase chlorophyll sensitivity Chlorophyll and nitrogen in moderate canopy Also responds to leaf area SAVI and OSAVI Red, NIR, soil factor Reduce soil background effect Young crops; wide rows; orchards Soil factor is an assumption CWSI (thermal) Canopy and air temperature Measure transpiration deficit Irrigation scheduling; water stress Needs references and soil masking The Normalized Difference Red Edge index, NDRE, replaces the red band with a red-edge band. Because red-edge light is not absorbed as completely as red light by the upper canopy, it continues to respond to changes in chlorophyll content after red reflectance has bottomed out. Research by Edward Barnes and colleagues around 2000, and much work since, has shown that red-edge indices track leaf chlorophyll and nitrogen status in dense canopies better than NDVI. For mid-season nitrogen decisions in maize, wheat and cotton, NDRE is generally preferred. Its values are lower than NDVI for the same canopy, typically 0.2 to 0.5 for healthy crops, which surprises people meeting it for the first time, and it is especially sensitive to the exact wavelength of the red-edge band. The Green NDVI, introduced by Anatoly Gitelson, Yoram Kaufman and Mark Merzlyak in 1996, uses green instead of red. Green reflectance varies more with chlorophyll content than red reflectance does at high chlorophyll levels, so GNDVI also extends the useful range, though less effectively than red-edge indices. The Soil-Adjusted Vegetation Index, SAVI, was proposed by Alfredo Huete in 1988 to reduce the influence of soil brightness on sparse canopies. It adds a constant to the denominator, typically 0.5, that dampens the effect of the soil background. Optimized variants such as OSAVI use a different constant. These indices are useful early in the season, in wide-row crops, in vineyards and orchards where soil or cover crops fill much of each pixel, and in arid systems. The Enhanced Vegetation Index, developed for NASA's MODIS satellites, adds a blue band to correct for atmospheric effects and further reduces saturation. It is common in satellite products but less used in drone work, where the atmosphere between camera and crop is thin. No index is universally best. The right choice depends on crop, growth stage and question. A practical rule is to use NDVI or a soil-adjusted index for stand assessment and early season variability, switch to a red-edge index once the canopy closes, and use thermal data when the question is water. Distortions and displays that mislead Beyond saturation, several effects can produce patterns on an index map that have nothing to do with crop condition. Soil background is the first. In young crops, a dark wet patch of soil and a light dry patch can produce different NDVI values even when the plants above them are identical. After rain, the darker wet soil lowers red reflectance more than near-infrared reflectance in some soils, nudging NDVI upward. Fields with variable soil color, such as eroded knolls exposing lighter subsoil, can show NDVI patterns that track soil rather than crop. Weeds are the second. A weedy patch in a young soybean crop has more green leaf than a clean patch. On an NDVI map it looks healthier. Unless the resolution is fine enough to separate weed from crop, the map rewards the problem. Shadows and sun angle are the third. Shaded leaves within the canopy, shadows of tall crop on shorter crop or on soil, and shadows from trees along field edges all depress red and near-infrared reflectance and distort the index. Directional reflectance effects, discussed in the previous chapter, add stripes along flight lines. Phenology is the fourth. Tasseling maize, flowering canola and heading wheat change canopy reflectance dramatically. Yellow canola flowers, for example, reflect strongly in the green and red, lowering NDVI during flowering even in a perfectly healthy crop. A map flown during flowering cannot be compared directly with one flown before it. Hybrid and variety differences are the fifth. Different maize hybrids differ in leaf angle, color and canopy architecture. A strip trial with several hybrids will often show stripes on an NDVI map that reflect genetics rather than stress. A vegetation index map reaches the farmer as colors, not numbers, and the choice of color scheme shapes what the reader believes. Most agricultural software uses a red-to-green palette, with red for low values and green for high. That convention carries an implicit judgment: red looks like danger. Applied to a map whose values range from 0.82 to 0.86, the palette paints the lowest few percent of the field red, even though the entire field may be healthy and the differences may lie within calibration error. The problem is compounded by how values are grouped into classes. Software commonly offers equal-interval classes, which divide the range from minimum to maximum into bands of equal width, and quantile classes, which put an equal number of pixels in each band. Equal-interval classes stretched over a narrow range exaggerate small differences. Quantile classes guarantee that a fixed fraction of the field, say a fifth, will always be painted in the lowest color, whatever the actual values. Neither is wrong, but both can produce a dramatic-looking map from a nearly uniform field. Consider two fields. In the first, NDVI ranges from 0.30 to 0.85, with a clear patch of poor emergence at 0.30 to 0.45. In the second, NDVI ranges from 0.81 to 0.85, with natural variation and some flight-line striping. With quantile classes and a red-to-green palette, both maps show a fifth of the field in red. In the first case the red is a real problem worth investigating. In the second, the red is largely an artifact of display choices. A reader who looks only at the colors cannot tell them apart. The defensive habits are simple. Always look at the legend and the actual range of values before interpreting colors. Prefer fixed scales across flights and fields, so that the same color means the same value every time, over scales stretched to each map. Compare the spread of values with what calibration can support; differences of a few hundredths of NDVI between zones are rarely meaningful without a reference strip in the same flight. And be wary of any service report that shows only colored maps without the underlying numbers. From reading to scouting The disciplined way to use an index map follows directly. First, check the map for artifacts before reading it as agronomy. Second, look for patterns, not pixels: zones of low or high values that persist across the field, have sensible shapes and relate to things the farmer knows about, such as soil types, drainage, previous crops or equipment paths. Third, ask what could explain each pattern, using the list of causes rather than jumping to the most familiar one. Fourth, go to the field. Ground truthing is not a polite gesture toward tradition. It is the step that converts a symptom map into a diagnosis. An agronomist with the map loaded on a phone, walking to the center of a low-index zone and to a nearby high-index zone, digging up roots, checking leaf color against a chart, looking for lesions or insects, measuring soil moisture and taking tissue samples, can usually identify the cause within an hour. That hour, guided by the map, replaces days of walking the whole field blindly. That is where much of the value of drone imagery actually lies: not in replacing the agronomist, but in directing the agronomist's limited time to the places that matter. Repeated flights add a further dimension. A low zone that appears at emergence and persists all season suggests a soil or stand problem. A zone that appears suddenly in mid-season and expands suggests a disease or pest. A zone that waxes and wanes with rainfall suggests water. Change over time is often more diagnostic than any single snapshot, which is why consistent calibration across flights, discussed in the previous chapter, matters so much. The deeper challenge comes when the goal is not to explain visible variation but to find stress before it becomes visible at all. That is the subject of the next chapter. Chapter 4: Seeing Stress Before It Shows The most compelling claim made for agricultural drones is that they can detect crop stress before it is visible to the human eye. The claim is true in a specific and limited sense, and false in the sense many people hear it. Drones carrying the right sensors can register physiological changes in plants days, sometimes weeks, before a person walking the field would notice anything. But registering a change is not the same as knowing what caused it, and an early warning is only valuable if it arrives early enough, reliably enough and specifically enough to change what the farmer does. This chapter examines how pre-visual detection works, where the evidence is strong, and where the claim outruns the science. Why stress is invisible at first A plant under stress goes through a sequence of changes, and the order matters. The first responses are physiological and reversible. When roots cannot supply enough water, guard cells close the stomata within minutes to hours. When light exceeds what photosynthesis can use, the xanthophyll cycle shifts to dissipate excess energy as heat. When a pathogen enters the vascular system, water transport becomes impeded, and the leaves it supplies begin to behave as though they were drought stressed. None of these responses changes leaf color. The next responses are biochemical. If stress persists, chlorophyll synthesis slows and existing chlorophyll begins to break down. Nitrogen is remobilized from older leaves to younger ones. Pigment ratios shift. These changes are gradual, and the eye is poor at detecting gradual color changes, especially across a large area with no healthy reference immediately beside the affected plants. A leaf can lose a meaningful fraction of its chlorophyll before a trained observer notices a difference in shade. The final responses are structural and visible: wilting, rolling, yellowing, necrosis, lesions, stunting, leaf drop. By this point, yield potential has often already been reduced. For nitrogen in maize, for example, the period of rapid uptake and ear size determination occurs before the crop shows obvious deficiency symptoms in many cases, so a correction made only after yellowing appears may be too late to recover all the lost yield. Each stage has a sensor that matches it. Stomatal closure appears first in temperature. Xanthophyll changes appear in narrow bands around 531 nanometers. Chlorophyll decline appears in the red edge before it appears in the broad red band or in visible color. Structural change appears in everything, including the eye. Pre-visual detection, properly understood, means choosing a sensor that operates at an earlier stage of this sequence than human vision does. Strong cases and weak ones Water stress is the clearest success. Because transpiration cools leaves and stomatal closure warms them, thermal imaging can map water status across a field or orchard at a resolution that resolves individual trees or vines. Research groups in Spain, California, Australia and Israel have shown over the past two decades that canopy temperature indices from aerial thermal imagery correlate well with direct measurements of plant water status, such as stem water potential measured with a pressure chamber, in crops including grapevines, almonds, olives, citrus and cotton. The practical application is irrigation management: identifying blocks, rows or zones that are under-watered or over-watered, finding blocked or leaking emitters in drip systems, and scheduling deficit irrigation in vineyards where moderate, controlled water stress improves wine quality. In these settings the thermal signal arrives before visible wilting, it is specific to water status in the sense that it measures the transpiration that water stress suppresses, and it leads directly to an action the grower controls. Nitrogen status is the second strong case. Because nitrogen is a central component of chlorophyll, nitrogen-deficient plants make less chlorophyll, and red-edge indices respond to that decline before it becomes visible as yellowing. Studies in maize and wheat have repeatedly shown that red-edge indices measured from drones or ground sensors track leaf nitrogen and chlorophyll meter readings, and that they can separate nitrogen rates in field trials at growth stages where visual differences are minimal. The signal is strongest when compared against a well-fertilized reference strip in the same field, a point the next chapter develops. The most cited example of truly pre-visual disease detection comes from a study published in Nature Plants in 2018 by Pablo Zarco-Tejada and colleagues. Working in olive orchards in the Apulia region of southern Italy, which had been devastated by the bacterium Xylella fastidiosa, the team flew crewed aircraft carrying hyperspectral and thermal imagers over thousands of trees. Xylella blocks the water-conducting xylem, and the team used a combination of indicators: thermal stress measures, chlorophyll fluorescence, and pigment-related spectral traits, interpreted through models of canopy structure. The combined approach detected infected trees with accuracy exceeding 80 percent before visible symptoms appeared, and trees the analysis flagged as affected while still looking healthy went on to develop symptoms at almost twice the rate of those it judged unaffected. The study is a landmark, and it is worth being precise about what it shows. It used high-end airborne instruments, not commodity drone cameras. It relied on sophisticated physical modeling, not a single index. And it addressed a pathogen whose primary effect, blocked water transport, produces a strong thermal and physiological signature. It demonstrates what is possible with the right sensor and the right biology, not what any drone with an NDVI camera can deliver. Many other stresses are harder to detect early, and the evidence is thinner or more mixed. Leaf diseases that begin as small, scattered lesions, such as many fungal leaf spots and rusts, produce local changes affecting a tiny fraction of the leaf area in the early stages. At typical mapping resolutions, these are diluted in each pixel by healthy tissue, and the canopy-level signal remains within the noise until infection is widespread. Very high resolution imagery, with pixels of a few millimeters, can sometimes resolve individual lesions, but that requires low, slow flights over small areas, and at that point the drone is acting more as a close-up camera than as a mapping tool. Machine learning models trained to recognize disease in such imagery have shown promising results in research, but performance often drops sharply when a model trained in one field, season or variety is applied to another. Insect damage varies widely. Pests that feed on roots or bore into stems can cause wilting and temperature changes detectable by thermal imaging, sometimes before visible symptoms. Spider mites, which damage leaf cells and reduce chlorophyll, have been detected in some studies through spectral changes. Defoliating insects produce structural damage that is, by definition, visible. In most cases, a drone map can show where crop vigor or temperature is abnormal, but identifying the insect requires someone to look. Soil-borne diseases and nematodes often reveal themselves on index maps as irregular patches of lower vigor that persist and expand over seasons. Here the drone map is useful less for pre-visual detection in any given week than for revealing the spatial pattern that suggests a soil-borne cause, which an agronomist then confirms by sampling. Micronutrient deficiencies, herbicide injury and many other stresses produce reductions in chlorophyll that look spectrally similar to nitrogen deficiency. A red-edge index will register them, sometimes early, but cannot distinguish among them. Detection, diagnosis and machine learning The central difficulty is specificity. Most spectral and thermal signals are generic stress responses. Stomatal closure is triggered by drought, but also by root rot, vascular disease, waterlogging, salinity and heat. Chlorophyll decline is triggered by nitrogen shortage, but also by sulfur, magnesium or iron deficiency, by senescence, by disease and by herbicide drift. The plant has a limited repertoire of responses to a large range of insults, and the sensor sees the response, not the insult. This has practical consequences. Suppose a drone thermal flight over a vineyard shows a block of vines running two degrees warmer than the rest. The cause might be an irrigation fault, a soil with lower water-holding capacity, a root disease, or a trunk disease restricting water flow. The first calls for a repair, the second for a different irrigation schedule, the third for investigation and perhaps replanting, and the fourth for removal of infected vines. The thermal map has done its job by finding the block, perhaps a week before the vines would have shown visible stress. It cannot, by itself, choose among the responses. Early detection is also a trade-off between sensitivity and false alarms. The earlier a signal is sought, the smaller it is relative to the noise, and the more often apparent anomalies will turn out to be artifacts of calibration, illumination, soil background or natural variation in the crop. A system tuned to catch every real case of early stress will send the agronomist to many places where nothing is wrong. A system tuned to send the agronomist only to genuine problems will miss some early cases. There is no setting that avoids both. The right balance depends on the cost of a missed case compared with the cost of a wasted scouting trip, and that varies enormously between, say, a quarantine pathogen in a high-value orchard and a minor leaf spot in a commodity grain crop. Finally, early detection only creates value if it changes the outcome. A warning about nitrogen deficiency at a growth stage when the crop can still respond to fertilizer is valuable. The same warning after the crop has passed the point of response is merely interesting. A warning about an incurable disease is valuable if it allows infected plants to be removed before the pathogen spreads, but not if no removal is possible. Before investing in early detection, it is worth asking a simple question: if the system detected this stress a week earlier, what would we do differently, and would it matter? Much of the recent excitement about early stress detection comes from machine learning. Instead of relying on a single index chosen by a scientist, a model is trained on many labeled examples, images of healthy and diseased plants, or of fields with and without a particular pest, and learns which combinations of spectral, spatial and textural features distinguish them. Convolutional neural networks, which learn features directly from image pixels, have achieved impressive accuracies in research papers on tasks such as detecting particular leaf diseases, identifying weeds among crop seedlings and counting plants. The difficulty is that a model learns whatever distinguishes its training examples, and in agriculture many things vary together. If all the diseased examples came from one field photographed on a cloudy day and all the healthy ones from another field on a sunny day, a model can achieve high accuracy by learning to recognize the weather. Less obvious versions of the same problem are common. A model trained on one maize hybrid may rely on leaf angle or color traits that differ in another. A model trained at one growth stage may fail at the next. A model trained with one camera may fail with another whose red-edge band sits 13 nanometers away. Researchers call this the domain shift or transfer problem, and it is the main reason that accuracies reported in research papers frequently fall when models are deployed on new farms. The practical implications are threefold. Accuracy claims mean little unless the model was tested on fields, seasons and conditions that were entirely separate from those used to train it. Models used commercially need continuing validation against ground truth on the farms where they are deployed, not a one-time demonstration. And the most robust approaches tend to combine learned models with physical understanding, using calibrated reflectance rather than raw pixel values, and features with a known biological meaning, such as red-edge position or canopy temperature, rather than relying on a model to rediscover physics from limited data. The Xylella study described above is an example of this combination: it used physical models of the canopy to extract plant traits before applying statistical classification. Machine learning will undoubtedly improve early detection. It does not remove the need for the farmer and agronomist to confirm what the model has flagged, and it adds a new question to ask of any detection service: on what data was this model trained, and how has it performed on fields like mine? Building an early-warning practice Farms and advisers who use drone imagery effectively for early stress detection tend to share a set of habits. They fly a baseline. Anomalies are only meaningful relative to what is normal for that field, crop and stage. A first flight shortly after emergence, and regular flights afterward, establish what the field looks like when things are going well, and make departures visible. They use references inside the field. A well-watered, well-fertilized strip or block provides a benchmark that experiences the same weather, soil and calibration conditions as the rest of the field. Comparing each zone with the reference cancels out many of the confounding factors that make absolute index values unreliable. They choose sensors for specific stresses. Thermal for water and vascular problems. Red edge for nitrogen and chlorophyll. High-resolution visible imagery for stand counts, weeds and visible damage. A generic NDVI map is a reasonable starting point but rarely the best tool for a specific early-warning question. They look for change rather than state. A zone whose index is falling relative to the rest of the field is more informative than a zone whose index is simply low, because the falling zone is where something is actively happening. They close the loop with the ground. Every anomaly that triggers action is checked in person, with tissue tests, soil probes, pressure chambers or laboratory diagnosis as appropriate. Over time, this builds a local library of what different problems look like on that farm's maps, which makes each subsequent season's interpretation faster and more reliable. Drone imagery, used in this way, rarely provides the full answer. What it provides is a way of finding the right questions early, in the right places, across areas far too large to inspect plant by plant. The next step is to turn the answers into a prescription that a machine can carry out. Chapter 5: From Map to Prescription A diagnosis does not apply itself. Between knowing that one part of a field needs something different from another and actually delivering it lies a translation: from continuous pixel values to discrete zones, from zones to rates, from rates to a file a machine can read, and from the file to a controller that opens and closes valves at the right moment over the right patch of ground. This is the core of variable rate application, and it is where agronomy, arithmetic and engineering meet. Done well, it puts inputs where they produce a return and withholds them where they do not. Done badly, it moves product around a field in an elaborate pattern that has no relationship to what the crop needs. Zones: the unit of decision A drone map may contain tens of millions of pixels. No sprayer or spreader can respond to each one, and no agronomic recommendation is precise enough to justify a different rate every few centimeters. The first step in turning a map into a prescription is therefore to group the field into management zones: contiguous areas that are similar enough internally, and different enough from each other, to justify different treatment. Zones can be drawn from a single drone flight, but they are usually more robust when built from several layers. Soil electrical conductivity maps, which reflect texture and water-holding capacity. Elevation, which governs drainage and where water collects. Several years of yield maps, which show where the field consistently performs well or poorly. Satellite vegetation index histories, which show how different areas behave across seasons. A drone image from the current season then adds a fresh layer showing how this year's crop is responding. Zones that persist across these layers are usually driven by soil and landscape and are stable from year to year. Zones that appear only in the current season's imagery suggest something transient, such as a localized pest problem or uneven fertilizer from an earlier pass. The size of zones is constrained by the application machine. A ground sprayer with a 36-meter boom cannot change rate across a two-meter strip unless it has individually controlled nozzles. A spreader throwing fertilizer across 24 meters blends rates at zone boundaries. Every rate change takes time to take effect, as pressure stabilizes or product travels down a line, and in that time the machine has moved several meters. These physical limits define a minimum practical zone size. Drawing zones smaller than the machine can resolve does not make the prescription more precise; it makes the actual application diverge from the intended one in ways that are hard to check. Spray drones are, in one respect, well suited to fine zoning: their swath is narrow, typically 5 to 13 meters for current large models, their flow can be adjusted quickly by changing pump speed, and they fly at a known position with centimeter-level accuracy when RTK is used. Rate changes can therefore follow zone boundaries more closely than a wide ground boom can. That advantage is real but only matters if the zones themselves are meaningful at that scale. The logic of rates Once zones exist, each needs a rate. Here it is important to separate two fundamentally different logics. The first is to apply more where the crop is doing worse. This is intuitive and often wrong. If a zone has low vigor because the soil there holds little water, adding more nitrogen will not fix the water shortage and may simply be wasted or leached. If a zone has low vigor because of a disease, more fertilizer may even worsen the problem. Applying more to poor areas makes sense only when the cause of poor performance is a shortage of the input being varied. The second is to apply more where the crop will respond. This requires knowing, or estimating, the yield response to the input in each zone. In a high-potential zone where yield is limited by nitrogen, extra nitrogen pays. In a low-potential zone limited by water or soil, the same nitrogen produces little additional yield, and the economically optimal rate is lower. This logic sometimes leads to the counterintuitive prescription of applying less to the poorest zones. Which logic applies depends on the diagnosis, which is why the previous chapters insisted that the map be explained before it is acted upon. For crop protection products, the logic is different again. Fungicides and insecticides are generally applied at the labeled rate where they are needed and not at all where they are not. Herbicides follow the same pattern: the opportunity lies not in varying the dose continuously but in spraying only where weeds are present. Variable rate for crop protection is therefore often better described as variable area: the product is either on or off, zone by zone or plant by plant, at a constant label-compliant rate. The most developed example of turning crop reflectance into a rate is in-season nitrogen management for cereals, and the method shows the general principles clearly. In the late 1990s and early 2000s, researchers at Oklahoma State University, led by William Raun and John Solie, developed an approach for winter wheat, later extended to maize, using NDVI from active ground sensors. Their method rests on two ideas. The first is a nitrogen-rich reference strip: a strip in each field that receives enough nitrogen early in the season to ensure nitrogen cannot limit growth there. The second is a response index: the ratio of the vegetation index in the reference strip to that in the rest of the field. If the farmer's normally fertilized crop looks as vigorous as the reference strip, the response index is close to one and additional nitrogen is unlikely to pay. If the reference strip is markedly greener, the crop is nitrogen-limited and will respond. The method then estimates yield potential from the vegetation index measured at a given growth stage, normalized by the number of growing days since planting, and calculates the additional nitrogen needed as the difference between the nitrogen uptake implied by the predicted yield with and without extra fertilizer, divided by an assumed efficiency. A related approach developed by Kyle Holland and James Schepers, published in 2010, uses a sufficiency index, the ratio of each zone's vegetation index to that of the reference, to scale the nitrogen rate between a minimum and a maximum. A simplified example shows how the arithmetic works. Consider a 40-hectare maize field at the stage where in-season nitrogen is planned, with a reference strip that received ample nitrogen at planting. A drone flight using a red-edge index gives the reference strip an average NDRE of 0.40. The field divides into three zones. Zone A, 15 hectares on good soil, averages 0.39. Zone B, 18 hectares, averages 0.35. Zone C, 7 hectares on a sandy ridge, averages 0.30. The sufficiency index is each zone's value divided by the reference. Zone A is at 0.975, essentially sufficient. Zone B is at 0.875, clearly deficient. Zone C is at 0.75, severely below the reference. Suppose the farmer's agronomist uses a rule that applies no extra nitrogen above a sufficiency of 0.95 and scales linearly up to a maximum of 80 kilograms per hectare at a sufficiency of 0.80 or below. Zone A receives nothing. Zone B, halfway between 0.95 and 0.80 at 0.875, receives 40 kilograms per hectare, or 720 kilograms in total. Zone C would, by the formula, receive the maximum, 560 kilograms in total. But here is where diagnosis enters. The agronomist walks Zone C and finds that the plants are short and the soil dry; tissue tests show nitrogen concentration not much below the reference. The low index reflects water limitation on a sandy ridge, not nitrogen deficiency. Adding 80 kilograms per hectare there would produce little yield and risk leaching. The agronomist caps Zone C at 20 kilograms, or 140 kilograms in total. The final prescription applies 860 kilograms of nitrogen across the field. A uniform application of 40 kilograms per hectare, the kind of blanket in-season top-up many farmers would otherwise make, would have used 1,600 kilograms. The variable rate prescription uses about 46 percent less nitrogen and places it where the crop can use it. This example is deliberately simple. Real algorithms use calibration curves derived from many site-years of trials, and the numbers above are illustrative rather than a recommendation. But the structure is general: a reference that defines what unconstrained growth looks like, a measurement of how far each zone falls short, a rule converting shortfall into rate, and a diagnostic check that the shortfall is caused by the input being varied. From prescription to machine It is worth being clear about which inputs a spray drone can sensibly deliver by variable rate, because payload constrains everything. In-season liquid nitrogen for maize, such as urea ammonium nitrate solution, is typically applied at tens of kilograms of nitrogen per hectare, which means hundreds of liters of product and diluent per hectare. A drone carrying 40 to 100 liters per load would need many refills per hectare, making it impractical for broadacre nitrogen except in small areas or difficult terrain. Granular fertilizer can be spread from drones with spreader attachments, and larger models advertise high flow rates, but the tonnages required for broadacre nutrient programs are better suited to ground equipment. Where drones excel is in inputs applied at low volumes: fungicides, insecticides and herbicides at the low carrier volumes that drone labels and practice allow, foliar micronutrients, plant growth regulators, and cover crop seed broadcast into standing crops. For these, the drone can carry enough product to treat a meaningful area per load, and its ability to follow a prescription closely becomes an advantage. In practice, therefore, the drone loop most often runs from imaging to targeted crop protection, while imaging to variable rate fertilizer usually ends in a prescription file sent to a ground spreader or sprayer. A prescription map is ultimately a file: commonly a shapefile or an ISO-XML task file, each zone polygon tagged with a rate. Farm management platforms and drone software can generate these files from imagery, and most modern controllers can read them. A spray drone's ground station typically allows the operator to load a prescription, assign rates to zones and plan flight routes so that the aircraft adjusts flow as it crosses zone boundaries. Several practical steps determine whether the machine does what the file says. The prescription must align with the field boundary and the machine's positioning system; a map offset by three meters will apply the wrong rate along every zone edge. The rates in the file must fall within the machine's achievable range at the planned speed; a rate that requires more flow than the pump can deliver will silently be capped. The controller must log what it actually applied, so that an as-applied map can be compared with the prescription. And the operator must calibrate flow, by measuring actual output against commanded output, at the start of each job and whenever product or nozzles change. Latency deserves particular attention, because its effects are easy to calculate and easy to ignore. Every controller takes some time between crossing a zone boundary and delivering the new rate: the software must register the position, command the pump, and the liquid already in the lines must be displaced before the new flow reaches the atomizers. Suppose that total delay is one second and the machine travels at 7 meters per second. The new rate arrives 7 meters after the boundary. For a ground sprayer with a 36-meter boom crossing a zone boundary at right angles, each crossing misapplies a strip of about 7 by 36 meters, roughly 250 square meters. A field divided into many small zones, crossed dozens of times per pass, accumulates a meaningful misapplied area, all of it concentrated along the zone edges where the prescription is supposed to change. Good controllers use look-ahead, commanding the change slightly before the boundary based on speed and known delay, which largely cancels the lag. The operator should know whether the machine does so. The same arithmetic applies to spot spraying, where the target may be a single weed patch a few meters across. A one-second delay at 7 meters per second would place the spray entirely beyond a two-meter patch. Spot application from any moving machine therefore depends on very short delays, fast-acting valves close to the nozzles, or look-ahead that triggers spray before the target is reached; it is one of the reasons why real-time camera-guided systems mount their cameras ahead of the nozzles and why map-based drone spot spraying adds buffers around each target. The as-applied map closes the loop at this stage. It shows where the machine went, what rate it delivered, and where it deviated from the plan. Comparing it with the prescription catches errors: missed strips, overlaps, rate lags at boundaries, sections where the pump hit its limit. Comparing it later with the following season's imagery and yield map shows whether the prescription worked. Without the as-applied record, variable rate application is an act of faith. Measuring whether it worked The most neglected step in variable rate practice is testing the prescription. A farm that applies a variable rate and then sees a good yield has learned very little; the yield might have been as good with a uniform rate. The only way to know is to leave comparisons in the field: strips where the uniform rate is applied alongside the variable rate, replicated across zones, ideally laid out with the help of an agronomist familiar with on-farm trial design. Modern yield monitors and drone imagery make such comparisons cheap to measure. Farms that run them for a few seasons develop a calibrated sense of where variable rate pays on their land and where it does not, and that knowledge is worth more than any single prescription. This testing discipline matters doubly for the claims made about drones. Statements that drone-based targeting saves a certain percentage of product, or raises yield by a certain amount, are only meaningful against a defined comparison. The next chapter turns to the machine that delivers the prescription, and to what it can and cannot do. Chapter 6: The Spray Drone as an Application Machine The imaging drone and the spray drone are often discussed together, but they are very different machines. The first is a light aircraft carrying a camera; if it flies a little high or a little fast, the map is slightly less sharp. The second is a heavy-lift aircraft carrying tens of kilograms of pesticide, and if it flies a little high or a little fast, chemicals land in the wrong place. The spray drone is the point in the loop where information becomes physical action, and its behavior is governed by aerodynamics, fluid mechanics and battery chemistry rather than optics. Understanding those mechanisms is the only way to judge what the machine can deliver. From helicopters to multirotors The idea of unmanned crop spraying is older than most people assume. In Japan, where rice is grown in small, irregular paddies surrounded by houses and roads that make crewed aerial application difficult, Yamaha Motor developed remote-controlled spraying helicopters in the late 1980s and 1990s. The R-50 and then the larger RMAX, a gasoline-powered helicopter with a payload of roughly 30 kilograms, became a standard tool in Japanese rice farming and later in South Korea. By 2015, the RMAX fleet had logged more than two million flight hours. These machines proved the principle: a small unmanned aircraft flying a few meters above the crop could apply pesticides efficiently in places where neither tractors nor crewed aircraft worked well. The step change came with electric multirotors. In the mid-2010s, Chinese manufacturers, above all DJI and XAG, began producing battery-powered spray drones with four, six or eight rotors, flight controllers inherited from the consumer drone industry, and steadily increasing payloads. Early models carried around 10 liters. By the mid-2020s, the largest mainstream models carried 50 to 100 liters. DJI's Agras T100, released in 2025, has a 100-liter liquid tank and a maximum takeoff weight of about 175 kilograms with a full load. China became by far the largest market, and DJI reported in 2026 that its agricultural drone fleet worldwide had reached about 600,000 aircraft by the end of 2025, operating in more than 100 countries. The scale of adoption in China reflects particular conditions: small and fragmented fields, a shrinking and aging rural workforce, extensive rice and wheat cultivation, government subsidies for agricultural machinery, and a domestic manufacturing base. A government-linked industry white paper published in 2025 reported that agricultural drones in China treated about 173 million hectares of cropland in 2024, a figure that counts each pass separately and therefore exceeds the country's cultivated area. Elsewhere, adoption has been rapid in Brazil, Southeast Asia and, until recent trade restrictions, the United States. Downwash, droplets and atomizers A spray drone flies at a height of typically two to four meters above the crop canopy, at speeds of a few meters per second up to the low teens, following a pre-planned route. Liquid is pumped from the tank to atomizers or nozzles mounted beneath or beside the rotors, broken into droplets and released into the airflow. Where those droplets land is determined by an interaction between three air movements: the downwash from the rotors, the forward motion of the aircraft and the ambient wind. Downwash is the defining feature. A multirotor stays aloft by pushing air downward. A drone weighing 100 kilograms or more must accelerate a substantial mass of air toward the ground every second, and the resulting column of air moving downward beneath the aircraft can reach speeds of several meters per second at canopy level. This downwash is the main reason drones can deliver spray into a canopy at all from such low volumes: it entrains the droplets, drives them downward, and can open the canopy so that droplets reach lower leaves. It is also why flight height and speed matter so much. Fly too high, and the downwash weakens and spreads before it reaches the canopy, leaving fine droplets drifting in the ambient wind. Fly too fast, and the downwash is swept backward into a trailing vortex system, lifting some droplets upward and distorting the spray pattern. The interaction produces an effective swath width that is narrower than the drone's physical dimensions might suggest and that depends strongly on conditions. Manufacturers quote ranges, such as 5 to 13 meters for the T100, rather than a single figure, because the effective swath changes with height, speed, droplet size and wind. Independent pattern testing, in which drones fly over lines of collectors and the deposited spray is measured, often reveals uneven distribution: a heavy center with lighter edges, or a twin-peaked pattern where the downwash of two rotor groups separates. Agricultural engineers who test drones in Canada and the United States have reported such bimodal patterns at low heights and slow speeds, and generally recommend flying lower than many operators instinctively choose. Getting uniform coverage requires choosing a swath spacing that overlaps the edges of adjacent passes correctly, which can only be determined by pattern testing for a given drone, nozzle and setting. Droplet size is the single most important variable in spray application. Small droplets cover leaves more thoroughly and penetrate dense canopies better, but they drift farther and evaporate faster. Large droplets resist drift and land closer to where they are aimed, but they cover less surface per liter and can bounce off waxy leaves. Spray specialists classify droplet spectra into categories from extremely fine to ultra coarse, under a standard maintained by the American Society of Agricultural and Biological Engineers, usually summarized by the volume median diameter: the droplet size for which half the spray volume is in smaller droplets and half in larger. Spray drones use two main types of atomizer. Hydraulic nozzles, the same basic technology found on ground sprayers, force liquid through a shaped orifice under pressure. Air-induction versions draw air into the liquid stream to produce larger, air-filled droplets that resist drift. Rotary atomizers, also called centrifugal atomizers, feed liquid onto a spinning disc or cage that flings it off as droplets, and the droplet size is controlled by rotation speed: faster spinning makes finer droplets. Rotary atomizers produce a narrower range of droplet sizes than hydraulic nozzles and allow the operator to change droplet size in flight. Their manufacturer specifications often quote wide adjustable ranges; the T100's standard atomizers are listed at 50 to 500 micrometers. Which type is better for drift depends on settings. Research at the Julius Kühn Institute, Germany's federal research center for cultivated plants, illustrates the point. A 2023 study by Andreas Herbst and colleagues in steep-slope vineyards found that drones with standard nozzles produced slightly higher drift than ground equipment, but drones fitted with air-induction nozzles produced drift below the established reference values for ground-based vineyard spraying at all measured downwind distances. A 2025 field comparison by Katrin Ahrens, Herbst and Jens Karl Wegener, following the ISO 22866 drift measurement standard, found that at practical settings of two meters height and 2.5 meters per second, rotary atomizers produced drift equal to or lower than air-induction nozzles, but that both types exceeded vineyard reference values when the drone flew at three meters and 3.5 meters per second. Height emerged as the primary driver of drift for both atomizer types. The lesson is not that one atomizer is superior but that a well-chosen combination of atomizer, height and speed can put drone drift within the range accepted for ground sprayers, and a poorly chosen one cannot. Low volume, batteries and work rate Drones apply very low carrier volumes. A ground sprayer typically applies tens to hundreds of liters of spray mix per hectare; a crewed aircraft commonly applies 20 to 50 liters per hectare; a drone often applies 10 to 30 liters, and sometimes less. Because the dose of active ingredient per hectare is fixed by the label, the lower volume means the tank mix is several times more concentrated. This has consequences. A concentrated mix is more hazardous to handle and mix, which matters for operator safety. Some formulations do not remain stable at high concentration; products can separate, gel or form precipitates, especially when several products are mixed together, which can block lines and atomizers in flight. Coverage, measured as droplets per square centimeter of leaf, falls when volume falls, unless droplets are made smaller, which in turn increases drift risk. For contact fungicides and insecticides that must physically reach the pest, low coverage can reduce efficacy. For systemic products absorbed and moved within the plant, coverage matters less. The regulatory consequence is that many pesticide labels specify minimum carrier volumes for aerial application, and some of those minimums are higher than drones typically deliver. In the United States, extension specialists at Purdue University and elsewhere have noted that labels written for crewed aircraft often do not address drone-specific variables such as carrier volume, release height, nozzle type and droplet size, and that the federal Environmental Protection Agency has largely left it to state regulators to decide whether drones may follow aerial label instructions. Canada has taken a stricter line: Health Canada requires pesticide labels to state explicitly that a product may be applied by a remotely piloted aircraft system before drone application is legal. The eighth chapter returns to these differences. The headline specification of a spray drone is its tank size, but the number that determines its usefulness on a farm is work rate: hectares treated per hour, including everything that happens on the ground. A worked estimate illustrates the arithmetic. Suppose a 100-liter drone applies 20 liters per hectare, so each full tank treats 5 hectares. With an effective swath of 10 meters and a spraying speed of 7 meters per second, the drone covers 70 square meters per second, or 0.007 hectares per second; 5 hectares therefore take about 12 minutes of spraying, before adding turns at row ends and the flight to and from the refill point. When the tank runs dry, the drone lands, the crew swaps the battery and refills the tank, and it takes off again. If that turnaround takes 3 minutes, and turns and transit add another 3, each cycle takes about 18 minutes and treats 5 hectares: roughly 16 hectares per hour. Change any assumption and the number moves substantially. A higher application rate of 30 liters per hectare cuts each load's area by a third. A field far from the refill point adds transit time. Small, irregular fields add turns. Practical work rates reported by operators for large drones vary widely and are frequently lower than manufacturers' peak figures. The battery cycle is the hidden constraint. Large spray drones draw very high power and drain a battery in roughly the time it takes to empty one tank. To keep a drone flying continuously, the crew needs several batteries in rotation, a fast charger, and a generator large enough to supply it, often in the range of tens of kilowatts. Battery temperature limits charging and flight; operators testing the T100 in hot conditions reported overheat warnings and recommend three batteries and a cooling station to sustain a working cycle. Batteries are also consumables: they lose capacity with each cycle, and replacement is a significant share of operating cost. Flying a heavy aircraft two or three meters above a crop also demands sensing that imaging drones do not need. Current large spray drones carry radar or other ranging sensors that measure height above the canopy many times per second, allowing the aircraft to follow undulating terrain and to hold a constant release height over a crop of varying height. That terrain following is directly linked to drift and deposition: a drone that rises to five meters over a dip in the field, or drops too close over a rise, changes its spray pattern with every meter. The same sensors, often combined with cameras, provide obstacle detection for trees, poles and power lines, which are the leading hazards in agricultural aviation of every kind. Obstacle sensing reduces risk but does not eliminate it. Thin wires, guy lines and branches remain difficult to detect, and operators still survey field edges before flying and mark hazards in the flight plan. The consequences of a crash are also different from those of an imaging drone. A spray drone carrying tens of liters of concentrated pesticide mix and a large lithium battery is both a chemical spill and a fire risk if it falls. Operators therefore plan routes that avoid flying over houses, roads, water and people; keep emergency spill kits on site; and set failsafe behaviors that return the aircraft to a safe location or land it in the field if the radio link or positioning is lost. These practices are part of every competent operation, and regulators increasingly ask to see them written down. The ground team typically includes a pilot and at least one assistant who mixes product, refills tanks and manages batteries. The equipment occupies a pickup truck or trailer: drone, batteries, charger, generator, water tank, mixing station and personal protective equipment. Fleet operations, where one pilot supervises several drones flying simultaneously, multiply work rate but require regulatory approval and more ground crew. How the drone compares Spray drones compete with two established methods: self-propelled or tractor-drawn ground sprayers and crewed agricultural aircraft. Table 3 compares their typical characteristics. Table 3. Typical characteristics of application methods (ranges from manufacturer specifications and extension guidance). Attribute Large spray drone Self-propelled ground sprayer Crewed agricultural aircraft Tank or hopper 40 to 100 liters About 3,000 to 6,000 liters About 1,500 to 3,000 liters Effective swath About 5 to 13 m About 27 to 40 m booms About 15 to 25 m Typical carrier volume About 10 to 30 L/ha About 50 to 200 L/ha About 20 to 50 L/ha Soil compaction and crop damage None Wheel tracks None Strengths Small, wet or steep fields; spot treatment High capacity; precise targeting Very high capacity; large fields The comparison shows where drones fit. They cannot match the raw capacity of a large ground sprayer or a crewed aircraft on big, open fields. They do not compact soil or crush crop, and they can work when fields are too wet for ground equipment, on steep slopes, in terraced rice, in small or irregularly shaped fields, near obstacles that make crewed aerial application dangerous, and in tall crops where a ground sprayer would damage plants late in the season. They are also cheaper to acquire than either alternative, and they can respond quickly to small outbreaks without mobilizing a large machine or booking an aircraft. Their natural role, therefore, is complementary: treating the parts of a farm or the moments in a season where other machines are poorly suited, and above all treating the targeted areas that imaging has identified. That last role brings the loop back to its purpose. The environmental case for spray drones rests not on the aircraft but on the targeting. The next chapter examines how much chemical actually ends up where it should, and how much does not. Chapter 7: Drift, Runoff and the Environmental Ledger The environmental case for precision agriculture drones is usually stated in a single sentence: by applying chemicals only where they are needed, drones reduce the quantity of pesticide used and the amount that escapes into streams, groundwater and neighboring land. That sentence is broadly right, but it conflates several distinct mechanisms, some of which drones improve, some of which they leave unchanged, and one of which they can make worse. An honest environmental ledger separates them. How pesticides leave the field A pesticide applied to a crop can reach unintended places by several routes, and each has its own physics. Spray drift is the movement of droplets away from the target during application. Small droplets, roughly below 150 micrometers, fall slowly enough that a breeze can carry them sideways for meters or tens of meters before they land; the smallest can remain airborne much longer. Drift is governed by droplet size, release height, wind speed, air turbulence, temperature and humidity, which control evaporation that shrinks droplets as they fall. Drift can damage neighboring crops, especially when herbicides reach sensitive broadleaf plants, contaminate water bodies directly, and expose people, livestock and wildlife at field edges. Vapor drift is a separate process: some active ingredients volatilize from leaves or soil after they land, in the hours and days following application, and move as gas. Dicamba is the best-known example in North America, where off-target damage from vapor movement led to repeated regulatory interventions. Vapor drift depends on the chemistry of the product and on temperature, not on how it was applied, so the choice of aircraft or ground sprayer makes little difference to it. Runoff is the movement of pesticide with surface water. When rain or irrigation falls on a recently treated field faster than the soil can absorb it, water flows across the surface, carrying dissolved pesticide and pesticide bound to soil particles. Runoff losses are highest when heavy rain falls within days of application, on sloping fields, on soils with low infiltration, and where there is no vegetated buffer to slow the water and trap sediment. Runoff is primarily a function of how much product is on the field, how soluble and persistent it is, and the weather and landscape after application. Leaching is the downward movement of dissolved pesticide through the soil to groundwater, favored by soluble, persistent compounds, sandy soils and heavy rain or irrigation. The crucial observation is that drift depends on the application method, while runoff and leaching depend mainly on the quantity, chemistry and timing of what is applied, and on the landscape. A spray drone that sprays the same product at the same rate over the same area as a ground sprayer will not, by itself, reduce runoff. It can only reduce runoff by reducing the quantity applied, or by changing where and when it is applied. Drift from drones: what the evidence shows Because drones release spray from a height of a few meters into strong downwash, their drift behavior differs from both ground sprayers, which release spray less than a meter above the crop, and crewed aircraft, which release from higher up at much higher speed. Early concerns that drones would produce heavy drift were understandable: their low carrier volumes push operators toward fine droplets, and their rotors create complex, turbulent airflow. The best controlled evidence comes from standardized drift trials. Germany's Julius Kühn Institute has run a series of such trials using the ISO 22866 protocol, in which collectors placed at intervals downwind of a treated area capture deposited spray. As described in the previous chapter, these trials found that drones with suitable nozzles, flown low and slowly, produced drift comparable to or lower than the reference values established for ground sprayers in vineyards, while drones flown higher and faster exceeded those values. Studies in China, where much of the early drone drift research was done, including wind tunnel and field experiments published in the International Journal of Agricultural and Biological Engineering, have similarly identified flight height, speed, wind and droplet size as the governing variables, and have found that drift-reducing adjuvants added to the tank mix can lower drift potential. The pattern across this literature is consistent. A spray drone is not inherently a high-drift or low-drift machine. It is a machine whose drift depends sharply on settings that the operator controls. Operated with coarse droplets, at two to three meters above the canopy, at moderate speed, in light and steady wind, it can match ground equipment. Operated with fine droplets, high and fast, in gusty conditions, it can drift considerably. Compared with crewed aircraft, which must release spray from higher altitudes and at speeds of tens of meters per second, drones generally have the advantage of lower release height and slower flight, which is why regulators in some jurisdictions consider them a lower-drift alternative for steep vineyards and other places where crewed aerial application was previously the only option. Those findings, though, come largely from trials conducted by research institutes under controlled conditions with trained operators. In commercial practice, drift outcomes depend on whether operators actually fly at the recommended height, check wind speed, choose appropriate nozzles and respect buffer zones. That gap between controlled performance and field practice is a familiar problem with every spraying technology, and it is the reason why operator training, certification and record keeping figure so prominently in regulatory debates. Where the real savings come from If drones neither inherently reduce runoff nor inherently reduce drift, where does the large environmental benefit claimed for precision application come from? The answer is targeting: applying product to a fraction of the field instead of all of it. The magnitude of savings depends on how much of the field actually needs treatment. For herbicides, the opportunity is large, because weeds are usually patchy. Many fields, particularly after a pre-emergence herbicide program, have weeds over only a minority of their area in-season. A broadcast application sprays the whole field to hit those patches. A targeted application sprays only the patches. The clearest large-scale evidence comes not from drones but from camera-guided ground sprayers, which demonstrate the principle at scale. John Deere's See & Spray system, which uses boom-mounted cameras and onboard computers to identify weeds and trigger individual nozzles, was used on more than one million acres in the United States in 2024. The company reported an average herbicide saving of 59 percent across corn, soybean and cotton fields, equivalent to roughly 8 million gallons of herbicide mix not applied. An independent evaluation by Iowa State University reported average product savings of 76 percent in its test fields. These figures depend on weed pressure: in heavily infested fields, savings are smaller because more nozzles fire; in clean fields, they are larger. Drones can achieve similar targeting by two routes. The first is map-based: an imaging drone flies the field, software identifies weed patches or diseased zones, and a prescription map directs the spray drone to treat only those areas. This works well for patchy weeds that are large enough to resolve, for perennial weed patches that recur in the same places, and for disease or pest outbreaks that occupy identifiable zones. The second is real-time: the spray drone itself carries cameras and software that identify targets as it flies and switch spray on and off accordingly. Precision AI, a Canadian company, has been developing such a system, reporting herbicide savings of 83 percent compared with a ground sprayer in a side-by-side trial while achieving equivalent weed control, with limited commercial availability planned for 2026. Such company-reported results are encouraging but have not yet been replicated at commercial scale in independent trials, and the difficulty of identifying small weeds from a moving aircraft above a crop is considerable. The environmental logic follows directly. If a herbicide is applied to 30 percent of a field instead of 100 percent, the mass of active ingredient on the field falls by 70 percent. Runoff and leaching losses, which scale broadly with the mass of product present, fall correspondingly, all else being equal. Drift, which scales with the area sprayed, falls too. And the farmer's bill for herbicide falls by the same fraction, which is why targeting tends to spread once it works: the environmental benefit and the economic benefit point in the same direction. A worked example makes the scale concrete. Consider a 100-hectare soybean field due for a post-emergence herbicide application at a label rate that delivers, for illustration, 1 kilogram of active ingredient per hectare. A broadcast application puts 100 kilograms of active ingredient on the field. A widely cited review of field studies by R. Don Wauchope, published in 1978, found that runoff losses of most pesticides are usually below about half a percent of the amount applied, but can reach several percent when intense rain falls soon after application. Suppose a wet spell follows the spray and 2 percent is lost: 2 kilograms reach the drainage ditch. If an imaging flight shows that weeds occupy patches covering 25 hectares, and those patches, with a buffer around each, are treated by a spray drone or a camera-guided sprayer, the mass on the field falls to perhaps 30 kilograms, and the expected runoff loss to about 0.6 kilograms. The proportional reduction, 70 percent, is the same whether the season is wet or dry; the absolute reduction is greatest in exactly the wet years when runoff is worst. Two cautions temper the example. First, targeted application only works when the weeds are actually where the map says and the treatment window is appropriate; weeds that emerge after the imaging flight, or that are too small to see, are missed and may escape control, which can lead to a second application that erodes the saving. Second, resistance management sometimes argues for treating whole fields with residual products, and targeting decisions should be made with the weed control program as a whole in mind, not just the in-season pass. Targeting is a tool of integrated weed management, not a substitute for it. The other entries in the ledger Several further items belong on the environmental balance sheet, some positive and some negative. On the positive side, drones do not compact soil. Ground sprayers, especially heavy self-propelled machines, compact soil along their wheel tracks, which reduces infiltration and can increase runoff from those tracks. Drones also allow application when fields are too wet for ground equipment, which can mean timely treatment of a disease outbreak that would otherwise spread. And they avoid crop damage from wheels in late-season applications. Manufacturers also point to water savings from low carrier volumes. DJI reported in 2026 that its global fleet had saved about 410 million tonnes of water, a figure derived from comparing drone carrier volumes with assumed conventional volumes; such calculations depend heavily on their assumptions and are best treated as estimates of order of magnitude rather than measurements. On the negative side, concentration is the most serious concern. A drone tank mix is several times more concentrated than a ground sprayer mix. That increases the hazard to the person mixing and loading, who handles concentrated product repeatedly through the day, often in hot conditions, refilling small tanks many times per hour. It also means that a spill, a crash or a leak during flight releases more active ingredient per liter than an equivalent ground sprayer accident would. Drone crashes, while not common, are not rare either, and a drone that falls into a stream with a partly full tank is a point-source contamination event. Pollinator exposure is a second concern. Drones can be flown over flowering crops and orchards more easily than ground sprayers, and the ease of deployment may tempt operators to spray at times or in places that put bees at risk. In April 2026, BeeLife, the European beekeeping coordination group, published a position on the European Union's proposal to allow certain drone spraying, arguing that independent evidence that drones systematically reduce drift and pollinator exposure is insufficient, and calling for mandatory tracking of drone applications, operator certification, wind limits, monitoring programs and notification of neighboring beekeepers. These demands are a reasonable statement of what accountability for a new application method requires. The rebound effect is a third concern. Because drones make spraying easier, cheaper and more flexible, they could increase the total number of applications made, for example by enabling treatments on fields that were previously not sprayed because access was difficult, or by encouraging prophylactic treatment because a flight is easy to arrange. Whether this happens depends on how drones are used. The targeting logic reduces use; convenience-driven use increases it. The aggregate environmental outcome is therefore not determined by the technology but by the decisions it is embedded in. The practical conclusions are clear enough to state as principles. Targeting, not flight, delivers the benefit. A drone that broadcasts a product across an entire field at the full rate offers modest environmental gains at best, mainly from avoided soil compaction and, in some settings, lower drift than crewed aircraft. A drone, or any machine, that applies product only where imaging and diagnosis have shown it is needed offers large gains. Settings determine drift. Coarse droplets, low release height, moderate speed and light steady wind make drone drift comparable to ground equipment. These settings must be specified, trained and recorded. Timing determines runoff. No application method prevents runoff if heavy rain falls on a treated field the next day. Weather forecasts, vegetated buffers and avoidance of treatment on saturated or sloping ground near water matter as much for drones as for anything else. Records create accountability. Spray drones log their flight paths, application rates and positions automatically. Those logs are a powerful tool for demonstrating that an application was made correctly, investigating complaints and building the evidence base that regulators need. Treating them as part of the farm record rather than as a manufacturer's diagnostic file is one of the simplest improvements the industry could make. The environmental promise of precision agriculture drones is real, but it is conditional. It comes from closing the loop: imaging that finds where treatment is needed, diagnosis that confirms why, prescription that limits treatment to those places, and application that puts the product there and nowhere else. The final chapter considers the rules and the economics that determine whether farms are able and willing to close that loop. Chapter 8: Rules, Markets and the Farm Balance Sheet Physics and agronomy determine what precision agriculture drones can do. Law, trade policy and money determine what they actually do. A technically sound loop of imaging, diagnosis, prescription and application will not be used if it is illegal, if the aircraft cannot be bought, or if it costs more than it saves. Over the past few years, all three conditions have shifted sharply, in different directions in different parts of the world. This chapter describes the regulatory regimes that govern drone spraying, the market upheaval that reshaped the American industry in 2025, and the arithmetic a farm faces when deciding whether to buy a drone, hire one, or do neither. Two regulators for one machine A spray drone sits at the intersection of two bodies of law that were written separately and for different purposes. Aviation law regulates the aircraft: who may fly it, where, at what height, beyond what line of sight, and with what safety systems. Pesticide law regulates what is in the tank: which products may be applied, at what rates, by what methods, with what buffer zones and protective measures. A drone application must satisfy both. A gap or contradiction between the two, which is common, can make an operation that is safe and sensible on the ground legally impossible or legally uncertain. Table 4 summarizes how several major jurisdictions handle these two layers. Table 4. Regulation of drone pesticide application in selected jurisdictions (status as of late 2026, from regulators' published rules and proposals). Jurisdiction Aviation layer Pesticide layer Overall stance United States FAA Part 107 or Parts 91 and 137, plus exemptions and an agricultural operator certificate EPA labels; states decide whether drones may follow aerial instructions Permitted; paperwork heavy European Union EASA specific category; risk assessment under SORA Aerial spraying banned by Directive 2009/128/EC; 2025 proposal would allow exemptions for qualifying drones Largely prohibited; reform pending Canada Transport Canada RPAS rules; special certificate for heavier drones Label must explicitly permit RPAS application Permitted product by product Brazil Civil aviation and agriculture ministry rules Ministry of Agriculture ordinance 298 of 2021 regulates drone application Permitted; rapid growth China Civil aviation registration and operator rules Permitted with national and provincial guidance Permitted; subsidized; largest market The contrasts are stark. China and Brazil have treated spray drones primarily as agricultural machinery to be promoted. Canada has opened the door cautiously, one product label at a time. The United States permits drone spraying but through a system designed for crewed aircraft and adapted by exemption. The European Union has, until now, largely prohibited it. How the major jurisdictions differ In the United States, the Federal Aviation Administration treats dispensing of pesticides, fertilizers, seed and similar substances from an aircraft as an agricultural aircraft operation under Part 137 of the federal aviation regulations, a rule written for crewed crop dusters. A drone operator must hold an agricultural aircraft operator certificate. Drones under 55 pounds operate under Part 107, the general small drone rule, but need exemptions from the provision barring carriage of hazardous materials and from several Part 137 requirements. Drones of 55 pounds or more, which includes every large spray drone once fueled and loaded, operate under Parts 91 and 137 and need exemptions from parts of the pilot, general operating and agricultural rules, granted under Section 44807 of the federal aviation statute, and the aircraft must be registered like a conventional aircraft. The FAA asks for petitions at least 120 days before operations begin. Over 2023 and 2024, the FAA streamlined this process considerably, issuing exemptions more quickly and with standardized conditions, and approvals for operating several drones simultaneously became available. In June 2025, an executive order directed the agency to accelerate rules for flight beyond visual line of sight, and the FAA subsequently proposed a new rule to normalize such operations. For agriculture, this matters because a single pilot supervising several spray drones over a large field, or a drone treating a field partly out of view behind trees, is a natural mode of operation that the existing framework handles only by special permission. On the pesticide side, the Environmental Protection Agency approves labels, but labels for aerial application were written for crewed aircraft and often specify minimum carrier volumes, boom lengths relative to wingspan and other parameters that do not map cleanly onto a multirotor. The EPA has generally left it to state lead agencies to decide whether drones may follow aerial label directions. The result is a patchwork, in which the same product applied by the same drone may be treated differently in neighboring states. Pesticide labels that explicitly address drones are beginning to appear, but the process is slow because it requires data on drift, deposition and operator exposure for drone-specific conditions. The European Union took the opposite starting point. Article 9 of Directive 2009/128/EC on the sustainable use of pesticides prohibits aerial spraying, with limited derogations for cases where there is no viable alternative or where aerial spraying has clear advantages for health and the environment. Member states have used those derogations sparingly. Germany, for example, has allowed drone spraying in steep-slope vineyards, where the alternative was crewed helicopter application or arduous and hazardous manual spraying, and the Julius Kühn Institute drift trials described earlier were conducted partly to underpin that decision. In December 2025, the European Commission proposed, as part of a food and feed safety simplification package, the first structural amendment to the directive since 2009. The proposal would add a new provision allowing member states to exempt certain types of drones from the general aerial spraying prohibition, provided that delegated acts adopted by the Commission identify drone types whose risks are equal to or lower than those of ground-based equipment for the same use. Products would still require authorization for the application method under the separate pesticide approval regulation. Consultation closed in January 2026, and as of late 2026 the legislation was still moving through the Parliament and Council, with the delegated acts defining qualifying drones the critical next step. Beekeeping and environmental organizations have pressed for strict conditions, including tracking, certification and notification requirements. Whatever its final form, the reform is likely to open Europe to drone application gradually and conditionally, with performance standards defined by drift and exposure data rather than by the type of aircraft. Canada's approach puts the pesticide label first. Health Canada's Pest Management Regulatory Agency requires that a product's label explicitly state that it may be applied by a remotely piloted aircraft system before drone application is legal. Each product therefore needs a specific assessment of efficacy and risk for drone use. On the aviation side, Transport Canada regulates drones under Part IX of its aviation regulations; the heaviest spray drones exceed the weight limit of its standard categories and require a special flight operations certificate. The system is slow, but it produces labels with drone-specific directions, which is exactly what is missing in many other jurisdictions. Brazil has moved quickly. Its Ministry of Agriculture issued a dedicated ordinance in 2021 setting rules for drone application of pesticides and other agricultural inputs, including operator training and registration, record keeping and buffer distances. With large areas of soybean, sugarcane, coffee and citrus, frequent rain that keeps ground equipment out of fields, and a strong crewed aerial application sector to learn from, Brazil has become one of the fastest-growing markets outside China. China's regulatory environment has been the most permissive, and its policy actively supportive. Agricultural drones are eligible for national machinery purchase subsidies, a network of service providers offers spraying on contract, and the government has treated drone application as part of agricultural modernization. The resulting scale, discussed in Chapter 6, has made China the testing ground for drone application technology and the base of its dominant manufacturers. Trade, security and the American market In the United States, the technical and regulatory story was overtaken in 2025 by a security and trade story. DJI, the Chinese manufacturer that dominates agricultural drones worldwide, held an estimated 80 percent of the American spray drone market. Concerns in Congress about data security and dependence on Chinese manufacturing led to provisions in the 2025 defense authorization act requiring a security review of DJI equipment, with the consequence that, absent a finding in its favor, its products would be added to the Federal Communications Commission's covered list of equipment posing national security risks. On December 22, 2025, the FCC went further than many expected and added all new foreign-made drones and their critical components, including flight controllers, motors, batteries and navigation systems, to the covered list. Drones already authorized and in use could continue to fly, and previously authorized inventory could still be sold, but new foreign-made models could no longer receive the equipment authorization required for sale in the United States. DJI filed a petition for review in the federal Ninth Circuit Court of Appeals. Farm organizations, including the Texas Farm Bureau, warned that limited availability of drones and parts could impair pesticide and fertilizer management. The effects were immediate and uneven. Existing DJI fleets kept flying, but their owners faced uncertainty about replacement parts, batteries and future models. American manufacturers, including Hylio in Texas and several smaller firms, saw a sudden opportunity but faced a large gap in price and maturity. Trade press reported that Hylio's entry-level systems started around 20,000 dollars and its larger systems reached about 85,000 dollars, compared with a list price of about 18,000 dollars for DJI's Agras T50 before the restrictions. Operators testifying to the FCC argued that no domestic product yet matched DJI's combination of reliability, capability and price. The American industry's fragility had been exposed months earlier. Guardian Agriculture, a Massachusetts company that built a large electric spray aircraft, the SC1, capable of carrying about 200 pounds of product, became in 2023 the first company to receive FAA approval to operate an electric vertical takeoff and landing aircraft commercially across the country. It raised more than 50 million dollars, but shut down in August 2025 after failing to secure further funding, with few aircraft built and few paying customers. A former executive attributed the failure to insufficient capital, slow regulatory approval and competition from low-cost Chinese drones. The lesson is that building agricultural aircraft is capital intensive and slow, and that domestic production cannot be summoned quickly by trade restrictions alone. For a farmer, the practical consequence in late 2026 is a market in transition: existing DJI aircraft still flying, domestic alternatives growing but more expensive, and uncertainty over parts and future models. That uncertainty belongs in any investment decision. The farm balance sheet Whether a drone pays depends on the farm, and the reasoning matters more than any single number. There are three distinct value streams, and conflating them leads to poor decisions. The first is the value of imaging. A small multispectral drone costs a few thousand dollars, and the main costs are the operator's time and the processing software. Its value comes from scouting efficiency, earlier detection of problems and better-targeted inputs. Because those benefits depend on what the farmer does with the maps, they are hard to estimate in advance, and many farms find that satellite imagery provides most of the scouting value at lower cost, reserving drone flights for specific questions that need higher resolution or cloud-free timing. The second is the value of the drone as an application machine. Here the comparison is with the alternative: owning a ground sprayer, hiring a contractor or booking an aircraft. A drone may let a farm treat wet fields on time, avoid crop damage from wheel tracks, or treat small and awkward fields that a contractor would charge heavily for. Each of these has a value that can be estimated from the farm's own history: how often were treatments delayed by wet ground, and what did those delays cost? The third is the value of targeting: reduced input cost from treating only part of the field. This depends entirely on how patchy the problem is and on whether the farm's workflow actually produces targeted prescriptions. A simple illustrative break-even calculation shows the structure. Suppose a farm buys a large spray drone system, with batteries, charger and generator, for 60,000 dollars, expects to use it for four seasons, and spends another 10,000 dollars per year on battery replacement, maintenance, insurance and licensing. Annual ownership cost is then about 25,000 dollars, before labor. If the drone treats 1,500 hectares per year, the ownership cost is roughly 17 dollars per hectare. If the farm would otherwise pay a contractor more than that, plus labor, the drone pays on application alone. If the drone also enables targeted herbicide treatments that cut herbicide spending by, say, 20 dollars per hectare on 500 hectares, that adds 10,000 dollars of annual savings. If it allows timely fungicide on wet fields that would otherwise have been treated late, the avoided yield loss may dwarf both. These figures are hypothetical, and each farm must substitute its own, but the structure makes clear that the decision hinges on hectares treated per year, the cost of the alternative, and whether targeting is real. Many farms conclude that hiring is better than owning. Contract drone operators spread the capital cost, regulatory burden and training over many clients, and they build expertise in calibration and drift control that an occasional user will not. The rapid growth of contract spraying services in the United States, Brazil and China reflects that logic. Across all of these jurisdictions, regulation has largely been organized around the aircraft rather than the outcome. It asks what kind of machine is flying, and how heavy it is, rather than how much product reaches the target, how much drifts, and how much is applied in total. A regime organized around outcomes would reward the behaviors that make drones environmentally valuable: low drift settings, logged applications, targeted prescriptions and trained operators. The European proposal's test, that a drone type must present risks equal to or lower than ground equipment, points in that direction. So does Canada's insistence on drone-specific label directions. The conclusion considers what it would take to align the whole system, from sensor to regulation, around the loop. Conclusion The argument of this book has been that the value of a precision agriculture drone lies in a closed loop, not in the aircraft or the image. Light is measured; the measurement is calibrated into a trustworthy signal; the signal is read as a symptom and diagnosed on the ground; the diagnosis becomes a prescription; and the prescription is applied by a machine whose dose and placement are known and recorded. Each link has physical, biological or legal limits, and the benefits that are claimed for drones, less chemical, less runoff, earlier intervention and better yields, appear only when every link holds. If that argument is right, several consequences follow, and they are not the ones that dominate most discussion of the technology. The scarce resource is interpretation, not flight The aircraft have become cheap, capable and easy to fly. An integrated multispectral drone that produces a calibrated red-edge map of 200 hectares in a single flight costs less than many pieces of farm equipment, and spray drones capable of carrying 100 liters are commercial products. What has not become cheap is the judgment needed to interpret a map: to recognize a calibration artifact, to tell a soil pattern from a nitrogen pattern, to know when NDVI has saturated, to decide whether a warm block of vines is thirsty or diseased. That judgment lives in agronomists, crop advisers and experienced farmers, and there are not enough of them. The practical implication is that investment should flow toward interpretation at least as much as toward hardware. Training programs for drone operators that teach flight but not agronomy produce people who can make maps but not use them. Software that presents colored maps without uncertainty, without warnings about saturation or soil background, and without prompts to ground-truth encourages exactly the misreadings that waste money. The most valuable drone services are not those with the most aircraft but those that pair flights with agronomic advice and follow-up. Targeting is the environmental technology The environmental case for drones is often made as though the aircraft itself were a green technology. It is not, or only marginally. A drone that broadcasts a herbicide across an entire field at the label rate delivers roughly the same mass of active ingredient to the same land as a ground sprayer. It avoids wheel compaction, and with the right settings its drift can be comparable to ground equipment, but it does not by itself reduce the load available for runoff. The environmental gain comes from targeting, from spraying 30 percent of a field instead of 100 percent because imaging and diagnosis showed where the weeds or the disease actually were. That gain is not unique to drones. Camera-guided ground sprayers have demonstrated it at a scale of millions of acres, with herbicide savings well above half in many fields. Drones contribute by enabling targeting in places and moments ground sprayers cannot reach, and by providing the imagery that defines where to target. Policy that wants to reduce pesticide loads should therefore reward targeting itself, by whatever machine, rather than favoring a particular aircraft. Measures that recognize documented spot or zone application in pesticide registration, in agri-environmental payments, or in buffer zone rules would do more for water quality than any rule about rotor configuration. Records, rules and supply chains Spray drones record where they flew, how high, how fast and how much they applied, automatically and in detail. Imaging drones record calibrated maps of crop condition before and after treatment. Together, these records could provide something agriculture has rarely had: an accurate, field-scale account of what was applied, where, and with what result. At present, much of that information sits in manufacturers' cloud services or on controllers, unused. Making it part of the farm record, available to the farmer, to advisers and, under suitable conditions, to regulators, would serve several purposes at once. It would allow farmers to check whether prescriptions worked. It would let operators demonstrate compliance with drift and buffer requirements when a neighbor complains. It would give regulators the field evidence they need to write drone-specific label directions and to judge whether drones meet the standard of risk equal to or lower than ground equipment. And it would answer the legitimate demand from beekeepers and environmental groups for transparency. Few changes would cost so little and do so much to build trust in the technology. Regulators currently govern drone spraying through two frameworks, aviation and pesticides, each designed for an older technology. Both can be improved by asking outcome questions. For aviation, the question is whether the operation is safe for people and property, which for a 100-kilogram aircraft flying three meters above a field away from roads and houses is a different question from the one posed by a crewed crop duster. For pesticides, the question is how much product reaches the target and how much goes elsewhere, which depends on droplet size, release height, speed and wind, all of which can be specified and logged. The European reform under negotiation, the Canadian insistence on drone-specific labels and the gradual normalization of beyond-visual-line-of-sight operations in the United States all point toward this kind of performance-based regulation. The remaining step is to connect it to targeting and records, so that an operator who demonstrates low-drift settings, logged applications and targeted prescriptions faces fewer obstacles than one who does not. The American experience of 2025 showed that a precision agriculture system is only as resilient as its supply chain. When most of a country's spray drone fleet comes from a single foreign manufacturer, a security decision can freeze replacement and upgrade overnight. Domestic manufacturing cannot be created quickly, as the failure of a well-funded American startup demonstrated, and farmers bear the cost of the transition in higher prices and uncertainty. The durable answer is not to depend on any single aircraft but on open standards. If prescription files, flight logs, sensor calibration data and controller interfaces follow common, documented formats, then the loop can survive a change of aircraft. A farm whose maps, prescriptions and application records are portable can switch suppliers without losing years of data. Standards bodies and public research institutes are better placed than manufacturers to promote that portability, and governments concerned about supply security have every reason to support it. What a farmer should do on Monday For the individual farmer or adviser, the book's argument reduces to a short practice. Start with a question, not with a flight: what decision would better information change? Choose the sensor and resolution for that question. Calibrate every flight and fly at consistent times. Read maps as symptoms and walk to the zones they flag. Use reference strips so that each zone can be compared with what unconstrained growth looks like. Write prescriptions only for causes that the input being varied can fix. Apply with settings chosen for low drift, and keep the logs. Leave comparison strips and check, at harvest, whether the prescription paid. None of this is exotic. It is the ordinary discipline of agronomy, applied to a new and powerful set of tools. The drones make it possible to see more, earlier, and to act more precisely than ever before. Whether they make farming more productive and less polluting depends on whether the people using them close the loop. Hashtags: #PrecisionAgriculture #AgriculturalDrones #DroneFarming #MultispectralImaging #VariableRateApplication #SmartFarming #PrecisionFarming #AgriculturalTechnology #AgriTech #RemoteSensing #CropMonitoring #CropHealthAssessment #VegetationIndices #NDVI #NDRE #ThermalImaging #CropStressDetection #DroneMapping #UAVTechnology #AgriculturalRemoteSensing #PrecisionSpraying #TargetedSpraying #VariableRateTechnology #CropManagement #SoilAndCropAnalysis #IrrigationManagement #SustainableAgriculture #AgriculturalAutomation #DataDrivenFarming #DigitalAgriculture

Download the Book (PDF): Introduction Show a three-year-old a single picture of an okapi and she will pick out the next okapi she sees, in a zoo, a cartoon, or a blurry photograph taken from behind. Show a standard deep neural network the same single picture and it learns almost nothing. The network that classifies a thousand kinds of object with superhuman accuracy needed, during its training, hundreds or thousands of labelled examples of each one. Give it a new category with five examples and it will either ignore them or memorise them, and in neither case will it generalise. That gap, between the child who learns from one example and the machine that needs a thousand, is the subject of this book. It is not a gap in raw computing power. The network has far more parameters than the child has words, and it can process images faster than any person. The gap is in something harder to name: the child arrives at the okapi already knowing an enormous amount about how animals look, which features of an animal are stable and which are accidents of lighting, and how a new name attaches to a new kind of thing. She has learned how to learn about animals. The network, trained once on a fixed problem, has learned only that problem. Meta-learning is the branch of machine learning that tries to close this gap directly. Its premise is simple to state. Instead of training a model to solve one task, train it on many related tasks, and measure success not by how well it does on any of them but by how quickly it masters a new one. The thing being learned is not a classifier or a policy but a learning procedure: a starting point, a way of comparing examples, an update rule, or a pattern of computation that turns a handful of new data points into a competent model. The phrase that stuck to the field, "learning to learn", is accurate. The controlling idea This book argues one thing, and every chapter returns to it. Fast learning is always paid for with slow learning. Whenever a system adapts to a new task from a few examples, it is drawing on a prior, an inductive bias, that was built up beforehand from a great deal of experience. Meta-learning is the discipline of deciding where that prior should live and how to acquire it from a distribution of tasks. The different families of method, which can look like an unrelated zoo of acronyms, are best understood as different answers to a single question: should the accumulated experience be stored in a good starting point for ordinary training, in a space where similar things sit close together, in a learned update rule, or inside the forward computation of a network that simply reads the examples and responds? Seen this way, several things that otherwise look puzzling fall into place. It explains why Model-Agnostic Meta-Learning, the algorithm that gave the field its most famous name, works the way it does, and why careful later analysis found that much of its power came from learning good features rather than from learning to adapt quickly. It explains why a series of papers around 2019 and 2020 found that plain pretraining followed by a simple classifier matched or beat elaborate meta-learning algorithms on the benchmarks those algorithms had been designed for. And it explains why the in-context learning of large language models, which no one set out to design as a meta-learner, behaves so much like one: a model trained on an immense and varied stream of text has met, implicitly, an immense and varied distribution of tasks. The same idea also sets the limits of the enterprise. A prior learned from one distribution of tasks helps only on tasks that resemble that distribution. Meta-learning does not produce general intelligence that can pick up anything from a single example; it produces specialists in a family of problems, and the quality of the specialist depends on how well the family was chosen. The most consequential decisions in building a meta-learning system are therefore not about the algorithm at all. They are about what tasks to train on, how to split them, and how to measure whether the system has learned to learn or only learned to memorise. What the book covers The first chapter sets out the problem: why learning from few examples is hard in principle, what inductive bias means, and how the idea of learning to learn developed from psychology experiments on monkeys in the 1940s through the machine learning of the 1990s to the deep learning revival. The second chapter introduces the central conceptual move of modern meta-learning, which is to treat tasks themselves as training data. It explains the episodic setup of support and query sets, the "N-way, K-shot" vocabulary, the two nested loops of optimisation that nearly every method shares, and the benchmarks on which the field has measured itself. Chapters three to six survey the main families of method, each organised around where it places the learned prior. Chapter three covers metric-based methods, which learn a space in which classification reduces to comparing distances, and which include Siamese networks, Matching Networks and Prototypical Networks. Chapter four is devoted to Model-Agnostic Meta-Learning itself, explained step by step: the inner and outer loops, the second-order gradient that makes it expensive, the first-order shortcuts, and what the method is really learning. Chapter five examines the large family of refinements and reinterpretations that followed, including probabilistic versions, implicit gradients, and the failure modes of meta-overfitting and memorisation. Chapter six turns to methods that learn the learning procedure outright: memory-augmented networks, recurrent meta-learners, learned optimisers, and meta-reinforcement learning, in which an agent learns how to explore. Chapters seven and eight deal with the upheaval that pretraining at scale brought to the field. Chapter seven tells the story of the baseline results that showed a well-trained feature extractor was most of what few-shot learning required, and what that finding does and does not imply. Chapter eight examines in-context learning in large transformers as a form of meta-learning that emerged without being asked for, and reviews the research that has tried to pin down what algorithm these models are running when they learn from examples in a prompt. Chapter nine is practical: how to design task distributions, evaluate honestly, and decide which kind of meta-learning, if any, a real problem needs, illustrated with applications in robotics, drug discovery, personalisation and language. What the book leaves out Some things are deliberately set aside. The book does not attempt a complete catalogue of the hundreds of published meta-learning variants; it discusses the ones that changed how the field thinks. It treats neural architecture search and automated hyperparameter tuning, which are sometimes filed under meta-learning, only briefly, because their concerns are mostly about search efficiency rather than about learning from little data. It does not reproduce mathematical derivations in full. Where an equation helps, the idea behind it is explained in words, and a reader who wants the formal treatment will find it in the papers listed in the Notes and Further Reading. The reader assumed here is someone who knows roughly what a neural network is and what it means to train one with gradient descent, but who need not be a specialist. A practitioner who has trained models will find the later chapters most immediately useful. A reader coming from outside the field should be able to follow the whole argument, because the vocabulary is introduced as it is needed and the mechanisms are described in plain sentences. Why it matters now It is fair to ask whether meta-learning still matters as a separate subject when the largest language and vision models already learn new tasks from a few examples placed in their input. The answer this book gives is that it matters more, not less, because the questions meta-learning formalised are exactly the questions those models raise. What distribution of experience produces a system that adapts well? When does adaptation genuinely generalise and when is it retrieval of something already seen? How should adaptation be divided between changing a model's weights and changing only its inputs? Researchers studying test-time training, prompt-based adaptation, and the emergence of in-context learning are working with the concepts of support sets, inner loops, and task distributions whether or not they use the words. There are also many settings in which the giant pretrained model is not available or not suitable: a robot that must adjust to a damaged leg in seconds, a laboratory with forty measured molecules for a new protein target, a hospital whose patients differ from those in any public dataset, a phone that must personalise a model without sending data to a server. In these settings the ideas in this book are not historical background. They are the tools. The child and the okapi make a useful reference point to keep in mind. She is not a counterexample to the principle that learning requires data. She is its best illustration. Her one-shot learning rests on years of looking at the world, and on a body that evolution shaped over far longer still. Meta-learning is the attempt to give machines the same kind of foundation on purpose, and to understand exactly what that foundation consists of. Chapter 1: The Problem of Learning from Little Every learning system faces the same logical difficulty. It sees a finite set of examples and must say something about cases it has not seen. Nothing in the examples themselves determines the answer. Five photographs of a new bird species are consistent with the rule "birds of this species are brown", with the rule "birds of this species are photographed against leaves", with the rule "any image whose top-left pixel is dark", and with infinitely many other rules that agree on those five images and disagree everywhere else. To choose among them, a learner has to bring something to the data that the data does not supply. That something is called inductive bias, and understanding it is the first step towards understanding meta-learning. Why data alone is never enough Philosophers have known since David Hume that induction cannot be justified by the observations it is drawn from. Machine learning made the point precise. The "no free lunch" theorems published by David Wolpert in the 1990s showed that, averaged over all possible problems, no learning algorithm outperforms any other, including random guessing. An algorithm that does well on some problems must do badly on others. The practical lesson is not that learning is hopeless but that every successful learner is successful because its assumptions match the problems it actually meets. Ordinary deep learning hides this fact behind large datasets. A convolutional network trained on a million labelled images does carry strong assumptions: that nearby pixels are related, that the same pattern means the same thing wherever it appears in an image, that useful features are built hierarchically from simpler ones. These assumptions are written into its architecture by human designers. But they are weak relative to the problem, and the network makes up the difference with data. With enough examples, the space of rules consistent with the training set shrinks until most of the rules that remain generalise acceptably. Take the data away and the situation changes completely. With five examples per class, a network with millions of free parameters can fit the training set in countless ways, almost all of them wrong. Standard remedies help only a little. Regularisation, such as penalising large weights, expresses a generic preference for simpler functions, but "simple" in weight space has little to do with "sensible" in the world. Data augmentation, such as flipping and cropping images, adds a specific kind of prior knowledge about which transformations do not change a label, but it requires a human to know and encode that knowledge in advance. The few-shot problem therefore forces a direct question. If a learner must generalise from a handful of examples, its inductive bias must be strong, and it must be the right one. Where can a strong, correct bias come from, if not from a human designer writing it by hand? Bias, variance and the price of a strong prior Statisticians describe the same difficulty in the language of bias and variance. A learner's errors on new data come from two sources. Bias is error from assumptions that are wrong: a learner that insists every relationship is a straight line will be systematically off when the truth is curved, no matter how much data it sees. Variance is error from sensitivity to the particular sample: a learner flexible enough to fit any curve will, given five points, fit a curve that depends heavily on which five points it happened to get, and a different five would give a very different answer. With little data, variance dominates. The only way to reduce it is to restrict what the learner can conclude, which means accepting stronger assumptions. A strong prior narrows the range of answers the learner will consider, so that five points are enough to choose among the remaining candidates. The danger is plain: if the prior is wrong, the narrowed range may not contain the right answer, and the learner will confidently converge on a wrong one. A learner that assumes all curves are sine waves will learn a new sine wave from five points and will be hopeless on a parabola. This is the trade that few-shot learning cannot escape. A learner can be fast or it can be general, and it buys speed by giving up generality in the specific directions where its prior rules things out. The goal is therefore not to remove assumptions but to have the right ones, strong in the directions where the tasks one will actually meet are constrained, and loose in the directions where they vary. Hand-designing such assumptions works for a few well-understood problems. For most problems nobody knows, in advance, exactly which directions are constrained and which are free. The answer from experience The answer that meta-learning gives is that the bias can itself be learned, from experience with other, related problems. This is not an invention of computer science. It is an old observation about animals and people. In 1949 the psychologist Harry Harlow published a paper titled "The Formation of Learning Sets", describing experiments with rhesus monkeys. Each monkey was shown two objects, one of which had a raisin or peanut hidden beneath it. The monkey's job was to learn which object paid off. On the first such problem, the monkeys learned slowly, by trial and error over many attempts. Harlow then gave them a new problem with two new objects, and another, and another, running to hundreds of distinct problems. As the series went on, the monkeys got faster. Eventually they solved a new problem after a single trial: if the first object they chose had food under it, they stuck with it; if not, they switched. They had learned not a fact about any particular pair of objects but a strategy for the whole family of problems. Harlow called this a learning set, and he described it, in a phrase that would later be borrowed by machine learning, as learning how to learn. Harlow's monkeys show the structure that every meta-learning system shares. There is a family of tasks that differ in their details but share an underlying regularity. There is a slow process, extended over many tasks, that extracts that regularity. And there is a fast process, operating within a single new task, that uses the extracted regularity to learn from very little. The monkeys' one-trial learning was not magic. It was the payoff of hundreds of earlier problems. Human children show the same pattern on a much larger scale. Developmental psychologists have documented that young children, once they have learned a certain number of object names, begin to generalise new names by shape rather than by colour or texture. Having learned that "cup" and "ball" and "shoe" pick out things of a particular shape, a child given a new word for a new object will extend it to other objects of the same shape. This shape bias is not present at birth; it emerges from experience with early vocabulary, and once present it makes subsequent word learning much faster. It is a learned inductive bias of exactly the kind meta-learning aims to produce. A short history of learning to learn in machines The idea that a machine could improve its own learning procedure appeared early in artificial intelligence and returned several times before it became a mainstream research area. In 1987 Jürgen Schmidhuber's diploma thesis at the Technical University of Munich described self-referential learning systems that could modify their own learning algorithms, a recursive idea he continued to develop through the following decades. In 1991 Yoshua Bengio, Samy Bengio and Jocelyn Cloutier proposed learning a synaptic learning rule: rather than fixing the rule by which a neural network's connections change, they parameterised the rule and optimised it so that networks using it would learn a family of tasks well. This is recognisably the modern idea of a learned optimiser. Through the 1990s the theme developed under several names. Rich Caruana's work on multitask learning, culminating in an influential 1997 paper, showed that training one network on several related tasks at once could improve performance on each, because the tasks shared useful internal representations. Sebastian Thrun and Lorien Pratt edited a 1998 book titled Learning to Learn, collecting work on transferring knowledge across tasks. Jonathan Baxter published a theoretical framework in 2000, "A Model of Inductive Bias Learning", that made the central idea mathematically explicit: if tasks are drawn from some environment, a learner can use a sample of tasks to choose a hypothesis space that will contain good solutions for future tasks from the same environment, and the number of examples needed per new task falls as the number of training tasks rises. Baxter's framework deserves a moment of attention because it states the bargain behind all later work. The learner trades examples per task for number of tasks. If you want to learn a new task from five examples, you had better have seen many tasks. The prior does not come free; it is paid for in experience with the task distribution. In 2001 Sepp Hochreiter, Steven Younger and Peter Conwell showed that a recurrent neural network, specifically a Long Short-Term Memory network, could be trained to act as a learning algorithm. Fed a sequence of input-output pairs from a function, the network learned to predict outputs for new inputs of that function, and after training on many functions it could do so for new functions it had never seen. The network's weights were fixed during this process. All the learning on a new task happened in its internal activations. This experiment anticipated, by nearly two decades, the in-context learning that large language models would later display. These early efforts were limited by the computing power and the network architectures of their time. The idea was sound but the tools could not yet make it work on rich problems like recognising images. That changed after 2012, when deep convolutional networks began to dominate computer vision, and researchers found themselves with powerful learners that were notably bad at learning from little. Omniglot and the human benchmark The modern revival of meta-learning owes a great deal to one dataset and one challenge. In 2015 Brenden Lake, Ruslan Salakhutdinov and Joshua Tenenbaum published a paper in Science titled "Human-level concept learning through probabilistic program induction". They introduced the Omniglot dataset: 1,623 handwritten characters drawn from 50 alphabets, both real and invented, each character drawn by 20 different people. Omniglot was designed as a counterpart to MNIST, the classic handwritten digit dataset. Where MNIST has a few classes with thousands of examples each, Omniglot has many classes with very few examples each. Lake and his colleagues asked whether a machine could learn a new character from a single example, as people readily do, and could pass a kind of visual Turing test: generating new examples of a character that human judges could not distinguish from human drawings. Their own system, called Bayesian Program Learning, represented each character as a small program describing the strokes used to draw it. It learned, from a background set of alphabets, the kinds of strokes people use and how they combine, and then explained a new character by inferring the most probable program that could have produced it. On one-shot classification of new characters it matched human performance and outperformed the deep learning systems available at the time. The paper was a challenge as much as a result. Its authors argued that human concept learning depends on rich, structured prior knowledge, about how characters are produced, about causality and composition, and that pattern-recognising neural networks lacked it. The deep learning community took up the challenge in a particular way. Rather than hand-building structured priors, researchers asked whether a neural network could acquire the necessary prior by being trained on the background alphabets in the right way. Within a year, several methods had reached very high one-shot accuracy on Omniglot, and the benchmark became a standard proving ground. Whether those networks had acquired anything like the compositional understanding Lake described is a separate question, and one the book will return to; but the shape of the modern research programme was set. Four places a prior can live To organise what follows, it helps to look at a learning system as a pipeline and ask where accumulated experience could be stored. A learner meeting a new task starts from some initial state, processes the new examples somehow, and ends up with a model that makes predictions. Prior experience can influence each stage. First, it can shape the starting point. If the initial parameters of a network already sit in a region from which a few steps of gradient descent on any task in the family lead to a good solution, then ordinary training becomes few-shot training. This is the idea behind Model-Agnostic Meta-Learning and its relatives. Second, it can shape the representation. If prior experience has produced a way of describing inputs in which members of the same class lie close together and members of different classes lie far apart, then a new class can be recognised by comparing a new input with the few labelled examples. Nothing needs to be trained on the new task at all. This is the idea behind metric-based methods. Third, it can shape the update rule. Gradient descent with a fixed learning rate is a generic procedure that knows nothing about the problem. A learned update rule, which takes in gradients or examples and outputs parameter changes, can encode knowledge about which directions in parameter space are likely to be useful. This is the idea behind learned optimisers. Fourth, it can shape the computation itself. A network can be trained to take an entire set of labelled examples and a new query as input, and to output a prediction directly, with the learning on the new task happening entirely in its forward pass. This is the idea behind memory-augmented and recurrent meta-learners, and it is, in a loose but important sense, what large language models do when they learn from examples in a prompt. These four options are not mutually exclusive, and many successful methods combine them. But they give a map. Each family of meta-learning methods makes a characteristic bet about where the prior should live, and that bet determines how the method adapts, how much computation adaptation needs, and what kinds of new task it can handle. What success would look like It is worth being clear at the outset about what meta-learning can and cannot be expected to deliver. A meta-learned system is good at learning new tasks that come from the same distribution as its training tasks. A model meta-trained on handwritten characters learns new handwritten characters quickly. It does not learn to recognise new bird species quickly, because nothing in the character tasks taught it about birds. The prior is specific to its environment, in Baxter's sense. This might sound like a limitation peculiar to machines, but it is equally true of people. An expert radiologist learns to recognise a new kind of lesion from a few cases far faster than a novice, because years of reading scans have given her a prior over what lesions look like. The same radiologist has no special advantage in learning a new chess opening. Fast learning is domain-specific because the priors that make it possible are domain-specific. The ambition of the field is therefore not a universal one-shot learner. It is a reliable method for turning experience with a family of tasks into fast learning on new members of that family, and an understanding of how the breadth and structure of the family determine how far the fast learning reaches. The large pretrained models of recent years push the second question to an extreme, by training on a family of tasks so broad that it covers much of what people write about and photograph. Whether that breadth amounts to generality, or only to a very large family, is one of the questions this book will try to answer. Before any of that, though, the field needed a way to turn the vague idea of "experience with related tasks" into something a computer could optimise. The next chapter describes how that was done. Chapter 2: Tasks as Training Data Ordinary supervised learning has a simple structure. There is a dataset of examples, each an input paired with a label. A model is trained to map inputs to labels on part of the data and tested on the rest. The thing being generalised across is the example: the hope is that a model which works on the training examples will work on new examples drawn from the same distribution. Meta-learning moves this whole structure up one level. The thing being generalised across is the task. A meta-learning system is trained on a collection of tasks and tested on new tasks it has never seen. Each task comes with its own small dataset, and within each task there is still a split between examples used to learn and examples used to check. This doubling of the structure is the single most important conceptual move in the field, and nearly every confusion about meta-learning can be traced to losing track of which level one is talking about. Episodes, support sets and query sets The standard formulation came into focus in 2016 with a paper from Oriol Vinyals and colleagues at Google DeepMind, "Matching Networks for One Shot Learning". Its guiding principle, which the authors stated plainly, was that test and train conditions must match. If a system will be asked at test time to learn a new class from one example, it should be trained by repeatedly being asked to learn new classes from one example. To do this, training is organised into episodes. Each episode is a small, self-contained learning problem. In the classification setting, an episode is built by first choosing N classes at random from a large pool of training classes, then sampling K labelled examples of each class. These N times K examples form the support set: the small training set for this episode. A further batch of examples from the same N classes forms the query set: the test set for this episode. The model is given the support set, allowed to learn from it in whatever way its design permits, and then asked to label the query examples. Its error on the query set is the training signal. The jargon of "N-way, K-shot" comes directly from this construction. A 5-way 1-shot task asks the model to distinguish among five classes given one example of each. A 20-way 5-shot task gives five examples each of twenty classes. Chance accuracy on an N-way task is one in N, so 20 per cent on a 5-way task, and reported results should always be read against that floor. Two features of episodic training are easy to overlook and matter a great deal. The first is that the labels within an episode are arbitrary. If an episode contains a dog, a car, a tulip, a violin and a bridge, they might be assigned labels 0 to 4 in any order, and in the next episode a different set of classes takes the same labels. The model therefore cannot learn that "dog" means label 2. It must learn, within each episode, which label goes with which kind of thing, using only the support set. This shuffling is what forces the model to learn a procedure rather than a fixed mapping. Later chapters show what happens when the shuffling is absent or insufficient: the model memorises the training tasks and the meta-learning collapses into ordinary learning. The second is that the classes used at test time are disjoint from those used in training. If the meta-training classes include dogs and cars, the meta-test classes must not. Otherwise the test measures how well the model remembers old classes, not how well it learns new ones. The pool of classes is split before any episodes are constructed, into meta-training classes, meta-validation classes for choosing hyperparameters, and meta-test classes for the final evaluation. Two loops Once training is cast as episodes, nearly every meta-learning method can be described as two nested processes, usually called the inner loop and the outer loop. The inner loop is learning within a single task. Given a support set, the inner loop produces a task-specific model. What the inner loop is depends on the method. It might be several steps of gradient descent from a learned starting point. It might be computing the average embedding of each class's examples. It might be a recurrent network reading the support set one example at a time. In every case it is the fast, data-poor learning that the system is supposed to be good at. The outer loop is learning across tasks. It adjusts whatever is shared across episodes, the meta-parameters, so that the inner loop works well. The quality of the inner loop's output is measured on the query set, and the outer loop changes the meta-parameters to improve that measurement, averaged across many episodes. The outer loop is slow and data-hungry. It may process hundreds of thousands of episodes. Written formally, this is a bilevel optimisation problem. The outer problem chooses meta-parameters to minimise the expected query loss across tasks. Inside it sits an inner problem, which for each task produces task-specific parameters from the support set and the meta-parameters. The outer objective depends on the meta-parameters partly through the inner problem's solution, which is what makes meta-learning computationally interesting: to know how changing the meta-parameters affects the final query loss, one must know how it affects the result of learning on the support set. The two-loop picture is the clearest way to compare methods. For each method, ask two questions. What does the inner loop do with the support set? And what does the outer loop adjust? For metric methods, the inner loop computes class representatives in an embedding space and the outer loop trains the embedding. For MAML, the inner loop runs gradient descent and the outer loop trains the starting point. For learned optimisers, the inner loop applies a learned update rule and the outer loop trains that rule. For black-box meta-learners, the inner loop is a forward pass through a network and the outer loop trains that network's weights. Generalisation at two levels The doubled structure brings with it two separate notions of generalisation, and a meta-learner can fail at either. The first is generalisation within a task: from the support set to the query set. A model that adapts to the five support examples by memorising them, without capturing what makes them members of their classes, will do badly on the query examples. This is the familiar overfitting of ordinary learning, compressed into a single episode. The outer loop guards against it, because it is trained only on query performance, so any inner-loop behaviour that fits the support set without generalising is penalised. The second is generalisation across tasks: from meta-training tasks to meta-test tasks. A meta-learner can do well on query sets for every task drawn from its training classes and still fail on tasks built from new classes, if what it learned depends on the specific training classes rather than on the general skill of learning from a support set. This is the failure that the class split between meta-training and meta-test is designed to detect. Theoretical work following Baxter has made the relationship between the two levels precise in various settings. Roughly, the error on a new task is bounded by terms that shrink as the number of training tasks grows and terms that shrink as the number of examples per new task grows. A meta-learner trained on few tasks cannot guarantee a good prior however many examples each training task contained, because a prior estimated from few tasks is itself unreliable. This is why the number and variety of training tasks, not just the total amount of data, is the quantity that governs how well a meta-learner transfers, a point that will become central in later chapters. A worked example in regression The classification setting can make episodes sound more exotic than they are. A simpler example, used in the original MAML paper and many tutorials since, makes the structure concrete. Imagine a family of tasks, each of which is to predict a sine wave. Every task has its own amplitude, chosen between 0.1 and 5, and its own phase, chosen between zero and pi. Inputs are numbers between minus five and five. The output is the amplitude times the sine of the input plus the phase. All tasks share the same basic shape but each is a different curve. An episode consists of choosing one such curve at random, sampling, say, five points on it as the support set, and another batch of points as the query set. A model sees the five support points and must predict the curve everywhere. Train an ordinary neural network on just those five points and it will fit them, but between and beyond them it will produce whatever smooth function its initialisation happens to favour, which usually looks nothing like a sine wave. Train a single network on data pooled from many such curves, ignoring which curve each point came from, and it will learn the average curve, which is close to a flat line at zero, because positive and negative phases cancel out. Neither approach exploits the real structure, which is that every task is a sinusoid and only two numbers need to be inferred. A meta-learner trained on many episodes of this kind learns that structure. After meta-training, given five points from a new curve, it produces something close to the right sinusoid, including in regions where it saw no data. It has learned a prior over curves, and five points are enough to pin down the two free parameters within that prior. This toy problem captures everything essential: a task distribution with shared structure, a few-shot inner problem that is ill-posed on its own, and an outer loop that extracts the structure and makes the inner problem well-posed. Benchmarks and what they measure A field moves at the speed of its benchmarks, and meta-learning's benchmarks shaped both its progress and its blind spots. Table 1 summarises the four most influential image-classification benchmarks. Table 1. Major few-shot image classification benchmarks. Benchmark Content Classes (train/val/test) Main test Omniglot (Lake et al., 2015) 1,623 characters from 50 alphabets, 20 drawings each Background vs evaluation alphabets One-shot learning of new characters miniImageNet (Vinyals et al., 2016) 100 ImageNet classes, 600 images each 64 / 16 / 20 Few-shot natural images tieredImageNet (Ren et al., 2018) 608 ImageNet classes in 34 categories 351 / 97 / 160, split by category Semantically distant test classes Meta-Dataset (Triantafillou et al., 2020) 10 source datasets, from ImageNet to fungi 8 sources in training, 2 held out Varied ways, shots and domains Source: the cited papers. Omniglot was quickly saturated. Within two years of its introduction, several methods reported accuracies above 98 per cent on 5-way one-shot tasks, leaving little room to tell methods apart. miniImageNet, a subset of the ImageNet dataset with small images, became the workhorse benchmark for several years; most of the methods discussed in the next three chapters reported their headline numbers on it. Its weakness is that its training and test classes, though disjoint, are all drawn from the same broad pool of everyday objects and animals. A test class such as a particular dog breed may have close relatives among the training classes. tieredImageNet addressed this by splitting classes according to ImageNet's category hierarchy, so that, for example, all musical instruments might be in the test set and none in training. This makes the test tasks more genuinely novel. Meta-Dataset, introduced by Eleni Triantafillou and colleagues, went further. It combined ten datasets of very different character, including drawings, textures, aircraft, birds, fungi, flowers and traffic signs, and generated episodes with varying numbers of classes and examples rather than a fixed N and K. Two of its sources, traffic signs and the MSCOCO object dataset, were held out entirely from training, so that a model was tested on domains it had never seen. Meta-Dataset was explicitly designed to be closer to the messy conditions of real applications, and, as chapter seven describes, its results were among the most influential in reshaping the field. The questions benchmarks hide Benchmarks of this kind standardise one particular version of the problem, and it is worth stating the assumptions they build in. They assume that classes at test time are balanced and that the number of classes is known in advance. Real few-shot problems often have neither property. A wildlife camera may produce hundreds of images of common animals and three of a rare one. They assume that tasks at meta-test time come from the same distribution as tasks at meta-training time, apart from the specific classes. When this assumption breaks, as it does in Meta-Dataset's held-out domains, many methods degrade sharply. They measure accuracy averaged over many randomly sampled episodes. That average conceals large variation between episodes. A method that averages 60 per cent might score 90 per cent on easy episodes and 30 per cent on hard ones. Papers typically report a 95 per cent confidence interval over several hundred or thousand test episodes, which is a measure of how precisely the average was estimated, not of how much individual episodes vary. Finally, they say nothing about where the representations used by a method came from. A method allowed to pretrain on all of ImageNet, or on billions of web images, before meta-training on miniImageNet has a very different starting point from one trained only on miniImageNet's 64 training classes. For several years this distinction was handled by convention rather than by rule, and comparisons across papers were sometimes less fair than they looked. When large pretrained models arrived, the question of what counts as prior experience became central rather than incidental. Beyond classification Although classification dominated the benchmarks, the episodic structure applies to any learning problem. In regression, as in the sine wave example, each task is a function and the support set is a handful of input-output pairs. In reinforcement learning, each task is an environment or a goal, such as a maze with a different layout or a robot that must run at a different target speed, and the support set is a few episodes of experience in that environment. In language, each task might be a different text classification problem, a different language, or a different user's writing style. In scientific settings, each task might be a different protein target, with the support set being a few dozen measured molecules. The framework transfers cleanly because it makes only two demands: that there be a distribution of tasks with shared structure, and that each new task supply a small amount of data. The method then extracts the shared structure in its outer loop and uses it to make the small amount of data sufficient in its inner loop. What the framework cannot do is create shared structure where there is none. If the tasks in a collection have nothing useful in common, meta-training will learn nothing that helps on a new one. The design of the task distribution, which tasks to include, how varied they should be, how to divide them, is therefore part of the learning problem rather than a preliminary to it. That point recurs throughout the book, and chapter nine treats it directly. With the episodic framework in place, the question becomes what to put in the inner loop. The simplest and in many ways most robust answer is to make the inner loop almost trivial: to learn a representation so good that learning a new task requires nothing more than measuring distances. That is the subject of the next chapter. Chapter 3: Learning to Compare There is a way to recognise a new kind of thing that requires no training at all. Given one labelled example of each of several categories, and a new item to classify, you can simply ask which labelled example the new item most resembles. This is nearest-neighbour classification, one of the oldest methods in pattern recognition, and it learns a new class instantly: adding a class means adding an example to the reference set. Its weakness has always been the meaning of "resembles". Measured pixel by pixel, two photographs of the same dog in different poses can be less similar than a photograph of the dog and a photograph of a brown sofa. Nearest-neighbour classification is only as good as its notion of distance. Metric-based meta-learning starts from the observation that this notion of distance can be learned, and that learning it across many tasks produces exactly the kind of transferable prior that few-shot learning needs. These methods place the accumulated experience in a representation, an embedding space, and keep the inner loop as simple as possible. Siamese networks and the learned similarity The first deep metric method to be widely cited in the few-shot literature was the Siamese network applied to one-shot recognition by Gregory Koch, Richard Zemel and Ruslan Salakhutdinov in 2015. The Siamese architecture itself was older, going back to work on signature verification in the early 1990s. It consists of two identical copies of a neural network, sharing all their weights, each processing one of a pair of inputs. Their outputs are compared, and the comparison produces a score for whether the two inputs belong to the same class. Koch and colleagues trained such a network on pairs of Omniglot characters from the background alphabets, labelled "same" or "different". The network learned features that made same-class pairs score high and different-class pairs score low. At test time, faced with a one-shot task on new alphabets, they compared the query character with each of the support characters and assigned the label of the highest-scoring match. This was not yet meta-learning in the episodic sense. The network was trained on a verification problem, not on few-shot episodes. But it established the core idea. A similarity function trained on many classes can transfer to new classes, because what it learns is not what any particular class looks like but which differences between inputs matter for telling classes apart. For handwritten characters, the network learned to attend to the topology and direction of strokes and to ignore variations in line thickness or exact position. Matching Networks: training as you test The Matching Networks paper described in the previous chapter made two contributions. One was the episodic training procedure. The other was a specific model that made comparison more flexible. A Matching Network embeds the support examples and the query example with neural networks and then classifies the query by a weighted vote over the support labels. The weight on each support example is computed by an attention mechanism: the softmax of the cosine similarity between the query's embedding and that support example's embedding. Support examples that look like the query get high weight, and the predicted label is a blend of their labels. This is a soft, differentiable version of nearest-neighbour classification, which means the whole system can be trained end to end by gradient descent on the query loss. The paper also introduced what its authors called full context embeddings. Instead of embedding each support example in isolation, the model passed the whole support set through a bidirectional recurrent network so that each example's embedding could depend on the others. The intuition is sensible: whether a feature is useful for discrimination depends on which other classes are present. If the episode contains a dog and a wolf, ear shape matters; if it contains a dog and a bicycle, almost anything works. Full context embeddings let the representation adapt to the episode, a modest form of inner-loop learning added to an otherwise fixed comparison. Matching Networks substantially improved on earlier results for miniImageNet, a benchmark the same paper introduced. More importantly, they showed that episodic training of an embedding was a workable recipe, and they set the template that most of the following work adopted. Prototypical Networks: the power of an average The method that became the reference point for metric-based meta-learning was simpler still. In 2017 Jake Snell, Kevin Swersky and Richard Zemel introduced Prototypical Networks. Their idea fits in a sentence: represent each class by the average of its support examples' embeddings, and classify a query by which class average it is closest to. The procedure in an episode runs as follows. An embedding network, typically a small convolutional network, maps every support image to a vector. For each class, the vectors of its K support examples are averaged to produce a single prototype. The query image is embedded by the same network. The distance from the query's embedding to each prototype is computed, and these distances, negated and passed through a softmax, become the predicted probabilities for each class. The training loss is the usual cross-entropy on the query labels, and gradients flow back through the distances and the averages into the embedding network. Several aspects of this design repay attention. First, the inner loop is almost nothing: an average. All of the learning lives in the embedding network, which is shaped by the outer loop to produce a space in which class averages are good summaries. The method bets everything on the representation. Second, the choice of distance matters, and the obvious choice is not the best. Snell and colleagues found that squared Euclidean distance worked better than cosine similarity, which Matching Networks had used. They gave a theoretical reason. Squared Euclidean distance belongs to a family called Bregman divergences, and for any distance in that family, the point that minimises the average distance to a set of points is their mean. So averaging embeddings to form a prototype is the natural operation exactly when the distance is of this type. They also showed that with squared Euclidean distance, the classifier induced by the prototypes is equivalent to a linear classifier on the embeddings, with weights and biases computed directly from the prototypes. Prototypical Networks are, in effect, a way of producing a linear classifier for a new task without any gradient steps. Third, the method revealed a useful training trick. Training with more classes per episode than would be used at test time, for example 20-way or 30-way episodes when testing on 5-way tasks, improved test accuracy. Harder training episodes force the embedding to make finer distinctions, which then pay off on easier test episodes. On miniImageNet, Prototypical Networks reported accuracy of roughly 49 per cent for 5-way one-shot and 68 per cent for 5-way 5-shot, competitive with or better than more complex methods of the time. Their simplicity, speed and reliability made them the default baseline for years, and in later large-scale comparisons such as Meta-Dataset they remained among the stronger approaches. A prototype at work It helps to follow a single episode through a trained Prototypical Network, because the simplicity of the procedure hides how much the embedding is doing. Suppose the episode is 3-way, 2-shot: two photographs each of a heron, a pelican and a flamingo, and a query photograph of a heron standing in shallow water. The embedding network turns each of the six support photographs into a vector with, say, a few hundred numbers. Nothing about these numbers is designed by hand. They are whatever the outer loop found useful across thousands of earlier episodes involving other birds, other animals and other objects. In a well-trained network, some directions in this space will have come to correspond loosely to properties that help separate classes in general: the length and shape of a bill, the proportions of neck and legs, overall colour, posture. The two heron vectors are averaged into a heron prototype, and likewise for the pelican and the flamingo. If the two heron photographs differ, one showing the bird in flight and one standing, their average sits somewhere between them, smoothing out the pose and keeping what they share. The query vector is computed and its squared distance to each of the three prototypes measured. Suppose the distances come out at 4, 11 and 9. Negated and passed through a softmax, these become probabilities heavily concentrated on the heron. The network has classified a bird species it may never have seen, from two examples, with no training on this episode at all. Two things can go wrong, and both are instructive. If the network's embedding has learned to put great weight on background, the query heron in water might land closer to a pelican photographed on a lake than to a heron photographed in a tree. The averaging cannot correct this; it can only average whatever the embedding provides. And if one of the support images is unrepresentative, for instance a heron partly hidden by reeds, the prototype is pulled towards it, since an average of two points is very sensitive to each. With five shots, a single odd example matters less, which is part of why five-shot accuracy is so much higher than one-shot accuracy for all methods. Using what is not labelled A natural extension of the prototype idea makes use of examples without labels. In many few-shot settings, a few labelled examples arrive alongside a larger pile of unlabelled ones from the same task: a handful of tagged images and many untagged ones, or, at test time, the whole batch of query images to be classified together. Mengye Ren and colleagues, in the 2018 paper that also introduced tieredImageNet, proposed refining prototypes with unlabelled examples. After computing initial prototypes from the labelled support set, each unlabelled example is softly assigned to the prototypes according to its distance, and the prototypes are recomputed as weighted averages including these soft assignments, much as in a single step of the classical clustering algorithm known as k-means. Their method also allowed for distractors, unlabelled examples belonging to none of the task's classes, by adding an extra cluster to absorb them. Refined prototypes improved accuracy, particularly when labelled examples were scarce. Using the query set itself in this way is called transductive inference: the model classifies all the queries jointly rather than one at a time, exploiting the structure among them. Transductive methods regularly report higher accuracy than inductive ones on the same benchmarks, and comparisons between the two kinds are not fair unless this is made explicit. In applications where queries arrive one at a time, such as a camera classifying each frame as it is captured, transduction is not available, and its benchmark gains are irrelevant. Learning the comparison itself If the embedding can be learned, so can the comparison. Relation Networks, introduced by Flood Sung and colleagues in 2018, replaced the fixed distance function with a second neural network. Support and query embeddings were concatenated and passed to this relation module, which output a score between zero and one for how related they were. The argument was that a fixed metric such as Euclidean distance imposes an assumption about the geometry of the embedding space that may not be ideal, and that a learned comparison could capture more complex relationships. In practice the gains from learning the metric were modest and depended heavily on implementation details. A recurring lesson of this period was that better embeddings mattered far more than cleverer comparisons. A related line of work showed that simply scaling the distances by a learned temperature parameter before the softmax, as in TADAM by Boris Oreshkin and colleagues in 2018, could make a substantial difference, because it controls how confident the classifier's predictions are and therefore how strongly gradients flow during training. Differentiable classifiers in the inner loop A different refinement kept the embedding idea but made the inner loop smarter. Instead of computing prototypes, why not fit a proper classifier on the support embeddings, one that can weigh features and handle classes whose examples are spread out? The difficulty is that fitting a classifier is itself an optimisation, and the outer loop needs gradients through it. Two papers in 2019 showed how to do this efficiently by choosing classifiers whose solutions can be computed in closed form or by fast convex solvers. Luca Bertinetto and colleagues used ridge regression, which has an exact formula for its solution, and exploited the fact that with few examples the formula can be rewritten to involve only a matrix whose size is the number of support examples rather than the number of features. Kwonjoon Lee and colleagues, in MetaOptNet, used a linear support vector machine, solving it with a differentiable quadratic programming layer and obtaining gradients through the solution's optimality conditions. Both approaches improved on Prototypical Networks on standard benchmarks. They also mark a conceptual bridge. The inner loop is now a genuine learning algorithm, a classifier fitted to data, but a simple and well-understood one whose behaviour the outer loop can shape by changing the features it receives. This sits halfway between the pure metric approach and the optimisation-based approach of the next chapter, where the inner loop is ordinary gradient descent on a full network. What metric methods get right Metric-based methods have durable strengths that explain why they remain in use well after the period of their invention. They are fast at test time. Adapting to a new task means embedding the support set and averaging, which costs a single forward pass per example. There is no inner optimisation to run, no learning rate to tune, and no risk of the adaptation diverging. They are stable to train. The outer loop is ordinary backpropagation through a feed-forward computation, without the second-order gradients and long computation chains that make optimisation-based methods fragile. They handle variable numbers of classes and examples naturally. A prototype can be computed from any number of examples, and a query can be compared with any number of prototypes, so the same trained model works for 5-way and 50-way tasks. They make the source of their power transparent. A Prototypical Network is nothing more than an embedding plus an average. If it works, the embedding is good. This clarity turned out to be prophetic. The finding, discussed in chapter seven, that a well-pretrained embedding with a simple classifier rivals sophisticated meta-learning, is in a sense a vindication of the metric approach's central bet: that the representation is where the prior should live. What they cannot do The same features impose limits. Metric methods adapt only by moving the decision boundaries between classes in a fixed embedding space. They cannot change the embedding itself in response to the new task. If a new task depends on a feature the embedding never learned to represent, no amount of averaging will recover it. A network trained to distinguish animals may embed all aircraft in a small region of its space, because nothing in its training required telling aircraft apart. Given a few-shot task on aircraft models, it will struggle, and a metric method has no mechanism to fix this from the support set alone. This weakness shows up clearly under domain shift. On Meta-Dataset's held-out domains, metric methods trained on natural images lose much of their advantage. The prior they learned is about which features distinguish everyday objects, and it does not transfer to traffic signs or fungus species without help. Metric methods are also naturally suited to classification and awkward for other problems. Regression, structured prediction and reinforcement learning do not have class prototypes to compare against. Extensions exist, but they lose much of the elegance that makes the methods attractive. These limits motivate the next family of methods, which keep the full flexibility of gradient-based learning in the inner loop. If the prior cannot be entirely captured by a fixed representation, perhaps it can be captured by a starting point from which the representation itself can be quickly adjusted. That idea is Model-Agnostic Meta-Learning. Chapter 4: MAML and the Art of the Starting Point Gradient descent is the engine of modern machine learning, and it has a peculiar relationship with where it starts. Begin training a network from one random initialisation and it may reach a good solution in a thousand steps; begin from another and it may take ten thousand, or settle somewhere worse. For large datasets this sensitivity is mostly an annoyance, managed with careful initialisation schemes. For tiny datasets it is decisive. With five examples and a few gradient steps, the network barely moves from where it began, and where it began determines almost everything about where it ends. In 2017 Chelsea Finn, Pieter Abbeel and Sergey Levine at the University of California, Berkeley published a paper that turned this sensitivity into a method. Their paper, "Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks", presented at the International Conference on Machine Learning, proposed to learn the starting point itself. If one could find initial parameters from which a few steps of ordinary gradient descent on any task in a family led to a good solution for that task, then ordinary fine-tuning would become few-shot learning. The method, universally called MAML, became the best-known algorithm in the field and the point of departure for hundreds of later papers. The algorithm in words MAML has a single set of meta-parameters: the initial weights of a network. Call them the shared initialisation. Meta-training adjusts this initialisation using the two-loop structure introduced in chapter two. The inner loop works as follows. Take a task and its support set. Start the network at the shared initialisation. Compute the loss on the support set and its gradient with respect to the weights. Take one gradient step, or a few, with a fixed inner learning rate. The result is a set of adapted weights specific to this task. Nothing unusual has happened yet; this is just fine-tuning with very little data. The outer loop is where MAML differs from ordinary training. Evaluate the adapted weights on the task's query set. This query loss measures how well fine-tuning from the shared initialisation worked for this task. Now compute how that query loss would change if the shared initialisation were moved slightly, accounting for the fact that moving the initialisation also changes where the inner gradient step lands. Average this signal across a batch of tasks and update the shared initialisation in the direction that reduces the average post-adaptation query loss. The key point is what the outer loop is optimising. It is not trying to make the initialisation good at any task directly. A network sitting at the learned initialisation may perform poorly on every individual task. It is trying to make the initialisation good to fine-tune from: a point in weight space from which a small step in the direction of any task's gradient produces a large improvement on that task. Finn and her colleagues described this as finding parameters that are sensitive to changes in the task, so that small changes in the parameters produce large improvements on the loss of any task drawn from the distribution. The second-order gradient The step "compute how the query loss would change if the shared initialisation were moved" hides the main computational difficulty of MAML. The adapted weights are the initialisation minus the inner learning rate times the gradient of the support loss, evaluated at the initialisation. When the initialisation changes, both terms change: the starting point moves, and the gradient computed at that starting point also changes. Differentiating the query loss with respect to the initialisation therefore requires differentiating through the gradient itself, which brings in second derivatives of the support loss, the Hessian matrix. For a network with millions of parameters the full Hessian is impossibly large, but the computation only needs its product with a vector, which automatic differentiation frameworks can compute at roughly the cost of an extra backward pass. With several inner steps, the chain of dependencies lengthens, and memory use grows with the number of steps because every intermediate state must be stored for the backward pass. Finn and colleagues tested a first-order approximation that simply ignores the second-derivative terms. In this version, the gradient of the query loss with respect to the adapted weights is applied directly to the initialisation, as if the inner step did not depend on where it started. Surprisingly, on miniImageNet this first-order MAML performed nearly as well as the full version while being substantially faster. The authors suggested that the networks they used, with ReLU activations, are locally almost linear, so that second derivatives are small for much of the time. Whatever the reason, the result showed that much of MAML's benefit did not depend on the expensive part of the computation. Reptile: an even simpler path In 2018 Alex Nichol, Joshua Achiam and John Schulman at OpenAI pushed the simplification further with an algorithm they called Reptile. Its procedure is almost disarmingly plain. Start at the shared initialisation. Pick a task. Run several steps of ordinary stochastic gradient descent on that task's data. Then move the shared initialisation a fraction of the way towards wherever those steps ended up. Repeat with a new task. Reptile has no support and query split within a task and no second derivatives. At first glance it looks like it should simply learn the average of the tasks' solutions, which for the sine wave problem would be a useless flat line. Nichol and colleagues showed, using a Taylor series expansion of the update, why it does something more useful. When more than one inner step is taken, the Reptile update contains a term that encourages the gradients of different mini-batches from the same task to point in similar directions. Gradient agreement within a task means that a step taken on one subset of the data also improves the loss on another subset, which is exactly what generalisation from a few examples requires. With a single inner step, Reptile does reduce to ordinary joint training, and the meta-learning benefit appears only with multiple steps. The analysis showed that first-order MAML and Reptile both optimise, to leading order, a mixture of the average task loss and this gradient-alignment term. Reptile matched MAML's performance on the standard benchmarks while being simpler to implement and cheaper to run. It also suggested a broader lesson: what makes an initialisation good for fast adaptation is that the tasks' loss surfaces near it are shaped so that a few steps of descent on a small sample of data reliably reduce the loss on unseen data from the same task. What MAML achieved The original paper evaluated MAML in three settings, and the breadth was part of its appeal. For regression, the sine wave problem from chapter two served as a demonstration. A network meta-trained with MAML, given five or ten points from a new sinusoid and a single gradient step, recovered a curve close to the true sine wave, including in regions where no support points lay. A network pretrained on the pooled data from all tasks and then fine-tuned on the same points did much worse, staying close to the average curve. The comparison made the point vividly: MAML's initialisation encoded the knowledge that tasks were sinusoids, and the inner step needed only to find the amplitude and phase. For classification, MAML reported accuracies of about 98.7 per cent on 5-way one-shot Omniglot and about 48.7 per cent and 63.1 per cent on 5-way one-shot and five-shot miniImageNet, using a small four-layer convolutional network. These figures were competitive with the best metric-based methods of the time, though not dramatically better. The authors used a network with only 32 filters per layer for miniImageNet, noting that larger networks overfitted, a sign of the difficulty of meta-training on a modest pool of 64 classes. For reinforcement learning, MAML was applied to simulated robots in the MuJoCo physics simulator. One family of tasks required a simulated cheetah-like robot to run at different target velocities; another required it to run either forwards or backwards. After meta-training, a single gradient update using a small batch of experience from a new task let the policy adapt its behaviour, for instance switching direction. Here MAML's model-agnostic character was essential. Metric methods have no natural application to locomotion, but any policy trained by policy gradients can have its initialisation meta-learned. One meta-training step, followed closely The abstract description can be made concrete by following a single outer-loop update in the sine wave problem. The network is small: one input, two hidden layers of forty units, one output. Its weights sit at the current shared initialisation. The outer loop samples a batch of tasks, say ten sine waves with different amplitudes and phases. For the first of these, it draws ten points as the support set and ten more as the query set. The inner loop computes the network's predictions at the ten support points, measures the mean squared error against the true values, and takes one gradient step with a learning rate of 0.01. The network's weights move slightly, producing a curve that passes a little closer to the support points. These adapted weights are then evaluated at the ten query points, giving a query loss for the first task. The same sequence is repeated independently for each of the other nine tasks, each starting from the same shared initialisation and each producing its own adapted weights and its own query loss. The outer loop sums the ten query losses and computes their gradient with respect to the shared initialisation, differentiating back through each task's inner step. It then updates the initialisation with a separate, outer learning rate, typically using the Adam optimiser. This is one meta-training iteration. The original paper ran tens of thousands of them. Over these iterations, something specific happens to the initialisation. Early on, a single inner step barely changes the network's curve, so query losses are high. Gradually, the outer loop shapes the initial weights so that the gradient of the support loss points in a direction that changes the curve's amplitude and phase efficiently. In effect, the network learns to make the two quantities that vary across tasks easy to change with one step, while the shared sinusoidal shape is built into the starting weights. A plain network trained on pooled data has no reason to arrange its weights this way, which is why it adapts so poorly. Practical sensitivities MAML's results depend on choices that the algorithm's elegant description leaves open. The original paper used one inner gradient step for Omniglot and five for miniImageNet during meta-training, with more steps allowed at test time, and small meta-batches of a few tasks per outer update. Practitioners who reimplemented the method found that changing these settings could shift accuracy by several percentage points, comparable to the differences between methods in published comparisons. One subtle issue concerned batch normalisation. In common implementations, the normalisation statistics at test time were computed from the batch of query examples being classified, which means that predictions for one query depended on the others in the same batch. This is a mild form of the transductive inference described in chapter three, and it inflated reported accuracies relative to a strictly one-query-at-a-time evaluation. Such details rarely change which method wins in a careful comparison, but they explain why numbers from different papers cannot always be compared directly, and why later work emphasised reimplementing every method under identical conditions. Why "model-agnostic" matters The name records MAML's most important property. It makes no assumption about the model except that it is trained by gradient descent. The same procedure applies to a classifier, a regressor, a policy, or a language model. There is no special architecture, no comparison module, no memory. One takes an existing learning setup, wraps it in an outer loop, and obtains a meta-learner. This generality has a theoretical counterpart. In 2018 Finn and Levine showed that, for a sufficiently deep network, gradient-based meta-learning of this kind is as expressive as any learning procedure that maps a support set and query to a prediction. A recurrent black-box meta-learner of the kind discussed in chapter six cannot, in principle, represent any learning algorithm that a deep enough MAML model could not also represent. The practical meaning of this result is limited, since in-principle expressiveness says nothing about what is learnable in practice, but it established that putting gradient descent in the inner loop does not cap what can be learned. MAML also has a built-in robustness that black-box methods lack. Because its inner loop is gradient descent, it keeps improving if given more steps or more data at test time, and on tasks somewhat outside the training distribution it degrades towards ordinary fine-tuning rather than towards nonsense. A learned procedure that has no gradient descent inside it has no such fallback. What MAML is really learning The story so far suggests that MAML learns an initialisation from which the whole network rapidly reshapes itself to each new task. In 2020 Aniruddh Raghu, Maithra Raghu, Samy Bengio and Oriol Vinyals published an analysis that questioned this picture. Their paper's title posed the question directly: "Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML". They examined what actually changed during MAML's inner loop. Using techniques for comparing the internal representations of networks, they found that the representations in the early and middle layers barely changed during adaptation. The network's body, the feature-extracting layers, stayed almost fixed, and nearly all the change happened in the final layer, the head that maps features to class labels. They then tested a simplified algorithm, which they called ANIL, for "Almost No Inner Loop", that adapted only the head in the inner loop and left the body frozen. ANIL matched MAML's accuracy on standard benchmarks while being considerably faster. They went further, showing that at test time even the head adaptation could be replaced by a simple classifier on the learned features with little loss. The conclusion was that MAML, on these benchmarks, worked primarily through feature reuse. Its outer loop learned a good feature extractor, and its inner loop fitted something close to a linear classifier on top. This is strikingly close to what Prototypical Networks do, by a more expensive route. The meta-learned prior lived mainly in the representation, not in a capacity for rapid change. This finding is sometimes read as a debunking of MAML. It is better read as a finding about benchmarks. On miniImageNet, training and test classes are similar enough that a good general-purpose feature extractor transfers well, and the only task-specific thing to learn is how to divide the feature space among the new classes. When tasks require genuinely different features, the balance shifts. Later work, including a 2021 paper by Jaehoon Oh and colleagues that adapted only the body and froze the head, found that representation change in the inner loop becomes more valuable when test tasks are drawn from a domain different from training. In reinforcement learning, where new tasks can demand different behaviour rather than a relabelling of the same features, inner-loop change in the whole network also matters more. The lesson for the book's argument MAML's history illustrates the controlling idea of this book with unusual clarity. Its designers framed it as learning a starting point for fast adaptation. Analysis revealed that, in practice, the starting point's main virtue was that it contained a good representation, and that the fast adaptation was mostly a thin layer on top. The prior had found its way into the place where it did the most good for the tasks at hand, regardless of where the method's design had invited it to go. This is not a flaw in MAML. It is a sign that the outer loop was doing its job: discovering what kind of prior the task distribution called for. When the task distribution called for a representation, the representation is what it learned. When it calls for something else, such as rapid change in behaviour or features, gradient-based meta-learning has the capacity to learn that too, which a pure metric method does not. That capacity comes at a price. MAML is notoriously finicky to train. Its outer loop can be unstable, its performance is sensitive to inner learning rates and numbers of steps, and its memory cost grows with the length of the inner loop. It produces a single point estimate of the adapted model with no measure of uncertainty, which is dangerous when five examples are genuinely ambiguous. And like every meta-learner, it can meta-overfit, learning tricks that work on the training tasks and fail on new ones. The next chapter examines the large body of work that addressed these problems, and what it revealed about how meta-learning can go wrong. Chapter 5: Beyond MAML: Refinements, Uncertainty and Failure A successful algorithm in machine learning attracts two kinds of follow-up. The first kind fixes its practical problems: instability, cost, sensitivity to settings. The second kind reinterprets it, finding that it is a special case of something more general and using the general view to build something better. MAML attracted both in unusual volume. This chapter does not attempt to survey every variant. It picks out the developments that changed what practitioners do or how researchers understand the method, and it ends with the failure modes that the variants exposed, which turn out to be among the most important lessons in the whole field. Making the outer loop behave Anyone who has trained MAML from scratch knows that it can be fragile. Small changes in the inner learning rate, the number of inner steps, or the random seed can produce large differences in final accuracy, and training sometimes collapses partway through. In 2019 Antreas Antoniou, Harrison Edwards and Amos Storkey published a paper with the frank title "How to Train Your MAML", which catalogued these problems and proposed a set of fixes collectively known as MAML++. Several of their fixes addressed specific mechanical issues. The original implementation computed the loss only after the final inner step, so gradients had to travel back through every step, and any instability compounded. MAML++ added the query loss after every inner step, weighted so that early steps mattered at the start of training and later steps came to dominate. This gave the outer loop a smoother signal. Batch normalisation, a standard technique in which each layer's activations are normalised using statistics from the current batch, interacted badly with the inner loop, because the statistics computed in the first inner step were not appropriate for later ones. MAML++ kept separate normalisation statistics and parameters for each inner step. It also learned a separate inner learning rate for every layer and every step, and annealed in the expensive second-order terms gradually rather than using them from the start. None of these changes altered the core idea, but together they made MAML considerably more stable and accurate, and they underline a general point. Meta-learning involves differentiating through a learning process, and every quirk of that learning process, from normalisation to learning rate schedules, becomes part of what the outer loop must cope with. Learning more than the starting point The original MAML learns only the initialisation and uses a fixed, hand-chosen inner learning rate. An obvious extension is to learn other aspects of the inner loop too. Meta-SGD, proposed by Zhenguo Li and colleagues in 2017, learns a separate inner learning rate for every single parameter of the network, which also allows the inner update to point in a direction other than the raw gradient, since some parameters can be moved more and others less or even in reverse. This doubles the number of meta-parameters but gives the outer loop direct control over how each part of the network adapts. In effect, it lets the meta-learner decide which parameters should change quickly for new tasks and which should stay put, which connects naturally with the feature-reuse finding of the previous chapter: a meta-learner given this freedom can learn to freeze the body and adapt the head without being told. A different direction restricts what the inner loop may change. In CAVIA, introduced by Luisa Zintgraf and colleagues in 2019, the network has a small set of extra input parameters called context parameters. The inner loop adapts only these, leaving all the network's ordinary weights fixed, while the outer loop learns the weights. Because the context is low-dimensional, the inner loop is cheap and less prone to overfitting the tiny support set, and the context parameters become an interpretable summary of the task. For the sine wave problem, a two-dimensional context can capture amplitude and phase. Latent Embedding Optimisation, or LEO, by Andrei Rusu and colleagues at DeepMind in 2019, took a related approach for image classification. Instead of adapting the high-dimensional weights of a classifier directly, LEO learns an encoder that maps the support set to a low-dimensional latent code and a decoder that generates the classifier's weights from that code. The inner loop runs gradient descent in the latent space. This combines the data-dependent initialisation of a black-box method, since the encoder produces a task-specific starting point, with the gradient-based refinement of MAML, and it achieved strong results on miniImageNet and tieredImageNet at the time. LEO also built on features pretrained on the meta-training classes rather than learned from scratch, an early sign of the role pretraining would come to play. Making the gradients cheaper The memory cost of differentiating through many inner steps limits how far MAML's inner loop can run. In 2019 Aravind Rajeswaran, Chelsea Finn, Sham Kakade and Sergey Levine introduced implicit MAML, or iMAML, which removes that limit by changing the problem slightly. In iMAML, the inner loop does not run a fixed number of steps. It solves, approximately, an optimisation problem: find the parameters that minimise the support loss plus a penalty for straying far from the shared initialisation. That penalty, called a proximal term, keeps the adapted parameters anchored to the meta-learned prior. Because the inner solution is defined as the optimum of a problem rather than as the result of a particular sequence of steps, its sensitivity to the initialisation can be computed using the implicit function theorem from calculus, from the solution alone, without storing the path taken to reach it. The memory cost becomes independent of the number of inner steps. One can run the inner optimiser for as long as needed, with any optimiser, and still compute the exact meta-gradient up to the accuracy of the solution. iMAML also clarifies what MAML-style methods are doing. The proximal term makes the role of the initialisation explicit: it is the centre of a prior, and adaptation is a compromise between fitting the support data and staying close to that centre. This view leads directly to the probabilistic interpretation. Meta-learning as hierarchical Bayes Five examples are often genuinely ambiguous. Shown five photographs of a bird species that all happen to be taken against blue sky, a learner cannot know whether "against blue sky" matters. A good few-shot learner should be uncertain in such cases and say so. MAML produces a single adapted model with no expression of uncertainty, which is acceptable for benchmarks but dangerous in medicine, robotics, or any setting where acting on a confident wrong answer has costs. In 2018 Erin Grant and colleagues showed that MAML can be understood as approximate inference in a hierarchical Bayesian model. In such a model, there is a prior distribution over task-specific parameters, governed by shared hyperparameters. Each task's parameters are drawn from that prior, and each task's data are generated from its parameters. Learning the hyperparameters from many tasks is called empirical Bayes. Grant and colleagues showed that a few steps of gradient descent from a shared initialisation, with early stopping, behaves like computing an approximate posterior estimate under a Gaussian prior centred on that initialisation. MAML's outer loop is then learning the prior. This is the formal version of the book's controlling idea: the meta-learned initialisation is literally a prior distribution, and fast adaptation is inference under it. The Bayesian view immediately suggests improvements. If the inner loop is approximate inference, it can be made to produce a distribution rather than a point. Probabilistic MAML, introduced by Finn, Kelvin Xu and Levine in 2018, used variational inference to sample multiple plausible adapted models for an ambiguous task. In a demonstration, a few-shot classification task was deliberately constructed to be ambiguous, with support examples consistent with more than one underlying rule, and the method produced samples corresponding to the different plausible rules rather than collapsing to one. Bayesian MAML, by Jaesik Yoon and colleagues in the same year, used a particle-based method to represent the posterior with a set of models, each adapted jointly so that together they covered the space of plausible solutions. These methods are more expensive than MAML, and on standard benchmarks their accuracy gains are modest, but they provide calibrated uncertainty, which matters wherever the cost of overconfidence is high. Table 2 sets out how the main variants discussed in this chapter differ in what they learn and what problem they address. Table 2. Selected refinements of gradient-based meta-learning. Method (year) Outer loop learns Inner loop adapts Problem addressed MAML (2017) Initial weights All weights Fast adaptation for any model Meta-SGD (2017) Initial weights, per-weight rates All weights, at learned rates Hand-tuned inner step Reptile (2018) Initial weights All weights Cost of second-order gradients Probabilistic MAML (2018) Prior over weights Distribution over weights No uncertainty MAML++ (2019) Weights, per-step rates, normalisation All weights Unstable training iMAML (2019) Centre of a proximal prior Weights, solved to near-optimum Memory cost of long inner loops CAVIA (2019) Network weights Small context vector Overfitting, cost of full adaptation LEO (2019) Encoder, decoder, latent space Low-dimensional latent code High-dimensional adaptation Source: the original papers, as cited in the Notes. For a practitioner, the table suggests a rough order of preference rather than a single winner. First-order MAML or Reptile is the sensible starting point when computation is limited, since both give most of MAML's benefit at a fraction of its cost. MAML++ is the natural choice when the full method is wanted and stability is a concern. iMAML suits problems where adaptation needs many inner steps, as in larger networks or harder tasks. Context-based methods such as CAVIA suit problems where the variation between tasks is believed to be low-dimensional, as with physical systems governed by a few parameters. Probabilistic variants are worth their extra cost wherever a wrong confident prediction is expensive. In every case, the gains from choosing among these are usually smaller than the gains from better features or a better task distribution, a point the following chapters make forcefully. When the task distribution has several modes MAML assumes one shared initialisation serves the whole task distribution. If the distribution contains clusters of quite different tasks, say a mixture of sine waves, straight lines and quadratic curves, a single starting point must compromise among them, and may sit far from all of them. Several methods in 2019 addressed this by making the initialisation itself depend on the task. Multimodal MAML, by Risto Vuorio and colleagues, used a small network to identify which mode a task belonged to from its support set and then modulated the shared initialisation accordingly. Hierarchically structured meta-learning, by Huaxiu Yao and colleagues, organised tasks into a learned hierarchy of clusters and tailored the starting point for each cluster. These methods shade into the black-box approach of the next chapter, since part of the adaptation is now done by a network reading the support set rather than by gradient descent. They also make a point that matters in practice. Real task distributions are rarely the homogeneous families of benchmarks. A company meta-learning across its customers, or a robot meta-learning across terrains, will meet tasks of several distinct kinds, and a method that assumes a single cluster will underperform. Meta-overfitting The most important lessons from this period concern how meta-learning fails. Ordinary overfitting happens when a model fits the training examples too closely and fails on new examples. Meta-overfitting happens when a meta-learner fits the training tasks too closely and fails on new tasks. With 64 training classes, as in miniImageNet, the number of distinct 5-way tasks is enormous, but the underlying diversity is limited: every task is built from the same 64 classes. A meta-learner with enough capacity can learn features specific to those 64 classes that do not transfer. This is why MAML's original authors used small networks, and why meta-learning performance on miniImageNet often improved when networks were made larger only if combined with stronger regularisation or more data. Meta-overfitting is usually detected the same way ordinary overfitting is: by the gap between performance on meta-training tasks and on held-out meta-validation tasks. The cure is also analogous. More tasks, more diverse tasks, and constraints on what the meta-learner can learn all help. The memorisation problem A subtler failure was identified in 2020 by Mingzhang Yin, George Tucker, Mingyuan Zhou, Sergey Levine and Chelsea Finn, in a paper titled "Meta-Learning without Memorization". They noticed that some task distributions allow a meta-learner to solve every training task without using the support set at all. Their example was pose prediction. Each task involved a different object, and the goal was to predict the object's orientation from an image. The support set showed a few images of the object with their orientations. But the object's identity was visible in the query image, and each object always appeared in the same task. A meta-learner could therefore learn a single network that recognised each training object and output the orientation directly, ignoring the support set entirely. On training tasks this worked perfectly. On a new object, it failed, because the network had never learned to use the support set to discover the new object's orientation convention. The underlying condition is that the tasks are not mutually exclusive: a single function can solve all of them at once. In standard few-shot classification this is prevented by randomly shuffling labels within each episode, which makes it impossible for one function to solve all tasks, since the same image gets different labels in different episodes. But in many real meta-learning problems, including regression, pose estimation, and reinforcement learning, there is no equivalent shuffle, and memorisation is a live danger. Yin and colleagues proposed a remedy based on information theory: penalise the amount of information about the task that the meta-learned parameters can store, so that the network is forced to get task information from the support set instead. Other researchers, including Janarthanan Rajendran and colleagues in a 2020 paper on meta-augmentation, proposed adding randomness to the tasks themselves, for example by randomly offsetting the target outputs for each task in a way that can only be inferred from the support set. Both approaches restore the condition that matters: the support set must contain information the model needs and cannot get elsewhere. The memorisation problem is a sharp illustration of this book's argument. A meta-learner will put the prior wherever the training signal rewards putting it. If the task distribution lets the model succeed by memorising tasks, it will memorise. If the task distribution makes the support set indispensable, it will learn to use the support set. The algorithm cannot tell these situations apart. The designer of the task distribution must. What the variants taught Taken together, the work that followed MAML points to three conclusions. First, gradient-based meta-learning is best understood as learning a prior, and its refinements are largely improvements in how that prior is represented, whether as a point, a distribution, a low-dimensional manifold, or a mixture. Second, the practical difficulties of the approach, from instability to memory cost, are the difficulties of differentiating through optimisation, and they can be managed but not eliminated. Third, and most important, the quality of a meta-learner is bounded by the quality of its task distribution. Too few tasks, or tasks that can be solved without the support set, produce a meta-learner that has learned the wrong thing. The methods in this and the previous chapter all keep gradient descent in the inner loop. The next chapter considers what happens when that constraint is dropped, and the whole learning procedure is itself learned. Chapter 6: Learning the Learner The methods of the previous two chapters keep a hand-designed algorithm, gradient descent, at the heart of the inner loop, and use the outer loop to give it a good place to start. The metric methods of chapter three keep a hand-designed algorithm too, nearest-prototype classification, and use the outer loop to give it a good space to work in. In both cases, the procedure by which the system learns a new task is fixed by its designers. Only its inputs are learned. A more radical option is to learn the procedure itself. Instead of telling the system how to learn from a support set, give it a flexible, trainable function that takes the support set as input and produces predictions or parameter updates as output, and train that function across many tasks. The learning algorithm becomes a neural network. This family is often called black-box or model-based meta-learning, and it includes three strands that developed somewhat separately: networks that learn within their activations, learned optimisers that replace gradient descent, and meta-reinforcement learning, in which an agent learns how to explore and adapt. Each strand puts the prior in the update rule or in the computation itself. Learning in the activations Chapter one mentioned the 2001 experiment by Hochreiter, Younger and Conwell in which a recurrent network learned to act as a learning algorithm. The idea returned with the deep learning revival. In 2016 Adam Santoro and colleagues at DeepMind introduced memory-augmented neural networks for one-shot learning. Their model was a neural network controller coupled to an external memory that it could write to and read from, a design descended from the Neural Turing Machine. During an episode, the network was shown a sequence of examples. At each step it received an input together with the correct label for the previous input, and it had to predict the label of the current input. This one-step offset was deliberate. The network could not simply copy the label it was shown; it had to store each input-label pair in memory, and when a new input arrived, retrieve the stored example it resembled and output that label. Labels were shuffled across episodes, so the only way to succeed was to learn, within each episode, which label went with which kind of input. After meta-training on Omniglot, the network classified new characters accurately after seeing only one or two examples of each, with all its weights frozen. The learning on a new task happened entirely in what it wrote to memory. Its weights encoded not knowledge of any particular characters but a procedure: how to encode an example for storage, and how to retrieve a relevant memory given a query. A year later, Nikhil Mishra and colleagues introduced SNAIL, a "simple neural attentive learner" that replaced the explicit memory with a combination of temporal convolutions and attention over all previous inputs in the episode. Attention lets a network look back at any earlier element of a sequence and weight it by relevance, which is a natural mechanism for comparing a query with stored examples. SNAIL performed well on few-shot classification and reinforcement learning benchmarks, and its reliance on attention over a sequence of examples prefigured the in-context learning of transformers, which are built almost entirely from attention. Another strand framed the same idea in probabilistic terms. Conditional Neural Processes, introduced by Marta Garnelo and colleagues in 2018, encode each support example independently, aggregate the encodings into a single summary by averaging, and feed the summary and a query input to a decoder that predicts the output with an estimate of uncertainty. Because the aggregation is an average, the model treats the support set as an unordered set, as it should, and handles any number of examples. Neural processes behave like a learned, fast approximation to Gaussian process regression, a classical method for inferring functions from points. The appeal of these methods is maximum flexibility. The network can in principle learn any procedure for using the support set, including ones no human would design. The cost is that the learned procedure is opaque and, critically, has no guarantee of behaving sensibly on tasks unlike those seen in training. A gradient-based learner given an unfamiliar task will at least move its parameters in a direction that reduces the loss. A black-box learner given an unfamiliar task does whatever its weights happen to compute, which may be nothing useful at all. Predicting the weights directly Between black-box learners and the classifiers of earlier chapters sits a strand of work that uses a network to write the weights of another network. The idea is to treat the final layer of a classifier, the weight vector for each class, as something that can be generated from that class's support examples by a learned function, rather than fitted by gradient descent. Siyuan Qiao and colleagues proposed in 2018 to predict the classifier weights for a new class directly from the network's activations on its few examples. Spyros Gidaris and Nikos Komodakis, in the same year, built a system that learned a weight generator able to produce a classifier for a new class from a handful of examples, using an attention mechanism over the classifier weights of the classes already known. Their system had a further aim, signalled in its title, "Dynamic Few-Shot Visual Learning without Forgetting": it added the new class to an existing classifier without degrading its performance on the many classes it already recognised. Using cosine similarity rather than a raw dot product for classification, so that the magnitude of a weight vector did not dominate, was important to making the generated and the conventionally trained weights compatible. This line of work makes concrete something that is easy to lose in benchmarks. Real systems rarely face a stream of isolated five-way problems. They face a large existing set of categories to which new ones must be added from little data, and the new categories must coexist with the old. A weight generator trained across many simulated additions is a meta-learner whose prior concerns how new class weights should relate to existing ones. The broader family of such methods, often called hypernetworks, is also one of the routes by which learning-to-learn ideas entered the design of large models, where small generated adjustments are used to specialise a shared network to a task. Learned optimisers A second strand keeps the structure of iterative parameter updates but learns the rule that produces them. Gradient descent and its widely used variants, such as momentum methods and Adam, are hand-designed formulas that turn a gradient, and perhaps some running statistics, into a parameter change. A learned optimiser replaces the formula with a small neural network. The paper that brought this idea into the deep learning era was published in 2016 by Marcin Andrychowicz and colleagues at DeepMind, under the memorable title "Learning to Learn by Gradient Descent by Gradient Descent". Their optimiser was a small recurrent network that took in the gradient for each parameter and output the update for that parameter. To make the approach practical for networks with many parameters, the same small network was applied separately to each parameter, sharing its weights across all of them, an arrangement called coordinate-wise. The optimiser was itself trained by gradient descent, by running it on training problems, measuring the loss the optimised model reached, and backpropagating through the whole optimisation trajectory. Trained on small problems, it beat hand-designed optimisers on similar problems and transferred to some modestly different ones. Sachin Ravi and Hugo Larochelle applied the same idea to few-shot classification in a 2017 paper, "Optimization as a Model for Few-Shot Learning". They noticed that the update equation for the cell state of a Long Short-Term Memory network resembles a gradient descent step, with the network's gates playing the roles of learning rate and weight decay. They used an LSTM as an optimiser that both learned the initial parameters of a classifier and controlled how each update step changed them. This paper also introduced the standard split of miniImageNet into 64 training, 16 validation and 20 test classes, which became the convention for later work. The central difficulty with learned optimisers is generalisation across problems. An optimiser trained on small networks for a few hundred steps may behave erratically when asked to train a large network for many thousands, because it has never seen gradients of that scale or trajectories of that length. Training the optimiser by backpropagating through long trajectories is also expensive and numerically unstable, since gradients can explode or vanish over many steps. Subsequent work tried to overcome these problems by scale. In 2022 Luke Metz and colleagues at Google described VeLO, a learned optimiser trained across thousands of diverse machine learning tasks using approximately four thousand TPU-months of computation. VeLO was designed to require no hyperparameter tuning and reported competitive or better performance than tuned hand-designed optimisers on a range of problems. Independent evaluations published afterwards gave a more mixed picture, finding that its advantages did not hold consistently across the settings they tested. Learned optimisers remain an active area, and their history is instructive: as with other meta-learners, their generalisation is bounded by the breadth of the tasks they were trained on, and covering the space of problems that practitioners care about turns out to require an enormous training distribution. Meta-reinforcement learning The third strand applies black-box meta-learning to reinforcement learning, where it addresses a problem that supervised few-shot learning does not have: exploration. A reinforcement learning agent does not receive a labelled support set. It must act in an environment, observe the results, and work out from its own experience what the task is. If an agent is placed in a maze whose goal location varies from task to task, the first thing it must do is find the goal, which requires exploring efficiently, and only then can it exploit what it found. The way an agent explores determines what data it gets, so the support set is itself a product of the agent's behaviour. In 2016 two groups independently proposed the same solution. Yan Duan and colleagues at Berkeley and OpenAI called their method RL squared, written RL², because it uses a slow reinforcement learning algorithm to train a fast one. Jane Wang and colleagues at DeepMind published "Learning to Reinforcement Learn". In both, a recurrent neural network acts as the agent's policy. Its input at each step includes not only the current observation but also the previous action and reward, and its hidden state is not reset between episodes within the same task. Trained by ordinary reinforcement learning across many tasks, the network learns to use its hidden state to accumulate information about the current task and to change its behaviour accordingly. What these agents learned was a reinforcement learning algorithm implemented in recurrent dynamics. On multi-armed bandit problems, in which an agent repeatedly chooses among options with unknown payoff probabilities, the meta-trained recurrent agents achieved performance close to well-known hand-designed algorithms for that setting. In navigation tasks, they learned to explore a new maze systematically on the first visit and then go straight to the goal on later visits. Wang and colleagues went on to argue, in a 2018 paper in Nature Neuroscience, that the brain's prefrontal cortex may implement something similar: a fast learning system in its recurrent activity, trained by a slower dopamine-driven learning process acting on synaptic weights. Whether or not that hypothesis holds in detail, it is a clear statement of the two-timescale structure that runs through all of meta-learning. Later work made the structure more explicit. PEARL, by Kate Rakelly and colleagues in 2019, separated the task-inference part of the agent from the policy. A network reads the agent's recent experience and produces a probability distribution over a latent variable representing the task; the policy then acts conditioned on a sample from that distribution. Sampling from an uncertain belief about the task produces a natural form of exploration, since the agent tries out behaviour suited to different plausible tasks until its experience narrows the possibilities. VariBAD, by Zintgraf and colleagues in 2020, pushed further towards approximating the theoretically ideal Bayes-adaptive behaviour, in which the agent explicitly weighs the value of information against the value of reward. At the far end of this strand sits work on agents trained across very large, procedurally generated task spaces. In 2023 a DeepMind team reported an agent, trained with a transformer-based memory across a vast space of three-dimensional tasks, that adapted to new tasks within a few attempts on timescales they compared to human players. That project made explicit what the earlier work implied. Fast in-context adaptation in reinforcement learning emerges when the agent is trained on a task distribution broad and rich enough that the only way to do well is to learn how to adapt. Choosing among the families With the black-box family described, the four places a prior can live, introduced in chapter one, now each have concrete methods attached. Table 3 compares them on the properties that matter most when choosing an approach. Table 3. Where the learned prior lives in the main families of meta-learning. Family Prior stored in Cost to adapt On unfamiliar tasks Natural fit Metric (e.g. Prototypical Networks) Embedding space One forward pass Limited by fixed features Classification Optimisation (e.g. MAML, Reptile) Initial weights, sometimes rates Several gradient steps Falls back to fine-tuning Any gradient-trained model Learned optimiser (e.g. VeLO) Update rule Many learned update steps Unreliable beyond training range Training full models Black-box (e.g. SNAIL, RL²) Weights and activations One pass over support set Can fail without warning Sequences, reinforcement learning Source: synthesis of the methods discussed in chapters three to six. The trade-off the table exposes is between flexibility and robustness. Black-box methods are the most flexible: they can learn any procedure, including ones adapted to quirks of the task distribution that no hand-designed inner loop would exploit. They are also the least predictable outside their training distribution. Optimisation-based methods are the least flexible in what the inner loop can do, but the presence of real gradient descent gives them a floor below which performance will not drop, since even on an alien task they will at least fit the support set. Metric methods are fast and stable but limited to the features they learned. The thread that connects them By around 2019 the field had a rich set of methods, a standard set of benchmarks, and a working theory: meta-learning learns a prior from a task distribution, and the families differ in where they store it. It also had a growing unease. Comparisons across papers were often confounded by differences in network size, training time, data augmentation and pretraining. Improvements reported on miniImageNet were sometimes smaller than the variation caused by implementation details. And the feature reuse finding about MAML hinted that the core of few-shot performance might be something simpler than any of the elaborate inner loops. That unease came to a head in a series of papers showing that a carefully trained ordinary network, with no meta-learning at all, could match the specialised methods. At almost the same time, large language models began to demonstrate in-context learning, the very capacity that black-box meta-learners had been designed to produce, without having been trained for it. The next two chapters take up these developments. Together they changed what meta-learning research is about, and they confirmed, more forcefully than any single method had, that the source of fast learning is the breadth and quality of slow learning. Chapter 7: The Baseline Shock Every research field has moments when a simple result embarrasses a complicated literature. For few-shot learning, that moment arrived across 2019 and 2020 in a handful of papers that asked a question the field had largely skipped: how well does the obvious approach actually work, if done properly? The obvious approach is to train an ordinary classifier on all the training classes at once, throw away its final layer, and on each new task fit a fresh simple classifier to the support examples using the features the network learned. No episodes, no inner loop, no meta-gradients. The answer, it turned out, was that it worked about as well as the specialised methods, and under some conditions better. These results are sometimes summarised as "meta-learning doesn't work". That is wrong, and this chapter explains why. But they did change the field's understanding of what few-shot performance depends on, and they fit the controlling idea of this book exactly. The prior that makes fast learning possible is mostly a representation, and the best way to get a good representation is to learn from as much varied data as possible, by whatever method extracts the most from it. A closer look The first widely noticed result came from Wei-Yu Chen and colleagues in a 2019 paper titled "A Closer Look at Few-Shot Classification". They reimplemented several leading meta-learning methods, including Matching Networks, Prototypical Networks, Relation Networks and MAML, in a single codebase with identical network architectures, data augmentation and training procedures. Alongside them they evaluated two baselines. The first, called simply Baseline, trained a standard network on all training classes with ordinary cross-entropy loss, then for each test task froze the network and trained a new linear classifier on the support set. The second, Baseline++, used a classifier based on cosine similarity rather than a plain linear layer, which reduces the variation in feature magnitudes across classes. Their findings had three parts. First, when everything else was held equal, the gaps between meta-learning methods were smaller than reported in the original papers, suggesting that some of the reported gains had come from implementation differences. Second, as the feature-extracting network was made deeper, the differences between all methods shrank, and the simple baselines caught up. With a shallow four-layer network, the meta-learning methods held an advantage; with a deeper residual network, Baseline++ was competitive with the best of them. Third, and most strikingly, under domain shift the baselines won. When methods were meta-trained on miniImageNet and tested on few-shot tasks from a dataset of bird species, the plain baselines outperformed every meta-learning method tested. The interpretation they offered was that meta-learning methods were reducing intra-class variation in the learned features, a benefit that mattered when the network was too small to learn good features on its own but faded as networks grew more capable. Under domain shift, the meta-learners' specialisation to the training task distribution became a liability, while the baselines' more generic features transferred better. Rethinking the embedding A year later, Yonglong Tian, Yue Wang, Dilip Krishnan, Joshua Tenenbaum and Phillip Isola published a paper whose title made the claim explicit: "Rethinking Few-Shot Image Classification: A Good Embedding Is All You Need?" The question mark was a courtesy; the content was assertive. They trained a single network on the union of all meta-training classes with standard supervised learning, extracted features for each test task's support set, and fitted a logistic regression classifier to them. This approach outperformed the leading meta-learning methods of the time on miniImageNet, tieredImageNet, and two other benchmarks. They then applied a technique called self-distillation, in which a network is trained to match the predictions of an earlier copy of itself, and improved results further. Around the same time, Guneet Dhillon and colleagues proposed what they called a baseline for few-shot classification: pretrain normally, then fine-tune on the support set, optionally also using the unlabelled query examples to regularise the fine-tuning so that predictions on them are confident. This transductive fine-tuning matched or beat specialised methods across several benchmarks. The authors also argued for more demanding evaluation, pointing out that reported confidence intervals understated how much accuracy varied from one episode to another, and that a method's performance on hard episodes deserved more attention than its average. Combined with the analysis of MAML described in chapter four, which showed that its success came largely from feature reuse, these papers made a coherent case. On the standard benchmarks, few-shot accuracy was determined mainly by the quality of the feature extractor. The mechanisms by which meta-learning methods adapted to new tasks, whether prototypes, gradient steps or attention, mattered less than whether the features they adapted were good. And the most effective way to get good features from a fixed pool of training classes was ordinary supervised training on all of them together, which uses every label in every batch, rather than episodic training, which uses only the few classes sampled for each episode. What Meta-Dataset showed The Meta-Dataset benchmark, introduced in chapter two, added nuance. In the original study, Eleni Triantafillou and colleagues evaluated a range of methods on its heterogeneous mix of domains. Fine-tuning a pretrained network was competitive. Pretraining the feature extractor on ImageNet as an ordinary classifier and then meta-training it episodically improved most meta-learning methods substantially. A hybrid they proposed, which used Prototypical Network prototypes to initialise the final layer and then took MAML-style gradient steps, performed among the best overall. Subsequent work on Meta-Dataset reinforced the pattern. The strongest methods tended to combine a pretrained universal feature extractor, often learned from all the training domains together or distilled from several domain-specific networks, with a lightweight, task-specific adaptation step at test time, such as a small set of feature-wise adjustments or a linear head fitted to the support set. The episodic meta-training, where present, worked best as a final stage refining features learned another way. The picture that emerged was not that meta-learning was useless, but that it was best seen as one ingredient. The representation should be learned from the broadest available data with whatever objective extracts the most from it. Meta-learning, where it helps, tunes that representation for the specific act of adapting from few examples and supplies the adaptation mechanism. Why episodes waste labels The baselines' advantage has a simple mechanical explanation worth spelling out. In episodic training on miniImageNet, each episode samples five of the 64 training classes. The loss for that episode teaches the network to tell those five apart and says nothing about the other 59. A pair of classes that are hard to distinguish, say two similar breeds of dog, appear together in an episode only occasionally, so the network gets few chances to learn the fine features that separate them. Ordinary classification training on all 64 classes uses a different signal. Every image in every batch is scored against all 64 classes at once, and the loss pushes its embedding away from every wrong class simultaneously. Each labelled image therefore contributes information about its relationship to the whole label set, not just to four randomly chosen alternatives. Per unit of computation and per labelled example, this is a far richer training signal. Episodic training buys a closer match between training and test conditions at the cost of using labels inefficiently, and when the representation is the dominant factor in test accuracy, the efficiency matters more than the match. This also explains why the gap between methods shrank as networks grew. A small network cannot learn excellent features from either signal, and episodic training's tailoring of the features to the few-shot setting compensates partly for their weakness. A large network can learn excellent features from the richer signal, and once it has them, the tailoring adds little. The cross-domain test Most of the early comparisons were within a single domain, everyday photographs. A study by Yunhui Guo and colleagues in 2020, the Broader Study of Cross-Domain Few-Shot Learning, asked what happens when the domain changes sharply. They trained models on miniImageNet and tested them on few-shot tasks from four very different image collections: photographs of diseased crop leaves, satellite images of land use, dermatology images of skin lesions, and chest X-rays. These domains were ordered roughly by how different they look from natural photographs, with X-rays the most remote. The results were consistent with Chen and colleagues' finding but more emphatic. Simple fine-tuning of a pretrained network generally outperformed the meta-learning methods tested, and in several cases meta-learning methods did worse than a network trained conventionally on miniImageNet with no meta-learning at all. All methods lost accuracy as the target domain became less like the source, and on chest X-rays, accuracy for every method was low. The prior learned from everyday photographs offered little help for reading radiographs, and no choice of adaptation mechanism could supply what the prior lacked. The cross-domain results are perhaps the most direct evidence for the controlling idea of this book. What an adaptation mechanism can do is bounded by the relevance of the prior it adapts. When the prior is relevant, a sophisticated mechanism is barely needed. When it is not, no mechanism suffices. The most useful thing a practitioner can do for few-shot performance in a new domain is find or build a prior from data that resembles it. Pretraining at scale The baseline papers compared methods trained on the same modest pool of classes. Meanwhile, the broader field was moving to a regime in which models were pretrained on vastly larger data, and this changed the comparison again. Self-supervised learning methods, which learn representations from unlabelled images by, for example, training a network to recognise that two differently cropped and coloured versions of the same image belong together, began to produce features that rivalled or exceeded supervised features. Vision-language models such as CLIP, introduced by Alec Radford and colleagues at OpenAI in 2021, were trained on hundreds of millions of image and caption pairs collected from the internet. CLIP could classify images into categories it had never been explicitly trained on, by comparing the image's embedding with the embeddings of text descriptions such as "a photo of a dog". This zero-shot ability, which requires not even a single example, is in a sense the extreme case of the metric approach of chapter three, with prototypes supplied by language rather than by support images. In 2022 Shell Xu Hu and colleagues examined what this meant for few-shot learning in a paper titled "Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Difference". Their pipeline had three stages: pretrain a vision transformer with self-supervised learning on a large external dataset, meta-train it with a simple Prototypical Network objective on the benchmark's training classes, and fine-tune on the support set of each test task. Each stage contributed, and the combination produced large improvements on miniImageNet and Meta-Dataset over methods without external pretraining. Their analysis made clear that the pretraining stage did most of the work, and also that meta-training and fine-tuning added measurable gains on top of it. This three-stage recipe of broad pretraining, then episodic meta-training, then light task-specific adaptation, is a good summary of where few-shot image classification settled. It also raised a methodological issue the field had to confront. If a model has been pretrained on a dataset that includes the "novel" classes of a few-shot benchmark, or images very like them, then its few-shot performance on those classes says little about learning from few examples. With web-scale pretraining data, it becomes almost impossible to guarantee that any test class is truly new. What the shock does and does not show It is worth stating precisely what these results establish. They establish that on standard image classification benchmarks, most of the few-shot performance of meta-learning methods comes from the learned representation, and that representations learned by ordinary supervised or self-supervised training on more data are generally better than those learned by episodic training on less. They establish that the episodic training procedure, which the field adopted on the principle of matching training and test conditions, is not necessary for good few-shot performance, and that it can be an inefficient way to use labelled data, since each episode trains on only a few classes. They do not establish that learning to adapt is unimportant. Several studies found that adding an episodic meta-training stage to a pretrained network improved few-shot accuracy, even if by less than the pretraining itself. A 2021 paper by Yinbo Chen and colleagues, introducing a method called Meta-Baseline, found that meta-training after classifier pretraining helped on novel classes, while also observing that it could reduce generalisation in some respects, a trade-off between specialising for the few-shot task and preserving broader feature quality. They do not apply straightforwardly outside image classification. In reinforcement learning, a good representation of observations does not tell an agent how to explore a new environment; learning to explore is a separate skill that meta-reinforcement learning addresses. In regression problems like the sine wave example, what must be learned is the structure of the function family, which an ordinary pretrained network does not capture. In domains where there is no large external dataset, such as many scientific problems, the question of whether to pretrain or meta-train is moot: one has a modest collection of related tasks and must extract as much as possible from it. And they do not show that meta-learning's concepts are obsolete. On the contrary, they are a confirmation of the central concept. The baselines succeed because they learn a strong prior from a broad set of classes. Pretraining on more data succeeds because it learns a stronger prior from a broader distribution. The baseline shock replaced one way of acquiring the prior with a more efficient one. It did not remove the need for a prior or change what the prior is for. Reading the lesson correctly There is a pattern in machine learning, described by the researcher Richard Sutton in a widely read 2019 essay as "the bitter lesson", that general methods leveraging large amounts of computation and data tend in the long run to beat methods that build in human knowledge about the problem. The baseline shock can be read as an instance of it. Carefully designed inner loops, an encoding of human ideas about how adaptation should work, were overtaken by simpler pipelines that learned more from more data. But the lesson needs to be read in light of this book's argument. What won was not the absence of a prior; it was a prior acquired from more data. And the most dramatic later demonstration of the bitter lesson in this domain, the in-context learning of large language models, is not an argument against meta-learning at all. It is meta-learning arising at scale, with the task distribution supplied implicitly by the variety of human text. The next chapter examines that phenomenon and what research has discovered about it. Chapter 8: Meta-Learning Inside the Transformer In 2020 a team at OpenAI led by Tom Brown published a paper on a language model called GPT-3, with 175 billion parameters, under the title "Language Models are Few-Shot Learners". Its central demonstration was simple to describe. Give the model a prompt containing a few examples of a task, such as English words followed by their French translations, or sentences followed by a label for their sentiment, and then a new input. The model continues the pattern, producing the translation or the label for the new input. No weights are changed. The model learns the task, to whatever extent it learns it, entirely from the examples in its input. This ability, now called in-context learning, is exactly what the black-box meta-learners of chapter six had been built to do. The memory-augmented networks and recurrent agents were trained on carefully designed episodes so that their forward pass would act as a learning algorithm. GPT-3 was trained only to predict the next word in a vast collection of text. Nobody designed its episodes. The authors of the GPT-3 paper made the connection explicit, describing pretraining as an outer loop in which the model develops a broad set of skills and pattern-recognition abilities, and in-context learning as an inner loop in which it uses those abilities to adapt to the task at hand. The language of meta-learning was the natural way to describe what had happened. This chapter examines that phenomenon through the lens of the book's argument. If fast learning is always paid for by slow learning, then in-context learning in large models should be explicable in terms of the task distribution implicit in their training data. A substantial body of research since 2020 has tested that idea, and it largely holds. Why next-word prediction contains a task distribution At first it seems mysterious that predicting the next word should produce a meta-learner. The explanation starts with the observation that text on the internet is full of implicit tasks. A web page listing countries and their capitals, a glossary of terms, a table of chemical formulas, a forum thread of questions and answers, a document that switches between two languages: each is a sequence of examples of some mapping, often with repeated structure. To predict the later entries in such a document well, a model benefits from inferring what mapping the earlier entries follow. Across billions of documents, each with its own implicit task, next-word prediction becomes training on an enormous and highly varied distribution of tasks, with each document serving as an episode. In 2022 Sang Michael Xie and colleagues formalised this intuition, framing in-context learning as implicit Bayesian inference. In their account, documents are generated by first choosing a latent concept, such as a topic, format or task, and then generating text consistent with it. A model trained to predict text from such a mixture learns, implicitly, to infer the latent concept from the context in order to predict well. When a prompt contains several examples of a task, the model infers the concept they share and continues accordingly. This is the hierarchical Bayesian picture of meta-learning from chapter five, arising without anyone having designed it. What the training data must look like If in-context learning comes from properties of the training distribution, it should be possible to identify which properties matter. In 2022 Stephanie Chan and colleagues at DeepMind did this by training transformers on carefully controlled image sequence data rather than natural language. They found that in-context learning emerged when the training data had certain distributional properties also found in language. One was burstiness: items tend to appear in clusters, so that if a rare word or class appears once in a document, it is likely to appear again nearby. Another was a large number of rarely occurring classes, following the long-tailed frequency distributions typical of natural language. When the training data lacked these properties, the transformers learned to memorise mappings in their weights instead. They also found a tension. Models tended to favour either in-context learning or in-weights learning, the ordinary storage of knowledge in parameters, and the training distribution determined the balance. A skewed distribution, with some classes very frequent and many very rare, allowed both to coexist: the model memorised the frequent classes and learned in context for the rare ones. Later work by Aaditya Singh and colleagues in 2023 found that in-context learning, having emerged during training, could fade again if training continued long enough, with the model gradually shifting towards in-weights solutions. In-context learning is therefore not a fixed property of the transformer architecture but a solution that the training distribution may or may not favour, and may favour only for a time. This is the memorisation problem from chapter five in a new setting. If the training tasks can be solved by storing answers in the weights, a model will do so. In-context learning appears when the data contain enough novelty, in the form of rare items whose meaning must be inferred from context, that storing everything is not an option. What algorithm is being run A second line of research asked what learning algorithm a transformer implements when it learns in context. Natural language is too complex to answer this directly, so researchers studied transformers trained from scratch on simple, mathematically clean task families. In 2022 Shivam Garg, Dimitris Tsipras, Percy Liang and Gregory Valiant trained transformers on sequences of input-output pairs from randomly chosen linear functions. Each sequence was an episode: a new linear function, a set of example points, and a query. After training, the transformers predicted query outputs about as accurately as the optimal method for this problem, least squares regression. They did similarly well for sparse linear functions, small decision trees, and simple two-layer neural networks, in each case matching or approaching strong task-specific algorithms. The transformer had learned, from a distribution of tasks, an algorithm suited to that distribution. Two groups then showed that the algorithm could be identified. Ekin Akyürek and colleagues, in a paper presented in 2023, showed that transformers can in principle implement gradient descent and closed-form ridge regression for linear models within their forward pass, and that trained transformers' predictions closely matched those of such algorithms, with the specific algorithm depending on model size and depth. Johannes von Oswald and colleagues showed in the same year that a single layer of a simplified, linear form of self-attention can implement exactly one step of gradient descent on a regression loss computed over the in-context examples, and that transformers trained on regression tasks converged to weights that did this or something close to it. Stacking layers yields multiple steps. These results have a striking implication for the history told in this book. The black-box and optimisation-based families, which looked like opposing answers to where the prior should live, can converge. A transformer trained as a black-box meta-learner on regression tasks may discover gradient descent and implement it in its activations. The outer loop finds the inner algorithm that best suits the task distribution, and for linear regression that algorithm is something like the one human designers would have written down. Allan Raventós and colleagues added an important qualification in 2023. They trained transformers on linear regression tasks while varying how many distinct tasks the training data contained. When the number of distinct pretraining tasks was small, the transformer behaved like a Bayesian estimator whose prior covered only those tasks; given a new task, it effectively picked the nearest familiar one. Above a threshold of task diversity, its behaviour changed, and it began to solve genuinely new tasks, approaching the optimal algorithm for the full distribution. Task diversity, not just the quantity of data, determined whether the model learned to learn or learned to recognise. Baxter's bargain from chapter one, trading tasks for examples, reappears here with a sharp threshold. What large models actually do with examples Studies of large language models in natural settings complicate the clean picture from synthetic tasks. A 2022 study by Sewon Min and colleagues found that, for many classification tasks, replacing the correct labels in the prompt's examples with random labels hurt performance surprisingly little. What mattered more was that the examples showed the format of the task, the space of possible labels, and the kind of input text involved. This suggested that, for these models and tasks, in-context examples often served to identify which task, among those the model already knew, was being requested, rather than to teach a new one. Later work by Jerry Wei and colleagues in 2023 found that this depended on scale. Small models largely ignored labels that contradicted their prior knowledge, for example examples in which positive reviews were labelled negative. Larger models were increasingly able to follow such flipped labels, overriding their prior to learn the mapping actually shown in context. In-context learning in the largest models is therefore a mixture of task recognition, drawing on what the model already knows, and genuine task learning, adapting to new mappings from the examples, with the balance shifting towards learning as models grow. Interpretability research has found some of the mechanisms involved. A team at Anthropic described in 2022 a circuit they called an induction head: a pair of attention heads that together find earlier occurrences of the current token in the context and predict that what followed it before will follow it again. Induction heads form at a particular point during training, coinciding with a sharp improvement in in-context learning, and they implement a simple form of pattern completion that underlies more elaborate in-context abilities. A fragile inner loop In-context learning differs from the meta-learners of earlier chapters in one practical respect that matters for anyone relying on it: it was never trained to be robust to the way examples are presented, and it often is not. A 2021 study by Tony Zhao and colleagues, titled "Calibrate Before Use", documented how much few-shot performance of GPT-3 class models varied with seemingly irrelevant choices. Changing the order of the same examples in the prompt, or the wording of the template around them, could move accuracy on a classification task from near chance to near the best achievable. They identified systematic biases behind this variation: a tendency to predict labels that appeared more often among the examples, a tendency to favour labels near the end of the prompt, and a tendency to favour labels that were common in the model's training text. They proposed a simple correction, estimating the model's bias by feeding it a content-free input and then adjusting its outputs to cancel that bias, which reduced the variation substantially. These sensitivities follow naturally from the account given here. The model's inner loop is whatever computation its pretraining distribution happened to reward, and that distribution did not consist of carefully shuffled, balanced episodes with randomised ordering. The explicit meta-learners of chapters three to six were trained on exactly such episodes and are correspondingly indifferent to support-set order. An implicitly meta-trained model inherits the quirks of its implicit task distribution. Larger and instruction-tuned models have become less sensitive to these factors, which is consistent with the view that broader and better-structured training tasks produce a more robust inner loop. Designing meta-learning into large models If in-context learning is meta-learning by accident, it can also be meta-learning by design. Several lines of work have made the episodic training of earlier chapters explicit in large models. Instruction tuning and multitask fine-tuning, in which models are trained on collections of many tasks each formatted with instructions or examples, improve in-context performance on held-out tasks, and are in effect meta-training on a curated task distribution. A 2023 paper in Nature by Brenden Lake and Marco Baroni made a pointed case for this approach. Their method, meta-learning for compositionality, trained a standard transformer on a stream of episodes, each of which defined a small invented language of novel words and their meanings through a few examples and then asked the model to interpret new combinations. After such training, the network generalised systematically to new combinations in ways that closely matched human behaviour on the same tasks, including some of the characteristic errors people make. Lake had co-authored the 2015 Omniglot paper arguing that neural networks lacked the structured priors needed for human-like learning. The 2023 result argued that a neural network could acquire such a prior, if trained on a task distribution designed to require it. The prior had to be learned from the right tasks, and when it was, the architecture did not need to change. Louis Kirsch and colleagues, in work published in 2022, studied the conditions under which transformers become general-purpose in-context learners when meta-trained on very diverse synthetic tasks, again finding that the diversity and size of the task distribution, along with the model's state size, determine whether a model learns to learn or merely to memorise. Combining weights and context In-context learning has limits. A model's context window is finite, so it can attend to only so many examples. In-context learning does not persist: the model forgets the task when the prompt is gone. And on tasks far from anything in the training distribution, in-context learning from a few examples is often weak. These limits have revived interest in combining in-context learning with gradient-based adaptation at the moment of use. One approach, called test-time training, updates some of a model's weights on the examples available for a specific test problem before making a prediction. In a 2025 study, Ekin Akyürek and colleagues applied test-time training to language models on hard reasoning benchmarks, including the Abstraction and Reasoning Corpus, a set of visual puzzles designed to test novel problem-solving from a handful of demonstrations. They created additional training examples for each test problem by transforming its demonstrations and fine-tuned lightweight adapter parameters on them, and reported accuracy gains of up to several-fold on that benchmark compared with the same fine-tuned models used without test-time training. Test-time training of this kind is an inner loop of gradient descent applied to a model whose initial weights come from pretraining, which is to say it is MAML's idea with the outer loop replaced by large-scale pretraining. Table 4 compares the main ways a pretrained model can now be adapted to a new task from few examples. Table 4. Ways to adapt a large pretrained model to a new task from few examples. Approach What changes Cost per new task Typical weakness In-context learning Prompt only One forward pass Weak far from training data; context limit Full fine-tuning All weights Full training run Overfits with very few examples Parameter-efficient fine-tuning Small added weight set Short training run Needs some data and tuning Test-time training Small weight set, per problem Short run per problem Compute at inference time Explicit meta-training All weights, across tasks Costly once, cheap per task Needs a well-designed task set Source: synthesis of approaches discussed in this chapter and chapter seven. What in-context learning tells us about meta-learning The study of in-context learning has turned out to be the most productive test of meta-learning's theory. It confirms that fast learning emerges from slow learning on a distribution of tasks. It shows that the properties of the task distribution, including its diversity, its burstiness and its long tail, determine whether a model learns to learn or learns to memorise. It shows that the learned inner algorithm reflects the structure of the task distribution, converging on recognisable algorithms such as gradient descent when those are optimal. And it shows that a model's adaptation may be mostly recognition of familiar tasks or genuine learning of new ones, depending on scale and on how far the new task lies from training. None of this makes the earlier methods obsolete. It does place them in context. The specialised algorithms of chapters three to six are ways of extracting a prior from a modest, explicit task distribution. In-context learning in large models is what happens when the task distribution is implicit and immense. The concepts that describe one describe the other. The practical question for anyone building a system is which situation they are in, and the final chapter turns to that question. Chapter 9: Designing Systems That Generalise to New Tasks The previous chapters built up a single picture. A system that learns a new task from a few examples does so by drawing on a prior acquired from experience with related tasks. That prior can live in a representation, a starting point, an update rule or a learned computation, and the choice among these shapes how the system adapts. Above all, the prior is only as good as the distribution of tasks it was learned from. This chapter turns that picture into practice. It is addressed to someone facing a real problem in which new tasks keep arriving with little data each: new products to classify, new patients to model, new environments for a robot, new molecules to screen. The questions it addresses are which approach to choose, how to construct the tasks that the system will learn from, how to tell whether it has learned to learn, and what has worked in the domains where meta-learning has been applied seriously. First question: is there a usable pretrained model? The most consequential decision comes before any meta-learning algorithm is considered. If a large model pretrained on broad data exists for the domain, and its representations or in-context abilities are relevant to the new tasks, the best starting point is almost always to use it. The baseline results of chapter seven and the in-context learning results of chapter eight both point the same way. For image classification of everyday objects, text classification in widely used languages, and many similar problems, a strong pretrained model with a simple adaptation step, such as a linear classifier on its features, a few examples in its prompt, or a short parameter-efficient fine-tune, will match or beat a meta-learning method built from scratch. Explicit meta-learning earns its place in three situations. The first is when no suitable pretrained model exists, because the data are specialised: molecular measurements, sensor readings from particular industrial equipment, physiological signals, robot proprioception. The second is when adaptation must happen under constraints that rule out large models or long fine-tuning: on a phone, in a control loop running many times per second, or without sending data off a device. The third is when a pretrained model exists but the specific act of adapting to new tasks from few examples needs to be optimised, in which case a stage of episodic meta-training on top of pretraining, as in the three-stage recipe described in chapter seven, often adds measurable gains. Having decided that meta-learning is warranted, the choice of family follows from the problem's structure, as summarised earlier in Table 3. For classification with a fixed representation that is likely to transfer, a metric method such as Prototypical Networks is fast, stable and hard to beat. When new tasks may require the representation itself to change, or when the problem is regression or control, an optimisation-based method such as MAML, Reptile or one of their more stable variants is the natural choice. When adaptation must happen continuously from a stream of experience, as in reinforcement learning, a recurrent or attention-based black-box learner may be necessary. A hybrid, such as initialising a final layer from prototypes and then taking a few gradient steps, is often the most robust option in practice. Defining what a task is The design decision that receives the least attention in papers and matters most in practice is the definition of a task. Benchmarks hand it over ready-made: a task is a random subset of N classes. Real problems rarely do. Consider a retailer that wants to classify product images into categories, where new categories are added regularly with a handful of examples each. A task might be "distinguish among five random categories", mimicking benchmarks. Or it might be "distinguish among the categories within one department", which better reflects how the system will be used, since new categories usually arrive in an existing department and must be told apart from their neighbours. The second definition produces harder, more realistic episodes, and a meta-learner trained on it learns the fine distinctions that matter. Or consider a hospital building a model to predict deterioration in patients. A task could be one patient, with the support set being that patient's first few hours of measurements. It could be one hospital ward, or one hospital in a network. Each definition corresponds to a different kind of new situation the model will face, and a model meta-trained across patients will learn to personalise to individuals, while a model meta-trained across hospitals will learn to adapt to different clinical practices and populations. The general principle is that meta-training tasks should be constructed to resemble, as closely as possible, the tasks the system will face at deployment, in their size, difficulty, and the kind of novelty they contain. This is the Matching Networks principle of training as you test, applied not to the format of episodes but to their content. Diversity, exclusivity and augmentation Three properties of the task distribution recur throughout the research reviewed in this book. The first is diversity. Baxter's theory, the transformer studies of Raventós and colleagues, and practical experience on benchmarks such as Meta-Dataset all point the same way: a meta-learner trained on more varied tasks generalises further. A meta-learner trained on a narrow family will learn a narrow prior, and new tasks outside that family will fall outside its competence, sometimes without warning. Where possible, the meta-training set should include tasks from the edges of the expected deployment distribution, not only its centre. The second is that tasks must require the support set. Chapter five described the memorisation problem: if a single function can solve all training tasks, the meta-learner will learn that function and ignore the support set. The test is simple to state. For each training task, could a model that has memorised all the other training tasks solve this one without looking at its support set? If so, the task distribution permits memorisation. The remedies are to shuffle labels where the problem allows it, to add task-specific randomness that can be inferred only from the support set, as in meta-augmentation, or to regularise the meta-learner against storing task information in its weights. The third is quantity, which is often the binding constraint. When only a few dozen genuine tasks are available, meta-overfitting is likely. Several techniques can create more. Task augmentation generates new tasks by transforming existing ones, for example by rotating all images in a task by the same angle to create a new class. Mixing methods, such as MetaMix by Huaxiu Yao and colleagues in 2021, interpolate between examples within or across tasks to create new training episodes. Unsupervised meta-learning constructs tasks from unlabelled data: in a 2019 method called CACTUs, Kyle Hsu, Sergey Levine and Chelsea Finn clustered the embeddings of unlabelled images and treated the clusters as pseudo-classes, generating episodes from them, and found that meta-learning on these automatically constructed tasks produced representations that were useful for genuine few-shot tasks later. Evaluating honestly A meta-learning system can look excellent and be useless, because the ways of evaluating it badly are numerous and tempting. Five principles, drawn from the failures documented in earlier chapters, guard against the most common errors. 1. Hold out tasks, not just examples. The meta-test set must contain tasks the system has never seen in any form during meta-training or hyperparameter selection. If tasks are defined by patients, test patients must be entirely absent from training. If they are defined by classes, test classes must be absent, and so should near-duplicates of them. 2. Split by the unit of real novelty. If deployment will bring new hospitals, hold out entire hospitals; if it will bring new product departments, hold out entire departments. Splitting randomly at a finer level, such as holding out some patients from each hospital, produces an optimistic estimate that will not survive deployment, as tieredImageNet's category-level split demonstrated for image classification. 3. Report the spread, not just the mean. Average accuracy over many episodes conceals large variation between them. Report the distribution of per-episode performance, and look specifically at the hardest episodes, since a few-shot system in use will meet them. 4. Check for leakage from pretraining. When a model has been pretrained on large external data, establish whether the test tasks, or close relatives of them, appeared in that data. Without this check, apparent few-shot learning may be recall. 5. Compare against strong simple baselines. Every evaluation should include a well-tuned version of pretraining followed by a linear classifier or fine-tuning, and, where relevant, in-context learning with a pretrained model. The lesson of chapter seven is that elaborate methods are frequently matched by these, and a result that does not include them cannot be trusted. Where decisions carry real consequences, a sixth principle applies: evaluate calibration. A few-shot model's confidence should reflect how ambiguous its support set is. The probabilistic methods of chapter five exist for this purpose, and even simpler methods can be checked by measuring whether their predicted probabilities match their observed accuracy across many test episodes. Robotics: adapting in seconds Robotics has been one of the most natural applications of meta-learning, because robots constantly face variations that are too numerous to train for individually and must be handled in real time. In 2017 Chelsea Finn, Tianhe Yu and colleagues showed that a robot could learn a new manipulation task, such as placing an object into a particular container, from a single video demonstration, after meta-training across many demonstrations of related tasks with different objects. The meta-learned policy used the demonstration as its support set, adapting with a gradient step so that it imitated the demonstrated behaviour with new objects in new positions. In 2019 Anusha Nagabandi and colleagues addressed a different problem: adapting online when the robot's own dynamics change. They meta-trained a model of how a robot's actions affect its state, so that a few recent observations could update the model quickly. A small six-legged robot using this approach could adjust to conditions it had not encountered in training, such as a damaged leg, a slope, or a payload, by updating its dynamics model from the last few moments of experience and replanning. Here the task was defined not as a different goal but as a different version of the robot's own body and environment, and the inner loop ran continuously as conditions changed. These applications illustrate both the promise and the requirement. The adaptation was fast because the meta-training distribution included variations of the kind encountered at test time: different objects in one case, different dynamics in the other. A robot meta-trained only on flat ground does not learn to adapt to stairs. Drug discovery: the data-poor science Early-stage drug discovery is full of few-shot problems. For a new protein target, a laboratory may have measured the activity of only a few dozen molecules, and must decide which of millions of candidates to test next. Each target is a task, the measured molecules are the support set, and there are many related targets with their own small datasets. The FS-Mol benchmark, released in 2021 by Megan Stanley and colleagues at Microsoft Research, organised several thousand such tasks from public bioactivity data into a few-shot learning benchmark, with a training set of tasks and held-out test tasks, each test task supplying a small support set. Its initial results were characteristic of the field as a whole. Prototypical Networks performed well among the methods tested, a model pretrained across many tasks and then fine-tuned was competitive, and the gains from all methods over simple single-task baselines, though real, were more modest than in image benchmarks. The chemistry of different targets shares structure, but less than images of different object classes do, and the prior a meta-learner can extract is correspondingly weaker. This is an honest picture of meta-learning in many scientific settings: helpful, worth doing, and limited by how much the tasks have in common. Personalisation and language Personalisation is meta-learning in disguise. A keyboard that adapts to its user's vocabulary, a recommender that adapts to a new customer, or a speech recogniser that adapts to a speaker are all fitting a task, one user, from a small support set, that user's early interactions. In federated learning, where models are trained across many devices without centralising data, Alireza Fallah, Aryan Mokhtari and Asuman Ozdaglar showed in 2020 that the goal of personalisation could be framed directly as MAML: learn a shared initialisation across users such that a few local gradient steps on each user's device produce a good personal model. Each device performs the inner loop privately, and only the information needed for the outer loop is shared. In language, an early application was low-resource machine translation. In 2018 Jiatao Gu and colleagues applied MAML to translation, treating each of many high-resource language pairs as a task and meta-learning an initialisation from which a model for a new low-resource language could be fine-tuned with little data. The approach outperformed standard multilingual transfer in their experiments. In the years since, large multilingual pretrained models have taken over much of this ground, an instance of the pattern from chapter seven in which broad pretraining absorbs the role that explicit meta-learning initially filled. When the tasks keep drifting One setting deserves a separate warning, because it is common in practice and poorly served by the standard framework. Meta-learning assumes that new tasks come from the same distribution as training tasks. In many deployments the distribution itself drifts over time. The kinds of fraud a payment system sees change as fraudsters adapt; the products a retailer sells change with fashion; the patients a hospital treats change with its catchment and its protocols. A prior learned from last year's tasks becomes progressively less suited to this year's, and nothing in the inner loop can detect that the prior has gone stale. Two responses are available. The first is to treat meta-training as a recurring process rather than a one-time event, periodically adding recent tasks to the meta-training set and retraining, with evaluation always on the most recent held-out tasks rather than on a fixed historical split. The second is to monitor the adaptation itself: if the inner loop starts to need more examples or more steps to reach the accuracy it once reached quickly, or if adapted models become less confident on query data, the prior is probably drifting away from the tasks. Research on online and continual meta-learning addresses these problems directly, but even without specialised methods, a system that tracks how hard adaptation has become has an early indicator that its slow learning needs renewing. A short checklist The guidance of this chapter can be compressed into a sequence of questions to ask before building a system that must learn new tasks from little data. 1. What, exactly, is a new task in deployment, and how much data will each one supply? 2. Is there a pretrained model whose knowledge covers these tasks, and how well does it do with simple adaptation? 3. If explicit meta-learning is needed, where should the prior live: a representation, a starting point, an update rule, or a learned computation? 4. Do the meta-training tasks resemble deployment tasks in size, difficulty and kind of novelty? 5. Does every training task require its support set, or can it be solved by memorisation? 6. Are there enough distinct tasks, and if not, can more be constructed by augmentation or from unlabelled data? 7. Is evaluation split at the unit of real novelty, with strong baselines, reported variation, checks for leakage, and, where it matters, calibration? None of these questions is about the inner workings of any particular algorithm. That is deliberate. The research of the last decade has shown repeatedly that, given reasonable choices of method, the decisive factors are the task distribution, the breadth of prior experience, and the honesty of evaluation. Conclusion The claim that a machine "learned from only five examples" is now made routinely, about research systems and commercial products alike. Having followed the history of meta-learning from Harlow's monkeys to in-context learning in large transformers, it should be clear that the claim is always incomplete. Five examples plus what? Every system that learns fast does so on top of something learned slowly, and the honest description of any few-shot result names both. A prototype classifier learning a new bird species from five photographs is drawing on the millions of images that shaped its embedding. A language model translating a new phrase pattern from three examples in its prompt is drawing on the vast range of text that shaped its weights. The five examples select and refine; the prior does the work. This is the most practical thing to take from the book. When evaluating any claim of few-shot or zero-shot learning, ask what the system's prior experience was, how the new task relates to it, and whether the test genuinely measures adaptation to something new or recognition of something familiar. Those three questions would have saved the field several years of over-interpreted benchmark results, and they will be just as useful for judging the next generation of systems. The task distribution is the design The chapters on specific methods, from Prototypical Networks to MAML to learned optimisers, could leave the impression that the main choice in meta-learning is the algorithm. The evidence assembled across the book points the other way. On standard benchmarks, different algorithms with similar network sizes and training budgets perform much more alike than their papers suggested. MAML, examined closely, turned out to learn mostly a representation, which is what metric methods learn directly. Black-box transformers trained on regression episodes discovered gradient descent. When the outer loop is given a rich enough task distribution, it tends to find the prior that distribution calls for, somewhat regardless of the container it was offered. What the outer loop cannot do is fix a bad task distribution. Give it tasks that can be solved by memorisation, and it memorises. Give it tasks that are too few, and it overfits them. Give it tasks that lack diversity, and, as the transformer studies showed with unusual precision, it learns to recognise the familiar rather than to learn the new, with a threshold of diversity separating the two behaviours. Give it data without the burstiness and long tail of natural language, and in-context learning does not emerge at all. It follows that the central engineering artefact of a meta-learning system is not the model or the algorithm but the collection of tasks it is trained on. That collection deserves the care usually given to architectures: deliberate design, documentation, analysis of what it covers and what it omits, and testing against the failure modes catalogued in chapters five and nine. This applies with even greater force to large pretrained models, whose task distribution is implicit in their training data and has rarely been designed with adaptation in mind. Understanding what implicit tasks a corpus contains, and therefore what kinds of in-context learning it will support, is a research problem that has barely begun. The line between learning and meta-learning is dissolving The book has treated meta-learning as a distinct field, and for most of its history it was one, with its own benchmarks, workshops and methods. That separation is fading, and the reasons are instructive. Large-scale pretraining is meta-learning whenever the training data contain an implicit distribution of tasks, which for natural language and web images they always do. Instruction tuning on collections of formatted tasks is episodic meta-training under another name. Test-time training on a model's inputs is MAML's inner loop applied to a pretrained initialisation. Personalised federated learning is MAML across devices. The vocabulary of support sets, inner loops and task distributions now describes much of what happens in mainstream machine learning, even where the vocabulary is not used. What remains distinctive is the explicit attention to the two timescales and the relationship between them. Ordinary training asks how well a model does on its data. Meta-learning asks how well a model, after a brief exposure to new data, does on a new task. As models are increasingly deployed into settings that differ from their training, adapted by prompts, fine-tuned on small datasets, and expected to handle tasks their builders did not anticipate, the second question becomes the one that matters. The field's concepts will outlast its separate identity. What remains unsettled Three questions are open, and each follows from the argument of the book rather than standing apart from it. The first is whether a task distribution can be broad enough to produce general learning ability, or whether every meta-learned prior, however large, remains a specialist in its family. The evidence is mixed. Large models adapt in context to a remarkable range of tasks, and larger models increasingly learn genuinely new mappings rather than merely recognising familiar ones. Yet benchmarks built specifically to require novel abstraction from a few examples, such as the Abstraction and Reasoning Corpus, have remained hard for in-context learning alone, and the large gains reported from test-time training on such benchmarks suggest that a forward pass is not enough when the task is far from the training distribution. The most defensible reading is that breadth of experience extends the range of fast learning enormously but does not abolish the boundary. Somewhere beyond the training distribution, a learner must fall back on slower, more general procedures, which is exactly where gradient-based adaptation retains its value. The second is where adaptation should live. The book has described four homes for a prior and several ways of adapting to a task: in activations, in a small set of parameters, in all the weights. Each has costs and benefits, set out in Table 4. Systems that must adapt continually over long periods, accumulating knowledge about a user or an environment without forgetting what they already know, need some combination: fast, temporary adaptation in context and slower, durable adaptation in weights, with a principled way of deciding what moves from one to the other. Biological brains appear to have such an arrangement. Machine learning systems mostly do not, and the meta-learning perspective, which treats the division between fast and slow learning as itself something to be learned, is the natural framework for designing one. The third is how to measure learning when novelty cannot be verified. Meta-learning's evaluation rests on holding out tasks the system has never seen. With pretraining on data at the scale of the public internet, that guarantee is increasingly hard to give. A test task may have appeared, or something very like it may have appeared, in training data that no one has fully inspected. Without a way to establish what a model has and has not seen, claims of few-shot generalisation become hard to distinguish from claims of good recall. Better tools for auditing training data, and benchmarks constructed after a model's training data was fixed or built from procedurally generated tasks with verifiable novelty, are not peripheral concerns. They are what makes the central claim of the field testable. The child and the okapi, again The introduction began with a child learning a new animal from a single picture, and it is worth ending there. The child is a meta-learner in every sense the book has described. Her prior was built from years of looking at animals, hearing them named, and noticing which differences between them matter. Her fast learning works because the okapi belongs to a family of tasks, naming animals, that her slow learning has prepared her for. Shown a single example of a new kind of mathematical object, she would learn far less from it, because her prior does not reach that far yet. What distinguishes her from the systems in this book is not that she learns without a prior. It is that her experience was not designed for any particular benchmark. It was broad, varied, bursty, and full of novelty; it mixed perception with action and language with consequence; and it continued throughout her life, so that each new thing learned became part of the prior for the next. If there is a single direction in which meta-learning research is heading, it is towards giving machines experience of that kind, and towards understanding precisely how the shape of that experience determines what they can learn next. The algorithms will keep changing. The principle that fast learning is purchased with slow learning, and that the purchase is made through the choice of tasks, will not. Hashtags: #MetaLearning #LearningToLearn #ArtificialIntelligence #MachineLearning #DeepLearning #FewShotLearning #OneShotLearning #ModelAgnosticMetaLearning #MAML #ReptileAlgorithm #PrototypicalNetworks #MatchingNetworks #SiameseNetworks #MetricLearning #EpisodicTraining #TransferLearning #InContextLearning #InductiveBias #NeuralNetworks #GradientBasedMetaLearning #BilevelOptimization #LearnedOptimizers #MetaReinforcementLearning #RepresentationLearning #TaskAdaptation #RapidAdaptation #NeuralNetworkOptimization #AIResearch #AdaptiveAI #GeneralizationInAI

Download the Book (PDF): Introduction At a quarter to six on a Tuesday, the parking lot of a community college looks nothing like the brochure. There are no students lounging on the lawn. There are people walking fast from their cars in work clothes, some still wearing lanyards or scrubs or steel-toed boots, one carrying a laptop bag and a fast-food sack in the same hand. A woman sits in her car with the engine running, finishing a phone call about who is picking up her son from practice. A man in a delivery uniform eats a granola bar in four bites while he walks. By six o'clock, most of them are seated in classrooms, and for the next three hours they will be asked to think about organic chemistry, accounting principles, nursing pharmacology, or the causes of the First World War. These are evening students, and there are far more of them than most people imagine. The Lumina Foundation, which tracks the changing profile of American higher education, reports that about a third of college students are 25 or older, that roughly four in five work while enrolled and about 30 percent of those work full time, and that close to one in five has children or other dependents.[1] The eighteen-year-old living in a residence hall, with no job and no one to care for, is no longer the typical student. For a large share of learners, college happens after something else: after a shift, after daycare pickup, after a commute, after a day that has already asked a great deal. This book is for them, and for the partners, parents, advisers, instructors and employers who want to help them finish. It deals with three practical questions that come up again and again for anyone whose classes start when other people are heading home. When and what should I eat, if the dinner hour falls in the middle of a lecture? How should I use caffeine, which is the obvious tool for staying awake through a long evening but also the obvious threat to the sleep I need? And how do I get home safely, whether that means a forty-minute drive on a dark highway, a wait at a bus stop, or a walk across a half-empty campus to a parking garage? Why the usual advice falls short There is no shortage of general wellness advice. Eat balanced meals, get seven or more hours of sleep, avoid caffeine late in the day, don't drive tired. All of it is correct. Very little of it is written for someone whose day is organized differently from the day the advice assumes. Consider the standard instruction to stop drinking coffee in the early afternoon. For someone who works from eight to four and then sits in a statistics lecture from six to nine, that instruction is not merely inconvenient; it asks them to face the hardest intellectual work of the day with the least help. Or consider the common suggestion to eat dinner early and lightly. A student who leaves work at 4:30, drives straight to campus and sits in class until 9:15 has no early dinner to eat, unless she eats it at her desk at three in the afternoon. And advice about "avoiding drowsy driving" is of little use to someone who has no alternative to driving home at ten at night after sixteen hours awake. The problem is not that general guidance is wrong. It is that the guidance was written around a clock the evening student does not live on. Night classes stretch the day at both ends of its natural shape: they push mental effort into hours when the body is preparing to wind down, they break the usual rhythm of meals, and they place the most dangerous part of the day, the trip home, at the point of greatest fatigue. Good habits for this schedule have to be built for this schedule. The central idea The argument of this book is simple enough to state in a sentence: plan the evening backward from the time you intend to be asleep, not forward from the time class starts. Most evening students plan forward. Class is at six, so I have to leave work by five, so I will grab something to eat on the way, so I will have a coffee to get through it, and then I will go home and deal with whatever is left. Every one of those decisions is made to serve the next hour, and the costs of all of them arrive together at the end of the night, when the student is trying to sleep with caffeine still in the bloodstream, a heavy meal still in the stomach, a mind still racing from the lecture and, often, a household that has been waiting for them. Planning backward reverses the logic. If you need to be asleep by 11:30 to get seven hours before a 6:30 alarm, then your last substantial caffeine belongs somewhere around the middle of the afternoon, your largest meal belongs before class rather than after it, your drive home has to be something you can do alert, and the half hour after you walk in the door has a specific job to do. The anchor point is sleep, because sleep is the one part of the day that cannot be compressed indefinitely without the rest of the day falling apart. Everything else can be arranged around it. This is not a counsel of perfection. Evening students have little slack, and plenty of nights will not go to plan. The value of a backward plan is that it tells you which compromises are cheap and which are expensive. Eating a sandwich in the car is a cheap compromise. A large energy drink at the break is an expensive one, because you will pay for it at one in the morning. Driving home alone when you are nodding off at red lights is the most expensive compromise of all, and this book will be blunt about it. What this book covers The chapters move roughly in the order of an evening. Chapter 1 explains the biology that makes night classes feel the way they do: the two systems, one tracking time of day and the other tracking how long you have been awake, that together determine how alert you are at any given hour. Understanding them explains why you can feel lively at seven and wrecked at nine-thirty, and why the late afternoon is so treacherous. Chapter 2 turns that biology into a method, showing how to build an evening plan backward from bedtime and how to adjust it for different schedules. Chapters 3 and 4 take on food and caffeine, the two levers students reach for most often and understand least. Chapter 3 looks at what the research says about late eating and how to feed yourself when there is no dinner hour. Chapter 4 explains how long caffeine really lasts, what the federal guidance on daily intake means in practice, and how to calculate a personal cut-off time. Chapter 5 deals with the classroom itself: how to stay attentive through a two- or three-hour session when the day has already been long. Chapters 6 and 7 cover the journey home, first by car, where drowsiness is the main danger, and then on foot and by public transit, where darkness and isolation change the risks. Chapter 8 addresses the part of the evening most students neglect entirely, the transition from class to sleep, and Chapter 9 widens the lens to the families and workplaces that surround every evening student and that can make these habits either possible or impossible. The conclusion does not summarize; it argues for a particular way of thinking about the whole term, not just the single night. A few notes before starting The scenarios in this book are illustrative composites, built to show how the principles work in realistic situations. They are not case studies of real people. Figures and research findings are drawn from published sources, listed in the Notes at the back. This book offers general information, not individual medical advice. People differ in how they metabolize caffeine, in how their digestive systems handle late meals, in their sleep needs and in their health conditions. If you are pregnant, have a heart condition, anxiety disorder, reflux disease, diabetes, a sleep disorder or any other medical condition, or take medications that interact with caffeine or affect alertness, talk with a doctor, nurse practitioner or pharmacist about how the general guidance here applies to you. If you ever feel chest pain, a racing or irregular heartbeat, or confusion after consuming caffeine or energy products, treat it as a medical emergency. And if the strain of combining school, work and family ever leaves you thinking about harming yourself, call or text 988 to reach the 988 Suicide and Crisis Lifeline in the United States, at any hour. Finally, a word about tone. Evening students are frequently the most motivated people on a campus. They have chosen to add study to lives that were already full, usually for reasons that matter a great deal to them: a better job, a career change, a promise to themselves or to their children. They do not need to be told to try harder. What they need, and what this book tries to offer, is a clearer picture of how their bodies behave in the evening and a set of practical ways to work with that behavior instead of against it. The goal is not only to get through the semester, but to get through it healthy, rested enough to learn, and safely home every night. Chapter 1: The Evening Student's Body Ask a group of night students when they feel worst during a typical class day and the answers cluster in two places. The first is somewhere between three and five in the afternoon, often in the car or on the bus on the way to campus, when the eyelids feel heavy and the idea of sitting through three hours of lecture seems absurd. The second arrives late in the evening, sometimes during the last half hour of class, more often on the way home. Many of the same students will also report something that puzzles them: a stretch in the early evening, around the first break, when they feel surprisingly clear-headed, even energetic, despite having been awake since dawn. None of this is imagined, and none of it is a failure of discipline. It is the predictable output of two biological systems that govern alertness in every human being. Understanding them is the foundation for everything else in this book, because once you know why your energy rises and falls when it does, you can stop fighting the pattern blindly and start planning around it. Two forces that decide how awake you are Sleep scientists have described the regulation of sleep and wakefulness for decades with what is known as the two-process model, first set out by the Swiss researcher Alexander Borbély in 1982.[1] The model has been refined many times since, but its central insight has held up remarkably well. At any moment, how sleepy or alert you feel is the result of two separate processes interacting. The first is sleep pressure, sometimes called the homeostatic sleep drive. It works much like hunger. From the moment you wake up, pressure to sleep begins to build, and it keeps building for every hour you stay awake. Sleep discharges it; staying awake accumulates it. One of the chemical signals involved is adenosine, a molecule that gradually builds up in the brain during waking hours and that promotes sleepiness by acting on specific receptors. This matters for the evening student because caffeine, as Chapter 4 will explain in detail, works mainly by blocking those same adenosine receptors. It does not remove sleep pressure. It hides it. The second process is the circadian rhythm, an internal clock that runs on a cycle of roughly twenty-four hours. Its master timekeeper is a small cluster of cells in the hypothalamus called the suprachiasmatic nucleus, which receives information about light directly from the eyes and uses it to keep the body's many internal rhythms aligned with the outside day. The circadian clock drives changes in body temperature, hormone release, digestion and, crucially, alertness. Unlike sleep pressure, it does not simply increase or decrease; it cycles, pushing toward wakefulness at some times of day and withdrawing that push at others. The two forces interact in a way that can seem counterintuitive. Across a normal day, the circadian system actually pushes harder toward wakefulness as the day goes on, rising into the evening. It does this precisely to offset the steadily mounting sleep pressure. Without it, a person who woke at six in the morning would be struggling to stay awake by mid-afternoon and would be barely functional by evening. Then, a couple of hours before your usual bedtime, the circadian push withdraws, the hormone melatonin begins to rise, and the accumulated sleep pressure is finally allowed to show itself. That is when sleep comes easily, if you let it. This is the most important piece of biology for a night student to understand. The clarity you may feel in the early evening is not a fluke. It is your circadian system working at full strength to keep you awake against a day's worth of accumulated pressure. Researchers have even given part of this period a name. In the 1980s the Israeli sleep scientist Peretz Lavie, studying people who were given brief sleep opportunities around the clock, found that it was unusually hard for participants to fall asleep in the few hours before their normal bedtime, and described this stretch as a "forbidden zone" for sleep.[2] Later researchers often call it the wake maintenance zone. For many evening students, the wake maintenance zone overlaps with the first half of class. It is a genuine asset: a biologically supported window of alertness that happens to line up with lecture time. It also has limits. When it ends, often in the later part of the evening, the full weight of the day's sleep pressure lands at once. That is why many students feel fine at seven and dreadful at nine-thirty. Nothing has gone wrong between those hours. The biological support has simply been withdrawn on its usual schedule. The treacherous late afternoon The first slump of the day, the one that tends to arrive between roughly two and five in the afternoon, has a different explanation. Many people blame it on lunch, and lunch can make it worse, but the dip happens even in people who have not eaten. The sleep researcher Timothy Monk, reviewing decades of work on what is often called the post-lunch dip, concluded that it is a real phenomenon rooted in human biology, one that can occur even when a person has had no lunch and does not know what time of day it is, and that a heavy, carbohydrate-rich meal makes it more pronounced.[3] In the two-process picture, the afternoon dip is a moment when the circadian clock's wake-promoting signal has not yet risen high enough to fully offset the sleep pressure built up since morning. It is a trough between the morning's alertness and the evening's circadian support. For most workers, it is an inconvenience. For night students, it is a hazard, because it tends to coincide with the transition between work and school: the drive to campus, the hurried meal, the wait for class to begin. This has three practical consequences that later chapters will develop. First, the afternoon is the time when caffeine is most tempting and, for many schedules, most defensible, as long as it is timed with bedtime in mind. Second, the commute to campus deserves more caution than most students give it, especially after a physically demanding shift or a short night's sleep. Third, the meal eaten in that window should be designed to avoid deepening the dip, which usually means avoiding a very large, carbohydrate-heavy plate immediately before sitting down in a warm lecture hall. Owls, larks and the rest of us Not everyone's clock runs on the same schedule. People differ in what researchers call chronotype: their natural preference for earlier or later timing of sleep and activity. Extreme "larks" wake early without effort and fade early in the evening; extreme "owls" struggle to wake in the morning and come alive at night. Most people sit somewhere in between. Chronotype is partly inherited and shifts with age: it tends to become later through adolescence, peaks in lateness around the end of the teenage years and early twenties, and then drifts earlier through adulthood. The chronobiologist Till Roenneberg and colleagues, who have collected data on sleep timing from very large numbers of people, have written extensively about how strongly these differences shape daily life.[4] Chronotype matters for night students in two directions. An owl may genuinely thrive in evening classes. The lecture falls in their best hours, and the problem is not staying awake in class but getting up for the job the next morning. For a lark, especially an older student whose chronotype has drifted earlier, a class that runs until nine or ten at night pushes against their biology. They may feel the evening slump earlier and harder, and they may struggle more on the drive home. You probably already know roughly where you fall. If you are unsure, notice what happens on a free week, such as a vacation, when you can sleep and wake without an alarm. The times you settle into after a few days are a reasonable guide to your natural rhythm. It is worth knowing, because the rest of this book's advice should be tuned to it. A lark taking a class that ends at 9:45 has less margin for error on caffeine timing and the drive home than an owl in the same seat. How much sleep you actually need The biology of sleep pressure implies something uncomfortable: the only thing that truly repays it is sleep. In 2015 the American Academy of Sleep Medicine and the Sleep Research Society published a joint consensus statement concluding that adults should sleep seven or more hours per night on a regular basis to promote optimal health, and that regularly sleeping less than seven hours is associated with a range of adverse outcomes, including impaired performance, more errors and greater risk of accidents.[5] Some people, particularly young adults and those recovering from sleep debt, may benefit from more than nine hours. Evening students are at obvious risk of falling short. If class ends at 9:30, the drive takes half an hour, and there is anything at all to do at home, going to bed before 11 is a challenge. If the alarm goes off at 5:45 for work, the arithmetic simply does not reach seven hours. One or two nights like that are survivable. A semester of them, week after week, produces a condition sleep researchers call chronic partial sleep restriction, in which performance declines steadily even though the person often reports feeling only moderately tired. The unsettling finding from this line of research is that people adapt to feeling tired much faster than they adapt to being impaired. Your sense of how well you are functioning becomes an unreliable instrument precisely when you most need it. That unreliability shows itself in several ways that matter to students. Attention wavers more often and for longer. Reading comprehension suffers; you find yourself reading the same paragraph three times. Memory consolidation, much of which happens during sleep, is compromised, which means that the effort you spent in class is partly wasted if the night after it is short. Mood becomes more fragile, and small frustrations at home or work loom larger. At the extreme end are microsleeps, brief lapses of a few seconds in which the brain effectively goes offline, sometimes without the person realizing it. In a lecture, a microsleep costs you a sentence or two of notes. At sixty miles an hour, it can cost everything, which is why Chapter 6 treats drowsy driving as the single most serious risk in the evening student's day. Light: the clock's steering wheel The circadian clock does not keep perfect time on its own. In most people it runs slightly longer than twenty-four hours, and it needs daily correction. The main signal it uses is light, received through specialized cells in the retina that are especially sensitive to short-wavelength, bluish light. Bright light in the morning tends to shift the clock earlier; bright light in the late evening tends to shift it later and suppresses the release of melatonin. This creates a quiet problem for evening students. Classrooms are brightly lit. So are libraries, laptop screens, phone screens and the fluorescent corridors of many campus buildings. A student who spends the hours from six to ten in bright artificial light and then goes home to a lit kitchen and a phone screen is sending the clock a strong message that it is still daytime. Over weeks, this can push the circadian rhythm later, making it harder to fall asleep at the desired hour and harder to wake in the morning. Chapter 8 returns to this in detail, including the experimental evidence on screens before bed, and Chapter 5 explains how the same principle can be used deliberately, since light that is unhelpful at eleven at night can be useful at seven. The morning side of the light equation is just as important and costs nothing. Getting outdoor light, even on an overcast day, within the first hour or two after waking helps anchor the clock to an earlier schedule, which in turn makes it easier to fall asleep after a late class. For a working student, this can be as simple as walking the last part of the commute, drinking morning coffee by a window, or taking a short walk at a morning break. Food and the clock The circadian system does not only govern sleep. Almost every tissue in the body contains its own clock, and these peripheral clocks, including those in the liver, gut and fat tissue, are influenced not just by light but by the timing of meals. The body processes food differently at different times of day. Research on meal timing, reviewed in Chapter 3, suggests that eating the same food later in the day can change appetite hormones and energy use in ways that are not trivial. For now, the key point is that the timing of dinner is not a neutral choice. It interacts with the same biological clock that determines when you will feel sleepy. What this means for your evening Taken together, these systems produce a recognizable daily profile for most evening students. There is a period of reasonable alertness in the morning. There is a dip in the mid-to-late afternoon that coincides with the end of work and the trip to campus. There is a lift in the early evening, supported by the circadian push toward wakefulness, that carries the student into and through the first part of class. And then there is a drop, sometimes quite steep, once that circadian support begins to withdraw in the hours before the usual bedtime, a drop that typically lands during the last part of class or on the journey home. Seen this way, several habits that feel natural turn out to be poorly timed. Drinking the strongest coffee of the day at the class break, around 7:30, spends caffeine during a window in which the body is already doing a good job of keeping you awake, and ensures that much of that caffeine is still active when you are trying to sleep. Eating the biggest meal of the day after class, at ten or eleven at night, puts food into the system right as the clock is preparing for sleep. Driving home without any plan for the late-evening slump treats the most dangerous part of the day as if it were the safest. The good news is that the same knowledge points toward better timing. Caffeine used in the afternoon to cover the dip, and then stopped, can carry you to the evening's natural lift without poisoning your sleep. A real meal before class, sized and composed to avoid deepening the slump, can make a late snack unnecessary. A deliberate plan for the drive home, informed by an honest sense of how tired you are, can turn the riskiest half hour of the day into a manageable one. It also points toward a more forgiving view of yourself. If you have ever felt ashamed of nodding off in a late lecture, or of feeling unable to face a textbook at ten at night, it is worth knowing that you were fighting two of the most powerful regulatory systems in the human body. Willpower plays a role in any demanding schedule, but it works best when it is spent on arranging the day sensibly, not on overriding biology minute by minute. The next chapter shows how to do that arranging. Chapter 2: Planning the Evening Backward from Bedtime Most people plan a school night the way they plan a road trip with no fixed destination: one leg at a time. Class starts at six, so I leave work at five. I am hungry, so I stop for food. I am tired, so I buy coffee. Class ends at nine, so I drive home. I am wired and hungry, so I snack and scroll. I get to bed at half past midnight and wonder why the alarm feels like an assault. Every decision in that sequence is reasonable in isolation. The trouble is that each is made to solve the problem immediately in front of it, and the costs are all deferred to the same place: the night's sleep. By the time those costs arrive, it is too late to do anything about them. This chapter proposes a different way to plan, one that starts from the end of the evening rather than the beginning. Choosing the anchor The anchor of a backward plan is the time you intend to be asleep. Not the time you go to bed, and not the time you intend to start "winding down," but the time you want to actually be asleep. The distinction matters, because most people need somewhere between ten and thirty minutes to fall asleep after lying down, and more on nights when they arrive home stimulated. To choose the anchor, start from the morning. When does your alarm go off on the day after class? Count back at least seven hours, the minimum the American Academy of Sleep Medicine and the Sleep Research Society recommend for adults on a regular basis.[1] If your alarm is at 6:30, your anchor is 11:30 at the latest. If you know you function better with more, and many people do, move it earlier. For some students, this arithmetic produces an anchor that seems impossible. If class ends at 9:45, the commute takes forty minutes and the alarm goes off at 5:30 for an early shift, seven hours would require being asleep by 10:30, which is barely possible even with no delay at all. In that case, the backward plan does not magically create time, but it does something useful: it shows clearly that the schedule is running a deficit, which means the deficit has to be paid somewhere else. That might mean protecting sleep on non-class nights, planning a nap on a day off, adjusting the work schedule on the morning after class, or, in some cases, reconsidering the number of evening classes taken in one term. It is far better to see that problem on paper in the first week than to discover it in week nine, when exhaustion has already begun to erode grades and health. Working backward, step by step Once the anchor is set, the rest of the evening falls into place by counting backward from it. The sequence below is a template, not a script, and the chapters that follow explain each step in detail. 1. Asleep time. The anchor, chosen as above. 1. Lights out. Typically fifteen to thirty minutes before the anchor, more if you tend to lie awake after class. 2. Wind-down. A period of thirty to sixty minutes before lights out, with dim light, no heavy food, no work email and as little screen time as you can manage. Chapter 8 covers what belongs in this window. 3. Arrival home. The wind-down cannot start until you get home, which means the commute has to be accounted for honestly, including any stops. 4. Departure from campus. Class end time plus the minutes it takes to pack up, talk to the instructor, walk to the car or the stop and, if needed, wait for a campus escort or a bus. 5. Caffeine cut-off. A point several hours before the anchor, which Chapter 4 explains how to calculate from caffeine's half-life and the research on its effects on sleep. For most people with a late-evening anchor, this falls somewhere in the afternoon. 6. Main meal. Placed early enough that it is well digested before lying down, ideally two to three hours or more before sleep, as Chapter 3 discusses, which usually means before class or at the break rather than after. Written out this way, the plan looks almost bureaucratic. In practice it takes only a few minutes to work through for a given schedule, and once done it rarely needs revisiting until the timetable changes. What it produces is a small number of fixed points, typically a caffeine cut-off, a meal time, a departure time and a lights-out time, that turn the evening from a series of improvisations into something closer to a routine. Three sample evenings The principle is easiest to see in examples. The three evenings below are illustrative composites, not real people, chosen to represent common situations. The full-time worker with one long weekly class. Consider a 34-year-old logistics coordinator who works from 8:00 to 4:30 and takes an accounting course that meets on Wednesdays from 6:00 to 9:15, twenty-five minutes from home. His alarm goes off at 6:15. Seven hours before that is 11:15, so he sets his anchor at 11:00 to give himself a little margin. Working backward: lights out at 10:40, wind-down beginning at about 10:00, arrival home at about 9:50 after a few minutes of packing up and the twenty-five-minute drive. That leaves him only ten minutes of slack between arriving and starting his wind-down, which tells him that the drive home and whatever he does in his first minutes at home need to be simple. There is no room for a large meal after class, and no room for a wired hour on the couch. His caffeine cut-off, calculated as in Chapter 4, lands in the early to middle part of the afternoon. On Wednesdays, he moves his usual 3:30 coffee to 1:30 and makes it a smaller one. His main meal becomes a packed dinner eaten at his desk at 4:30 or in the car before class, sized to hold him until the morning with perhaps a small snack at the break. The result is not a transformed life. It is an evening in which the three hours of accounting still feel long, but in which he falls asleep at 11 rather than at 12:30, and in which Thursday morning is merely tired rather than wrecked. The parent with two weeknight classes. Now consider a 29-year-old medical assistant and single mother of a seven-year-old who takes two courses on Monday and Thursday evenings from 6:30 to 9:30, twenty minutes from home. Her mother watches her son on class nights. Her alarm goes off at 5:45 because she has to get her son ready for school before a 7:30 shift. Seven hours before 5:45 is 10:45. Working backward from 10:45 is sobering. Class ends at 9:30; she is home by about 10:00 at the earliest. Her mother usually wants to talk for a few minutes before leaving, and her son is sometimes still awake. A realistic lights-out is 10:45 and a realistic asleep time is 11:15, which gives her six and a half hours on class nights. The backward plan does not solve this, but it clarifies the options. She can protect sleep on the other five nights, making sure they are genuinely seven hours or more. She can talk with her mother about shifting the handover conversation to the next day by phone. She can ask her employer whether a slightly later start time is possible on Tuesdays and Fridays, the mornings after class. She can set a firm caffeine cut-off of early afternoon on class days, knowing that anything later will cost her the margin she cannot afford. And she can decide, before the semester begins, that she will not take a third evening class in the same term. The student who takes the bus. Finally, consider a 22-year-old retail worker whose shift ends at 3:00 and who takes a biology class with lab from 5:30 to 9:20. He does not own a car and takes two buses home, a trip that takes about fifty minutes at night because the second bus runs only every half hour after nine. His alarm on the following day is at 8:00 because he works a late shift. His anchor is easier: seven hours before 8:00 is 1:00, so he can be asleep by 12:30 with room to spare. His challenges are different. The long, uncertain transit ride means that his evening's end point is less predictable, and his safety plan for the walk to the stop and the wait matters a great deal, as Chapter 7 discusses. Because he is an owl who naturally stays up late, his risk is not so much short sleep on class nights as drifting later and later, eventually shifting his whole rhythm until he struggles on the mornings when he does have to get up early. His plan focuses less on caffeine and more on light and routine: a consistent lights-out, a dim, quiet wind-down after the bus ride and outdoor light the next morning. Why sleep is the anchor It is fair to ask why sleep should be the anchor rather than something else, such as study time, family time or work performance. There are three reasons. The first is that sleep is the least compressible part of the day. You can eat dinner in fifteen minutes instead of forty. You can commute by a different route. You can study in shorter blocks. But sleep that falls below what your body needs does not simply produce a proportionally shorter night's rest; it produces measurable impairment the next day, and the impairment compounds over consecutive nights. Research on chronic sleep restriction consistently finds that performance keeps declining across successive nights of short sleep even as people's sense of how sleepy they are levels off. In a widely cited laboratory study, Hans Van Dongen, David Dinges and colleagues restricted healthy adults to four, six or eight hours in bed for fourteen consecutive nights. Those limited to six hours showed steadily accumulating deficits in attention and cognitive performance, and by the end of two weeks their deficits were comparable to those seen after up to two nights of total sleep deprivation. Yet their ratings of their own sleepiness rose only modestly after the first few days, and the researchers concluded that participants were largely unaware of how impaired they had become.[2] Six hours a night is a common figure for evening students during a busy term. That study is a good reason to treat it as a warning rather than a norm. The second reason is that sleep serves learning directly. Much of the consolidation of new information into long-term memory takes place during sleep. A student who attends a three-hour lecture and then sleeps five hours has not merely lost two hours of rest; they have undermined the process by which the lecture becomes knowledge. For evening students, whose study time is scarce, this is a waste they can ill afford. The third reason is safety. The most dangerous thing most evening students do on a class night is travel home, and the biggest factor that makes that trip dangerous is sleep debt. A student who consistently gets enough sleep begins each class night with a smaller load of accumulated sleep pressure and faces the drive home with a larger margin of alertness. Anchoring on sleep is, among other things, an anchor on getting home. The weekly view A backward plan for a single evening is useful. A backward plan for a week is better, because the evenings are not independent. Two late nights in a row do more harm than two separated by a normal night, and weekend habits can undo the gains of careful weekdays. Start by marking the nights of the week when you have class and the mornings that follow. Those mornings are your pinch points. If you have any control over your work schedule, it is often worth spending it there: a later start, a lighter task load, a remote morning. If you do not, protect the nights before class nights instead, so that you enter each class evening with as little sleep debt as possible. Next, look at the non-class nights. They are not free time in the sense of time to be spent without limit. They are your recovery nights, and they should be treated as such, with a bedtime at least as early as your class-night anchor and ideally earlier. The temptation, particularly for students who feel they have neglected family or friends on class nights, is to stay up late on the other evenings to make up for it. That is understandable, and occasionally worth it, but as a regular pattern it turns every night into a short one. Finally, watch the weekend. Many adults sleep much later on Saturday and Sunday than during the week, a pattern Till Roenneberg and colleagues have described as social jetlag, because the body experiences it much as it would a flight across time zones.[3] A student whose weekday sleep runs from 11:30 to 6:30 and whose weekend sleep runs from 1:30 to 9:30 is effectively shifting two time zones every weekend. The Monday evening class after that shift can be especially difficult. Some extra sleep on weekends is reasonable if you are carrying a debt, but keeping wake times within an hour or so of the weekday time, and adding sleep at the start of the night rather than the end, keeps the clock steadier. When the plan breaks It will break. Class will run over. The instructor will want to talk. The car will not start. A child will be sick. The shift will run late and you will arrive at class hungry, with no packed dinner and no time to buy one. A backward plan is not a promise that the evening will go as intended; it is a reference point that lets you make better decisions when it does not. On a night when things go wrong, the plan helps you decide which rules to bend and which to hold. If you arrived without dinner, it is better to eat something modest at the break, or on the way home, than to go to bed ravenous or to eat a large meal at eleven. If you are running an hour behind on everything, it is better to cut the wind-down short and go to bed than to insist on the full routine and lose sleep. If you are exhausted and facing a long drive, the rule about driving alert is not one to bend at all; Chapter 6 explains the alternatives. A useful habit is a thirty-second review at the end of each class night, or the next morning, asking a single question: what cost me the most sleep tonight? Over a few weeks, the answer tends to repeat. For some students it is caffeine at the break. For others it is the half hour of scrolling after they get home, or a late meal, or a conversation that ran long. Once you know your particular leak, you can focus your effort there instead of trying to perfect every part of the evening at once. A plan written down For the plan to work, it helps to put it somewhere visible. Some students write their key times, caffeine cut-off, dinner, departure and lights-out, on an index card in their bag or a note on their phone. Others set a small number of phone alarms: one for the last coffee, one for dinner, one for the start of the wind-down. Alarms are not glamorous, but they turn a good intention into a prompt that arrives at the right moment, when your memory of the plan is likely to be crowded out by everything else going on. The rest of this book fills in the detail behind each of those fixed points, starting with the one that most evening students handle worst: the meal that has no natural place in their day. Chapter 3: Dinner Without a Dinner Hour For many evening students, dinner is the meal that disappears. Lunch happens at work. Breakfast, if it happens, is rushed. Dinner, the meal that most households organize the evening around, falls squarely in the hours when the student is either driving to campus or sitting in class. What replaces it tends to follow a pattern: nothing much between a noon lunch and a 9:30 class ending, then a large, fast and often high-fat meal at ten or eleven at night, eaten in the car, in front of the television or standing at the kitchen counter. That pattern has costs that go beyond nutrition in the narrow sense. It leaves the student hungry and distracted in the second half of class, it loads the stomach just before sleep, and, according to a growing body of research, it may shift the way the body handles food in ways that matter over months and years. This chapter looks at the evidence on late eating and then offers practical ways to feed yourself on a schedule that has no dinner hour. What the research says about late eating The study of meal timing, sometimes called chrononutrition, rests on the insight described in Chapter 1: the body's internal clocks extend to the organs that process food, and those organs do not behave identically at all hours. The same meal eaten at different times of day can produce different effects. One of the most carefully controlled experiments on this question was published in 2022 by Nina Vujović, Frank Scheer and colleagues at Brigham and Women's Hospital in Boston.[1] They recruited sixteen adults with overweight or obesity and had each of them go through two laboratory protocols in random order. In both protocols, participants ate exactly the same meals, and researchers controlled their sleep, physical activity and light exposure. The only difference was timing: in one protocol, the meals were scheduled about four hours later in the day than in the other. When participants ate late, they reported more hunger, their ratio of the hunger-promoting hormone ghrelin to the satiety hormone leptin rose, they burned calories at a slower rate while awake, and their fat tissue showed changes in gene activity consistent with storing more fat and breaking down less. The authors concluded that these were converging mechanisms by which late eating may increase the risk of weight gain. This was a small study in a specific group, and the researchers themselves noted that it needs to be extended, including to more women. But it is consistent with other work. An earlier study led by Marta Garaulet followed 420 people in Spain through a twenty-week weight-loss program and compared those who ate their main meal, which in that Mediterranean population was lunch, before 3 p.m. with those who ate it later.[2] The late eaters lost less weight and lost it more slowly, even though their reported calorie intake, diet composition, estimated energy expenditure, appetite hormone levels and sleep duration were similar to those of the early eaters. As an observational study, it cannot prove that meal timing caused the difference, but it points in the same direction as the laboratory evidence. It is important not to overstate what these findings mean for an individual evening student. They do not show that eating after a certain hour causes weight gain on its own, and they do not show that a single late dinner does harm. What they suggest is that, all else being equal, a pattern of shifting most of the day's eating into the late evening is not metabolically neutral, and that moving the main meal earlier, where possible, is a reasonable default. Reflux, sleep and the stomach The more immediate cost of late eating for many people is felt the same night. Lying down with a full stomach makes it easier for stomach contents to flow back into the esophagus, producing the burning sensation of heartburn and, for some, coughing, a sour taste or disrupted sleep. The National Institute of Diabetes and Digestive and Kidney Diseases lists eating meals two to three hours before lying down among the lifestyle changes doctors may recommend for people with gastroesophageal reflux disease.[3] The evidence behind that advice has some direct support. In a case-control study published in 2005, Yasuhiro Fujiwara and colleagues in Osaka compared 147 patients with reflux disease with 294 matched controls and asked how long each person typically waited between finishing dinner and going to bed.[4] After adjusting for smoking, drinking and body mass index, people whose dinner-to-bed time was less than three hours had odds of reflux disease more than seven times higher than those who waited four hours or more. Like any case-control study, it shows an association rather than proof of cause, but it matches both clinical experience and the physiology of digestion. For the evening student, the practical implication is that the post-class meal is badly placed. A student who gets home at ten and eats a large dinner at 10:15 before trying to sleep at 11:15 is doing almost exactly what the reflux guidance warns against. Even people without reflux disease often find that a heavy late meal makes it harder to settle, leaves them feeling uncomfortably full in bed and produces lighter, more broken sleep. There is a counterweight to all of this. Going to bed very hungry can also disrupt sleep, and a student who eats almost nothing between noon and midnight is likely to find both the second half of class and the night that follows harder than they need to be. The goal is not to stop eating at a particular hour at all costs. It is to move the bulk of the evening's food earlier and to keep anything eaten after class small and easy to digest. Three ways to place the meal There is no single right answer to where dinner belongs, because class schedules vary. But most evening schedules can be handled with one of three basic patterns, summarized in Table 1 and discussed below. Before class. For students who have a gap of an hour or more between the end of work and the start of class, or who can eat at work late in the afternoon, the simplest pattern is to eat a real, moderate dinner between about four and six. This fits well with the biology in Chapter 1: it arrives after the worst of the afternoon dip, it is digested well before sleep and it carries the student through class. The main risk is eating too large a meal just before sitting down, which can deepen drowsiness in the first hour of class. A moderate plate, with protein, vegetables and some whole-grain carbohydrate, tends to work better than a very large, starchy one, in line with Monk's observation that a high-carbohydrate meal can accentuate the post-lunch dip.[5] Split dinner. For students who come straight from work to class with no time to eat, or whose classes run late, a split pattern often works best: a substantial snack or small meal late in the afternoon, perhaps in the car or at a desk before leaving work, and a second small meal at the class break. The two halves together make a dinner. Neither is large enough to cause sleepiness, and by the time class ends there is no strong hunger driving a late meal at home. Small late meal. For some students, particularly those whose classes end early in the evening, or whose schedules make eating before class impossible, the only option is to eat after class. In that case, the aim is to make that meal modest, to eat it as soon as possible after class rather than just before bed and to favor foods that are easy to digest. A bowl of soup and bread, yogurt with fruit, eggs and toast or leftovers in a small portion are all better choices than a drive-through meal eaten at 10:45. Table 1. Where the evening meal can go, by schedule type. Schedule Main food At the break After class Gap of an hour or more before class Moderate dinner, 4:30 to 6:00 Water; fruit if hungry Nothing, or a small snack Straight from work, class ends by 9:30 Substantial snack before leaving work Second half of dinner Small snack only if needed Class ends after 9:30, long commute Split: late-afternoon meal and break meal Protein-based portion Small, light snack at least an hour before bed No chance to eat before class Small meal right after class Something to prevent extreme hunger Keep portion modest; eat soon after arriving What to eat, and how to make it happen The quality of an evening student's food depends less on nutritional knowledge than on logistics. Most adults know that vegetables, whole grains, beans, fish and lean proteins are good for them and that meals built mostly from fried food, refined starches and sugary drinks are not. The problem is that a person leaving work at 4:45 to reach a 6:00 class does not have the chance to buy, let alone prepare, a balanced meal, and the food that is fast and available on the way to campus is overwhelmingly the second kind. The solution is almost always to make the decision earlier in the day or earlier in the week, when you have more time and less hunger. A few approaches work well for many students. Cook once, eat twice. On a non-class night or on the weekend, cook a double portion of something that keeps and travels well: a grain bowl, a stew, a pasta dish with vegetables, a batch of rice and beans. Pack half into containers for class nights. This single habit does more for most evening students' diets than any amount of good intention at 5 p.m. Build a portable dinner. A dinner does not need to be hot or elaborate. A wrap with chicken or beans and vegetables, a container of pasta salad, a hard-boiled egg or two with a piece of fruit and some whole-grain crackers, or hummus with vegetables and pita can all serve as a real meal. What matters is that it includes some protein, some fiber from vegetables, fruit, beans or whole grains, and enough substance to hold you for several hours. Stock a class bag. Keep a small supply of shelf-stable foods in your bag or car for nights when the plan fails: nuts, a nut butter packet, whole-grain crackers, dried fruit, a shelf-stable milk or soy milk carton, a protein bar you actually like. These are not ideal dinners, but they keep you out of the vending machine and prevent the extreme hunger that leads to the late drive-through meal. Drink water. Mild dehydration can contribute to fatigue and headaches, and evening students who drink coffee through the afternoon and then sit in dry, heated or air-conditioned classrooms often drink little else. A refillable water bottle is one of the simplest tools in this book. Keep in mind that drinking a great deal in the last hour before bed may wake you during the night, so front-load fluids earlier in the evening. Plan the break. Many evening classes include a ten- or fifteen-minute break halfway through. It is short, and it is often spent in line at a vending machine or coffee cart. If you know what you will eat at the break, and have it with you, you can use the whole ten minutes to eat, stretch and get some air. Know the room rules. Some classrooms and labs do not allow food, and in laboratory courses eating is often banned for good safety reasons. Check at the first session, and if food is not permitted, plan to eat in a hallway or lounge at the break, or shift more of the meal to before class. Many instructors of evening classes are sympathetic to students eating quietly in lectures, since they know their students have come straight from work, but it is courteous to ask, and to choose foods that are not noisy or strongly scented. When the budget is tight Evening students are more likely than traditional students to be supporting others on limited incomes, and food insecurity is a serious and common problem in higher education. The Lumina Foundation reports that 22 percent of college students reported low or very low food security in the preceding thirty days.[6] Advice about packing balanced meals means little to a student who cannot reliably afford groceries. If this describes your situation, it is worth knowing that many colleges now run campus food pantries, sometimes with evening hours or pickup options designed for working students, and that student services offices can often help with applications for public benefits such as the Supplemental Nutrition Assistance Program, where eligibility rules for students are specific and sometimes confusing. Asking is not a sign of failure. Colleges run these programs because hunger is a direct obstacle to finishing a degree, and students who are fed learn better. On a limited budget, the most economical approach to class-night meals is usually the same as the healthiest: cooking in batches at home from inexpensive staples such as rice, dried or canned beans, lentils, eggs, oats, frozen vegetables, bananas and seasonal produce, and packing it. The most expensive approach, and usually the least nutritious, is buying food on the way to class several nights a week. Special situations Some students need to be more careful with meal timing than others. People with diabetes, particularly those who use insulin or certain oral medications that can cause low blood sugar, should talk with their clinician or diabetes educator before changing the timing or size of their evening meal, since an evening class can easily lead to a long gap without food or a mismatch between medication and meals. People with reflux disease, as discussed above, have particular reason to finish eating well before lying down. Pregnant students, students with a history of disordered eating and anyone following a medically prescribed diet should treat the patterns in this chapter as starting points for a conversation with a professional rather than as instructions. It is also worth noting what this chapter does not recommend. It does not recommend skipping meals as a strategy to stay alert, which tends to backfire through distraction, irritability and overeating later. It does not recommend any particular diet or rigid time window. The evidence on late eating supports a modest, practical shift, moving the main evening meal earlier and keeping late food light, not a set of rules to be followed at any cost. Dinner as part of the plan The previous chapter proposed working backward from the time you want to be asleep. Food fits into that plan naturally. If the anchor is 11:30, the main meal should ideally be finished by about 8:30, and preferably much earlier. On most evening schedules, that means before class or at the break, not after. Once that is decided, the rest follows: what to pack, when to eat it and what to keep in the bag for nights when the plan fails. The student who makes this one change, from a large late meal to an earlier main meal and a small late snack if needed, often notices a difference within a week or two. The second half of class is less distracted, the drive home is not accompanied by the smell of fries, and falling asleep becomes easier. The next chapter turns to the other lever that most evening students pull without thinking: caffeine. Chapter 4: Caffeine by the Clock Caffeine is the evening student's oldest ally. It is cheap, legal, available in every building on campus and effective. It genuinely does improve alertness, reaction time and the ability to sustain attention on dull tasks, and for someone facing three hours of lecture after a full working day, those are not small benefits. Used well, it can carry a tired student through the afternoon dip and into the evening. Used badly, it quietly steals the one thing that would have made the next day easier: sleep. The difference between using it well and using it badly is almost entirely a matter of timing. This chapter explains how caffeine works, how long it lasts, how much is too much and how to calculate a personal cut-off time that fits an evening schedule. What caffeine does, and what it does not do Chapter 1 described adenosine, one of the chemical signals that builds up in the brain the longer you are awake and that contributes to the feeling of sleep pressure. Caffeine's main effect is to block the receptors that adenosine acts on.[1] With those receptors occupied, the brain registers less of the sleep pressure that has accumulated, and you feel more alert. This mechanism explains both caffeine's power and its limits. It does not create energy, and it does not repay sleep debt. The adenosine is still there, waiting. When the caffeine wears off, the underlying sleep pressure returns, often abruptly, which is one reason why people who rely on repeated doses through the day can feel suddenly exhausted in the late evening. It also explains why caffeine works better on moderate tiredness than on severe sleep deprivation: when sleep pressure is very high, blocking some of its signal is not enough to restore normal function. Caffeine is absorbed quickly from the gut. Most people begin to feel its effects within roughly half an hour, which is why fatigue-management guidance for drivers and shift workers commonly notes that caffeine takes about thirty minutes to start working.[2] Peak levels in the blood typically come within an hour or so of drinking it. After that, the body begins to clear it, and the pace of that clearing is the most important fact in this chapter. Half-life: the number that matters most Caffeine is broken down mainly by the liver, and the rate at which that happens is usually described in terms of half-life, the time it takes for the amount in the body to fall by half. In the average adult, the half-life of caffeine is approximately five hours.[1] That average hides a great deal of variation. According to the same clinical review, smoking can shorten the half-life by up to 50 percent, and pregnancy, especially in the final trimester, can lengthen it to as much as fifteen hours. Genetics, liver function and certain medications also influence it, so two people drinking the same cup of coffee at the same time can carry very different amounts of caffeine to bed. The implications of a five-hour half-life for an evening student are striking when you work them through. Table 2 shows how much of a 200-milligram dose, roughly what you might get from a medium brewed coffee, would remain in the body at various times after drinking it at 5:00 p.m., assuming a half-life of three, five or eight hours. The figures are simplified estimates that ignore the time it takes to absorb the dose and treat clearance as a steady exponential decline, but they give an honest sense of the scale involved. Table 2. Estimated caffeine remaining from a 200 mg dose taken at 5:00 p.m. Clock time Hours later Half-life 3 h Half-life 5 h Half-life 8 h 7:00 p.m. 2 126 mg 152 mg 168 mg 9:00 p.m. 4 79 mg 115 mg 141 mg 11:00 p.m. 6 50 mg 87 mg 119 mg 1:00 a.m. 8 31 mg 66 mg 100 mg 6:00 a.m. 13 10 mg 33 mg 65 mg Calculated from the standard half-life formula; the five-hour average half-life is from Evans, Richards and Battisti (StatPearls). Look at the middle column. A student with an average half-life who drinks a medium coffee at 5:00, just before a 6:00 class, still has nearly 90 milligrams of caffeine in the body at 11:00 p.m., which is roughly the amount in a small cup of coffee. In effect, they are going to bed shortly after drinking one. A student whose half-life happens to be longer is going to bed with more caffeine in their system than a typical can of cola contains, and will still carry a meaningful amount at breakfast. The evidence on caffeine and sleep How much does this actually matter? The most useful study for evening students was published in 2013 by Christopher Drake and colleagues at the Henry Ford Hospital sleep center in Detroit.[3] They gave healthy adults 400 milligrams of caffeine, about the amount in two to three large coffees, at one of three times: at bedtime, three hours before bedtime or six hours before bedtime, and compared the results with a placebo. Participants slept at home, and their sleep was measured both by their own reports and by a portable monitor. All three timings significantly disrupted sleep compared with placebo. Even when the caffeine was taken six hours before bed, objectively measured total sleep time was reduced by more than an hour. Just as important, participants tended not to notice the disruption as fully as the monitor recorded it. The authors concluded that their findings supported advice to avoid substantial caffeine for at least six hours before bedtime. Six hours is a sobering number for someone with an 11:30 anchor: it points to a cut-off around 5:30 p.m. for a large dose, which is before many evening classes even start. And the study used a fixed, fairly large dose, so it does not tell us precisely how smaller amounts would behave. But taken together with the half-life figures in Table 2, it strongly suggests that the classic evening student habit, a large coffee or energy drink before class or at the break, is costing real sleep, whether or not the student feels it. How much is too much? Timing is one question; total amount is another. The U.S. Food and Drug Administration has cited 400 milligrams a day as an amount not generally associated with dangerous, negative effects for most healthy adults, describing it as roughly two to three 12-ounce cups of coffee.[4] The FDA also notes that people vary widely in their sensitivity to caffeine and in how fast they metabolize it, and that pregnant or breastfeeding people, and those who take certain medications or have certain health conditions, may need to limit it more and should ask their health care provider. The same FDA guidance warns that toxic effects, such as seizures, can occur with rapid consumption of around 1,200 milligrams of caffeine, and it specifically cautions against pure and highly concentrated caffeine products, which can have serious consequences including death. A single teaspoon of powdered caffeine, the Mayo Clinic notes, can contain roughly as much caffeine as twenty-eight cups of coffee.[5] Such products have no place in a student's study routine. The 400-milligram figure is a ceiling, not a target, and it says nothing about timing. A student who drinks 300 milligrams of caffeine a day, well within the FDA figure, can still have badly disrupted sleep if much of it comes after four in the afternoon. Knowing how much caffeine you actually consume is harder than it sounds, because the content of common drinks varies widely. Table 3 shows the ranges the FDA reports for 12-ounce servings of several common drinks. Table 3. Caffeine in a 12-fluid-ounce serving of common drinks. Drink Typical caffeine range Brewed coffee 113 to 247 mg Energy drink 41 to 246 mg Caffeinated soft drink 23 to 83 mg Decaffeinated coffee, per 8 oz 2 to 15 mg Source: U.S. Food and Drug Administration, "Spilling the Beans: How Much Caffeine Is Too Much?" (2024). Two features of these figures deserve attention. First, the ranges are wide: a 12-ounce brewed coffee from one shop may contain twice as much caffeine as the same size from another. Large coffee-shop sizes are often 16 or 20 ounces, which can push a single drink toward or beyond 300 milligrams. Second, decaffeinated coffee is not caffeine-free. For most people the small amount is irrelevant, but for someone who is very sensitive, several cups of decaf in the evening are not nothing. Caffeine also turns up in places students may not think of: tea, both black and green; chocolate and cocoa; some pre-workout powders and "focus" supplements; caffeinated water and gum; and some over-the-counter pain relievers, which list caffeine among their active ingredients. Reading labels for a week can be an eye-opening exercise. Calculating your personal cut-off Putting the pieces together gives a practical method for setting a caffeine cut-off time on class days. Start with your anchor, the time you want to be asleep, from Chapter 2. Count back at least six hours, in line with the Drake study. That is your default cut-off for any substantial dose. For an 11:30 anchor, it falls at 5:30 p.m.; for an 11:00 anchor, at 5:00 p.m. Then adjust for what you know about yourself. If you have noticed that even an afternoon coffee keeps you awake, or if you are pregnant, taking medication that slows caffeine clearance or are simply a "slow metabolizer" by experience, move the cut-off earlier, perhaps to eight hours before your anchor. If you smoke, your caffeine may clear faster, although the far larger health gain would come from quitting smoking, which your campus health service can help with. Next, adjust for dose. The six-hour rule was based on a large dose. A smaller amount, such as a cup of black tea or a small coffee, taken close to the cut-off will leave less in your system at bedtime. A useful way to think about it is that the later in the day you drink caffeine, the smaller the amount should be. To see how this works in practice, return to the logistics coordinator from Chapter 2, whose anchor on class nights is 11:00. Six hours earlier is 5:00 p.m., so on paper a coffee at 4:30 before leaving work would just about qualify. But he knows from experience that a large afternoon coffee keeps him awake, and his usual drink is a 16-ounce brewed coffee that could easily contain well over 200 milligrams. So he moves his cut-off to about 1:30, well over eight hours before his anchor, and switches his late-afternoon drink to a small black tea, which contains far less caffeine, or to decaf when he wants only the ritual. His total daily caffeine barely changes. Its timing changes a great deal. Finally, test it. For two weeks, follow your cut-off strictly and pay attention to how quickly you fall asleep, how often you wake and how you feel in the morning. Then, if you want, experiment carefully with small changes. Your own experience, gathered deliberately rather than assumed, is the best guide you have, with the caveat from the Drake study that people tend to underestimate how much caffeine is disturbing their sleep. Using caffeine strategically For most evening students with a typical schedule, the backward plan leads to a pattern something like this: caffeine in the morning as usual; a moderate dose in the early to middle afternoon, timed to cover the post-lunch dip and the transition to campus; and then nothing, or only a small amount of something mild, from the late afternoon on. The early-evening lift described in Chapter 1 does much of the work of keeping you alert through the first half of class, and the tools in Chapter 5, movement, light and active engagement, can help with the second half. This is harder than it sounds, because the moment of greatest temptation is the class break, when the energy of the evening lift has begun to fade and the coffee cart is right there. A student who knows why the break coffee is expensive, and who has a plan for what to do instead, is far more likely to resist it than one who simply tries to say no. There is one important exception, which Chapter 6 addresses in detail: when you are too drowsy to drive safely. Safety guidance for drowsy drivers recommends pulling over to rest, and a combination of caffeine and a short nap can provide temporary relief when a driver has to continue.[2] A student who is dangerously sleepy at the wheel should not let concern about sleep quality stop them from using caffeine in that situation, although the real solution is a schedule that avoids reaching that state. Energy drinks Energy drinks deserve a separate word, because they are especially popular with students and especially easy to misuse. Some contain no more caffeine than a cup of coffee; others contain much more, and it is easy to drink them quickly and cold, taking in a large dose in a few minutes. Many also contain large amounts of sugar and other stimulant ingredients whose effects in combination are less well studied. Two cautions matter most. First, the FDA's concern about rapid consumption of large doses applies with particular force to energy drinks and energy shots. Drinking several in a short period, especially on an empty stomach or after little sleep, can produce a racing heart, tremors, anxiety and nausea. Anyone who experiences chest pain, a very rapid or irregular heartbeat, fainting, confusion or seizures after consuming caffeine should seek emergency medical care. Second, mixing energy drinks with alcohol is a poor idea, because the stimulant can mask the feeling of intoxication without reducing actual impairment, which can lead people to drink more and to believe they are fit to drive when they are not. Tolerance, dependence and cutting back People who consume caffeine regularly develop tolerance to some of its effects, meaning that the same dose produces less alertness over time. This is one reason why caffeine use tends to creep upward during a demanding semester. Regular users can also experience withdrawal symptoms, such as headache, fatigue, difficulty concentrating and irritability, when they stop suddenly, typically beginning within a day or so and fading over several days.[6] If you decide that you are drinking more caffeine than you want, or that your afternoon and evening caffeine needs to shift earlier, it is usually better to make the change gradually, reducing by a modest amount every few days, than to stop all at once in the middle of a busy week. A good time to start is a break between terms, or a week with fewer demands. Replacing a later cup with decaffeinated coffee or herbal tea can preserve the ritual, which for many people is part of the appeal, without the stimulant. Caffeine in the backward plan Caffeine fits into the backward plan as one of its fixed points: a cut-off time, written down and ideally set as a phone reminder, that protects your sleep. Around that point sit the other choices: how much to drink earlier, what to drink instead later and how to handle the break. Of all the changes described in this book, moving caffeine earlier is among the easiest to make and the quickest to show results. Many students find that within a week or two they fall asleep faster after class and wake less tired, without any other change to their routine. The next chapter looks at what to do instead of caffeine when the lecture is long and your attention is flagging. Chapter 5: Staying Sharp in the Seat There is a particular kind of silence that settles over an evening classroom at about 8:15. The instructor is still talking. The slides are still changing. But half the room has the fixed, slightly glazed look of people who are physically present and mentally elsewhere. A few are fighting it openly, shifting in their seats, rubbing their eyes, taking sips from empty cups. One or two have lost the fight, their heads dropping slowly forward and jerking back up. Nobody in that room is lazy. Most of them have been awake for fourteen hours or more, many have worked a full shift and all of them are now sitting still, in a warm room, listening to a single voice, at the point in the evening when the circadian lift described in Chapter 1 is beginning to fade. The conditions could hardly be better designed to produce drowsiness. The previous chapter argued that reaching for more caffeine at this point is usually a poor trade. This chapter is about what to do instead. None of the strategies here is as powerful as a good night's sleep, but together they can make a real difference to how much of a long evening class you actually absorb. Arrive less depleted The best time to prepare for the second half of class is before the first half begins. Much of what determines how you feel at 8:15 has already been decided by 5:30: how much you slept the night before, whether you ate a sensible meal, whether you spent the afternoon dip in a state of mounting exhaustion or managed to take the edge off it. For students whose schedule allows it, one of the most effective tools is a short nap in the late afternoon. Research on napping in healthy adults, reviewed by Catherine Milner and Kimberly Cote, finds that short naps can improve alertness and performance, and that very brief naps, on the order of ten to twenty minutes, have the advantage of producing benefits quickly while minimizing sleep inertia, the grogginess that follows waking from deeper sleep.[1] In one well-known study, Amber Brooks and Leon Lack compared naps of different lengths taken in the afternoon after a night of restricted sleep and found that a ten-minute nap produced prompt and sustained improvements in alertness and performance, while longer naps were followed by a period of grogginess before any benefits appeared.[2] A short nap in a parked car, in a quiet corner of the library or on a break room couch between work and class can therefore be worth more than a coffee, and it has no effect on the caffeine level you take to bed. The key is to keep it short. Set an alarm for twenty minutes, allowing a few minutes to fall asleep, and get up when it goes off. A long nap late in the afternoon can leave you groggy for the start of class and, by reducing sleep pressure, can make it harder to fall asleep at your anchor time. Some people combine the two: they drink a coffee immediately before a short nap, so that the caffeine begins to take effect at about the time they wake. This is the same approach recommended for drowsy drivers in Chapter 6. For an evening student, it only makes sense if the timing still respects the caffeine cut-off from Chapter 4, which in practice means it works best for classes that start in the late afternoon. If you park in a campus lot or garage to nap, choose a well-lit, busy area, lock the doors and let someone know where you are. Chapter 7 discusses campus safety in more detail. Use your body The single most reliable way to fight drowsiness in class, short of sleeping, is to stop sitting still. Posture and movement both matter. Sleepiness is harder to maintain when you are upright and engaged than when you are slumped and passive, and even small amounts of movement raise alertness temporarily. At the class break, get up and walk. A few minutes of brisk walking, ideally outside or at least out of the classroom, is more effective than standing in line for a vending machine. Climbing a flight or two of stairs is even better. Swing your arms, stretch, breathe deeply. The goal is not exercise in any meaningful fitness sense but a brief change in physiological state. During class, sit upright with both feet on the floor. If the room and instructor allow it, stand at the back for part of the session; many instructors are happy for adult students to do this if asked beforehand, and some classrooms have high tables for exactly this purpose. Small movements, such as flexing your feet, shifting position or rolling your shoulders, are unobtrusive and can help. If you are fighting sleep hard, it is better to step out quietly for a two-minute walk to the water fountain than to spend twenty minutes nodding. Regular physical activity outside class also helps, both by improving the quality of sleep and by building general fitness. The federal Physical Activity Guidelines for Americans recommend that adults get at least 150 to 300 minutes of moderate-intensity aerobic activity a week, along with muscle-strengthening activity on two or more days.[3] For an evening student, that can sound like one more demand on an overfull schedule, but it can often be built into existing routines: walking or cycling part of the commute, a brisk walk at lunch, a few sessions a week on non-class days. Vigorous exercise very close to bedtime may make it harder for some people to fall asleep, so on class nights it is usually better to be active earlier in the day. Use light Chapter 1 described how light influences the circadian clock. Light also has a more immediate effect: bright light, particularly light rich in the shorter blue wavelengths, has an alerting effect on the brain, independent of its effects on the clock.[4] This is part of the reason why evening screen time can make it harder to fall asleep, a problem Chapter 8 discusses. But in the context of an evening class, the same effect can be used deliberately. If you have a choice of where to sit, choose a well-lit part of the room rather than a dim corner. If the instructor dims the lights for slides, sit closer to the screen or the windows. At the break, step into a brightly lit hallway or lobby rather than a dark courtyard. These are small effects, but they add up. There is a tension here, of course. The same light that helps you stay alert at 7:30 may, if it continues all evening, push your internal clock later and make it harder to fall asleep. The answer is to use light when you need alertness, in class and on the drive home, and then to dim it deliberately once you are home and moving toward sleep. Engage actively Passive listening is far more conducive to drowsiness than active engagement. Students who take notes by hand, ask questions, work through problems as the instructor presents them and participate in discussions tend to report less difficulty staying awake than those who simply listen. There is also good reason to think that this kind of active processing improves learning itself. A few techniques work well for evening students. Sit near the front, where it is harder to drift and easier to engage. Take notes in your own words rather than trying to transcribe the slides; the effort of summarizing keeps your mind working. Prepare a question or two from the reading before class, so you have a reason to listen for particular points. When the class includes group work or discussion, participate fully rather than letting others carry it; talking is one of the most effective antidotes to drowsiness there is. It also helps to know the structure of the evening. If you can see from the syllabus or slides which topics will be covered when, you can plan to be at your most attentive for the material that matters most and use the less demanding stretches as recovery. Mind the environment Warm, stuffy rooms promote drowsiness. If you can, dress in layers so you can shed a sweater when the room gets warm, and choose a seat near a window, door or air vent. A cool drink of water can help, and staying hydrated through the evening, as Chapter 3 suggested, prevents the mild dehydration that can add to fatigue and headaches. Some students chew gum or eat something crunchy to stay alert. Evidence on these strategies is limited, but if they work for you and your instructor does not object, they are harmless. Be careful with eating as a way of staying awake, though: grazing on sugary snacks through the second half of class tends to add calories without solving the underlying problem. Put the hard thinking in the right hours Evening students often assume that the time to study is after class, while the material is fresh. It is an appealing idea and, for most people on most nights, a poor one. By the time a three-hour class ends and the trip home is done, the circadian support that carried you through the early evening has usually withdrawn, sleep pressure is at its peak for the day, and the anchor time from Chapter 2 is close. Reading a chapter at 10:30 p.m. under those conditions tends to produce the familiar experience of turning pages without retaining them, and every minute spent doing it comes directly out of sleep, which is itself when much of the day's learning is consolidated. A better approach is to protect a short period for review within a day of class, but not on the night itself. Twenty or thirty minutes the next morning, at lunch or on the commute if you ride transit, spent going back over your notes and filling in gaps, is usually worth more than an hour of bleary effort the night before. Heavier work, such as problem sets, papers and reading for the next session, generally belongs on non-class days, weekends or the earlier part of the day, when your attention is freshest. For students with fixed work hours, even a regular early-morning block of forty-five minutes before the household wakes can be the most productive study time of the week, provided it does not come at the cost of short sleep. The same logic applies to preparation. Skimming the reading or the slides for the next class, even briefly, gives you a scaffold for the lecture, so that you spend the evening connecting new material to something you have already seen rather than meeting it cold. Students who arrive prepared consistently find it easier to stay engaged, which in turn makes it easier to stay awake. Preparation is, among other things, a fatigue strategy. An evening class, hour by hour It can help to see these tools arranged across a single session. Consider an illustrative three-hour class running from 6:00 to 9:00, taken by a student who has worked a day shift. In the late afternoon, between leaving work and class, the student eats the first half of a split dinner, as described in Chapter 3, and, if there is time, takes a ten- to twenty-minute nap in a safe place. Any caffeine for the day was finished in the early afternoon. Arriving a few minutes early, the student takes a seat in a well-lit part of the room near the front and looks over the slides or reading. From 6:00 to about 7:15, the early-evening lift is at work. This is the time for the most demanding material, and the student takes notes in their own words and asks at least one question. At the break, the student walks for five minutes, ideally up and down a flight of stairs or outside, eats the second half of dinner and refills the water bottle, skipping the coffee cart. From about 7:30 to 9:00, attention is harder to hold. The student sits upright, stands at the back for part of the time if the instructor allows it, and uses small movements to stay alert. If the fight against sleep becomes intense, a brief walk to the water fountain is better than twenty minutes of nodding. At the end, rather than lingering to start homework, the student packs up, notes any questions to review the next day and turns attention to the trip home. Nothing in this sequence is dramatic, and none of it requires unusual discipline. It simply lines up the student's choices with the biology of the evening instead of working against it. Talk to your instructors Many instructors who teach evening classes are acutely aware of the challenges their students face and structure their sessions accordingly: with breaks, changes of activity and a variety of teaching methods. Others, particularly those who also teach day classes, may simply transplant a daytime lecture into the evening without adjustment. If you find a particular class very hard to stay alert in, it is reasonable to raise it, politely and constructively. An instructor may be willing to add a short break, to allow students to stand at the back, to break up a long lecture with a brief activity or to post slides or recordings so that a moment of inattention does not mean losing material. Evening students are often more comfortable than younger students in having this kind of adult conversation, and instructors often appreciate the feedback. If you have a disability or health condition that affects your alertness or energy, your college's disability services office can help arrange formal accommodations. When sleepiness is a warning Occasional drowsiness in a long evening class is normal. Persistent, overwhelming sleepiness, the kind that makes it hard to stay awake in class even when you have slept reasonably well, is not, and it deserves attention. Several common sleep disorders cause excessive daytime sleepiness. Obstructive sleep apnea, in which breathing repeatedly stops or becomes shallow during sleep, is common among adults, particularly those who snore loudly, are overweight or have high blood pressure, and it often goes undiagnosed. Insomnia, in which people have persistent difficulty falling or staying asleep, is also common, and it frequently responds well to a structured treatment called cognitive behavioral therapy for insomnia. Depression, some medications and a range of medical conditions can also cause fatigue. If you find yourself falling asleep in class despite reasonable sleep, if others have told you that you snore heavily or stop breathing in your sleep, if you regularly struggle to stay awake while driving or if your fatigue is getting worse over time, talk to a clinician. Many colleges have health services with evening hours; your own primary care provider is another option. Sleep disorders are treatable, and treating them can transform both academic performance and safety. Accept some limits Finally, it is worth being honest about what these strategies can and cannot do. They can help a reasonably rested student stay alert and engaged through a long evening class. They cannot make a severely sleep-deprived student function well, and no amount of standing, walking or note-taking will fully make up for a week of five-hour nights. If you find that you are routinely fighting sleep in class despite doing everything in this chapter, the problem is almost certainly upstream: in the amount of sleep you are getting, the timing of your caffeine or the overall load you are carrying. The backward plan in Chapter 2 is the place to start looking for the answer. The same honesty matters even more once class ends. A student who has fought drowsiness through the last hour of a lecture is not going to be magically alert for the drive home, and that drive is the subject of the next chapter. Chapter 6: The Drive Home Of all the risks an evening student faces, the drive home after a late class is the most serious, and it is the one most often dismissed. Students who would never consider driving after three drinks will routinely drive forty minutes on a dark highway after sixteen hours awake, a full shift and a three-hour lecture, with the radio turned up and the window cracked open, trusting that they will be fine because they have always been fine before. Most of the time they are. But drowsy driving is not a problem of averages. It is a problem of the one night when the microsleep lasts four seconds instead of one, on the one stretch of road where a four-second lapse is enough. This chapter explains how common drowsy driving crashes are, why they are especially dangerous, how to recognize when you are too tired to drive and what to do about it. It is the most important chapter in this book, and the one where the advice is least negotiable. How big is the problem? The National Highway Traffic Safety Administration estimates that in 2017, 91,000 police-reported crashes involved drowsy drivers, leading to an estimated 50,000 people injured and nearly 800 deaths.[1] NHTSA's own drowsy driving page also reports 644 deaths in drowsy-driving-related crashes in 2024. The agency acknowledges that these figures are widely believed to understate the problem, because, unlike alcohol, drowsiness leaves no measurable trace after a crash. It is identified only if the driver admits to it or an investigator infers it, and a driver who has died cannot be interviewed. Research using other methods suggests that the true toll is much higher. A 2014 analysis by Brian Tefft for the AAA Foundation for Traffic Safety, using a national sample of crashes investigated in detail, estimated that a drowsy driver was involved in about 6 percent of crashes in which a vehicle was towed, 13 percent of crashes in which someone was hospitalized and 21 percent of crashes in which someone was killed.[2] A later AAA Foundation study, led by Justin Owens, examined in-vehicle video recorded in the moments before more than 700 real-world crashes, measuring the proportion of time drivers' eyes were closed. It found signs of drowsiness in between 8.8 and 9.5 percent of all crashes and in between 10.6 and 10.8 percent of the more serious crashes that would be reported to police, far above the share identified in official statistics.[3] The exact number is uncertain, but the pattern is not. Drowsy driving is a leading contributor to serious and fatal crashes, and it is badly undercounted. Why drowsy crashes are so dangerous Drowsy driving crashes have a distinctive profile. According to NHTSA, they occur most often between midnight and 6 a.m. or in the late afternoon, frequently involve a single driver with no passengers, often happen on rural roads and highways and typically show no evidence of braking before the crash.[1] Every element of that profile is relevant to evening students. The late afternoon is the time of the post-lunch dip described in Chapter 1, and it is when many students are driving from work to campus. Late evening is not quite the midnight to 6 a.m. peak, but for a student who has been awake since 5:30 in the morning, 10 p.m. represents more than sixteen hours of wakefulness, and a longer commute or a later class pushes the drive into the highest-risk window. Students almost always drive home alone. And many community colleges and commuter campuses are located on the edges of towns, with a significant part of the trip home on highways or rural roads. The absence of braking is the most frightening part of the profile. It reflects the fact that a drowsy driver who crashes has often, for a few seconds, simply stopped driving. In a microsleep, the brain briefly disengages from the outside world. The driver does not see the curve, the stopped car or the edge of the road and so does not react to it. Unlike a distracted driver who glances up and swerves, a sleeping driver may hit an obstacle at full speed. That is one reason why drowsy crashes are disproportionately severe. It is often said that driving drowsy is like driving drunk, and there is truth in the comparison. The National Institute for Occupational Safety and Health, drawing on the research literature, notes that being awake for 17 hours produces impairment similar to that of a blood alcohol concentration of 0.05 percent, and that being awake for 24 hours is similar to a blood alcohol concentration of 0.10 percent, above the legal limit for driving in every U.S. state.[4] This line of work traces back to a well-known 1997 study in Nature by Drew Dawson and Kathryn Reid, which compared the performance effects of sustained wakefulness with those of alcohol and concluded that moderate levels of fatigue produced impairment greater than the legally prescribed level of alcohol intoxication in many countries.[5] Consider what this means in practice. A student who wakes at 5:30 a.m., works a full day and drives home from a class that ends at 9:45 p.m. will have been awake for more than sixteen hours by the time they reach the car and seventeen by the time they get home. If they slept poorly the night before, or have been getting six hours a night all week, the effective impairment may be considerably greater. The same student would never consider driving after a few drinks. The biological reality is not so different. NIOSH adds a crucial caution: combining even a small amount of alcohol with extended wakefulness makes performance worse still.[4] A single beer with dinner before class, or a glass of wine at a work event beforehand, is a far larger risk for a student who will drive home late than it would be for someone driving home at six after a normal day. Knowing when you are too tired The central difficulty with drowsy driving is that people are poor judges of their own impairment. The Van Dongen study described in Chapter 2 found that people restricted to six hours of sleep a night for two weeks became steadily more impaired while reporting only modest increases in sleepiness. Many drivers who have had a drowsy crash or near miss report that they did not feel dangerously tired until the moment it happened. That does not mean that the warning signs are invisible, only that you have to look for them deliberately rather than waiting for an overwhelming feeling of tiredness. Research from the Loughborough University sleep laboratory found that drivers in a simulator generally did acknowledge feeling sleepy before their driving deteriorated, which suggests that noticing and acting on sleepiness is possible.[6] The problem is less that drivers do not feel sleepy than that they talk themselves out of stopping. Table 4 lists common warning signs, drawn from NHTSA and NIOSH guidance, and what each should prompt you to do.[1][4] The rule is simple: if you notice any of them, treat it as a signal to stop driving as soon as you safely can, not as a signal to try harder. Table 4. Drowsy driving warning signs and what to do. Warning sign What it means What to do Frequent yawning, heavy eyelids, rubbing your eyes Sleep pressure is high and rising Plan a stop; do not start a long drive this way Drifting from your lane or hitting a rumble strip Attention is lapsing Pull over at the next safe place Missing an exit or a sign you normally notice Brief lapses are occurring Pull over at the next safe place No memory of the last few miles Possible microsleeps Stop driving now, as soon as it is safe Head nodding or jerking upright You are falling asleep Stop driving now, as soon as it is safe Before you even start the car, ask yourself a few honest questions. How long have I been awake? How much did I sleep last night, and how much this week? Did I struggle to stay awake in class? Have I had any alcohol or medication that could make me drowsy? If the answers worry you, the decision about how to get home should be made in the parking lot, not on the highway. What works and what does not When drivers feel drowsy, they tend to reach for a familiar set of remedies: turning up the radio, opening the window, turning the air conditioning to cold, singing, slapping their own cheeks, chewing gum. These strategies share a feature: they may make you feel more alert for a few minutes without substantially reducing your actual risk. The same Loughborough researchers tested two of these strategies directly. In a simulator study published in 1998, Louise Reyner and Jim Horne had sleep-restricted young drivers drive for two and a half hours in monotonous conditions, sometimes with cold air blowing on their faces and sometimes with the radio playing. Neither had a significant effect on lane-drifting incidents overall. The radio reduced drivers' reported sleepiness for much of the drive, but the effects of cold air were only transient, and the authors concluded that these measures were at best temporary expedients, perhaps enough to help a driver reach a safe place to stop.[7] This is a sobering finding, because it means that the radio can make you feel better without making you safer. What does work, in the short term, is stopping and addressing the sleepiness itself. In an earlier study, Horne and Reyner compared three things sleepy drivers might do during a thirty-minute rest break between two monotonous simulated drives: take a nap of less than fifteen minutes, drink coffee containing 150 milligrams of caffeine or drink decaffeinated coffee as a placebo. Both the nap and the caffeine significantly reduced driving impairment and sleepiness. Simply taking a break, without either, proved ineffective.[6] NHTSA's guidance reflects this evidence. It advises that the only true protection against drowsy driving is getting adequate sleep, and it suggests that drivers who feel drowsy pull over at a safe place and consider a combination of caffeine and a short nap of about twenty minutes, noting that coffee alone is not enough for a driver who is seriously sleep-deprived.[1] NIOSH's guidance for shift workers similarly recommends pulling over and napping for fifteen to twenty minutes, and notes that because caffeine takes about thirty minutes to begin working, drinking it just before a short nap can mean that it takes effect around the time you wake.[4] Two caveats matter. First, these countermeasures are temporary. They can buy you some alertness for a limited time, enough, perhaps, to finish a modest drive home safely, but they do not remove the underlying sleep debt, and a driver who uses them and then drives for another two hours on a dark highway is running out of margin. Second, the stop itself has to be safe. Pulling onto the shoulder of a highway to nap is dangerous. Choose a well-lit rest area, a busy gas station or a campus lot you know, lock the doors and set an alarm. Alternatives to driving tired The best strategy for drowsy driving is not a trick for getting through the drive but a plan that avoids the drive when you are too tired to make it safely. For evening students, this requires some advance thought, because the options that exist at ten at night are fewer than those at five in the afternoon. Carpool with a classmate. Sharing a drive with another student from the same class means that one of you can drive while the other stays alert and talks, and that you can swap if one of you becomes too tired. It also halves the number of nights each person drives, if you alternate. Many evening classes have students who live in the same area; asking at the first session is easy. Arrange a backup ride. Before the term starts, identify someone who could pick you up on a night when you are too tired to drive safely: a partner, a family member, a friend. Agree on it explicitly, so that when the night comes, the call is a plan rather than a favor. Keep enough money or credit available for a taxi or rideshare in case your backup is unavailable. Nap before you drive. If you feel too drowsy to drive at the end of class, a twenty-minute nap in a locked car in a well-lit, busy part of a campus lot, possibly combined with caffeine, may make the drive home safer. It is not a solution for a long drive after severe sleep loss, and it has costs for your sleep that night, but it is far better than driving drowsy. Stay over. For students with long commutes, particularly those who drive an hour or more on rural roads, it may occasionally make sense to stay nearby, with a friend or relative close to campus, on nights when fatigue is severe. Change the schedule. If you find that you regularly finish class too tired to drive safely, that is not a problem to solve with coffee. It is a sign that your overall schedule, the combination of work hours, class times, commute length and sleep, is not sustainable, and the backward plan in Chapter 2 should be revisited. Options may include a later start at work on the morning after class, taking fewer evening classes in a term, choosing online or hybrid sections for some courses or moving closer to campus. None of these is easy, but they are far easier than the alternative. Before you drive, not only during Most of the work of avoiding drowsy driving happens long before the drive itself. The student who consistently sleeps seven or more hours, keeps caffeine to the afternoon, eats a sensible dinner before class and moves during the break arrives at the car in a far better state than one who has done none of those things. In that sense, the whole of this book is about the drive home. A few specific preparations also help. Check your medications, including over-the-counter antihistamines, sleep aids and some cold remedies, for warnings about drowsiness and operating vehicles; if something you take regularly carries such a warning, talk to your pharmacist or doctor about timing or alternatives. Avoid alcohol entirely before a late class if you will be driving afterward. Be especially careful in the week after the change to or from daylight saving time, and in the first weeks of a term, when schedules are new and sleep is often disrupted. A note for the people who love evening students If you share a home with an evening student, you can help in ways that matter. Offer to pick them up on nights when they seem exhausted, without making them feel judged for needing it. Keep your phone on when they are driving home. If they call and say they are pulling over to rest, tell them that is the right decision. And if you notice that they are consistently arriving home drained, help them think through whether their schedule can be changed. Drowsy driving is the one part of the evening student's day where the consequences of a bad night can be permanent. Everything else in this book is about feeling better and learning more. This chapter is about getting home. Chapter 7: Walking, Waiting and Riding After Dark Not every evening student drives home. Many take the bus or the train, some walk or cycle, and nearly all of them, even those with cars, spend part of the evening on foot: crossing a parking lot, walking from a classroom building to a garage, waiting at a stop, walking the last few blocks from a station to their front door. These moments rarely get the attention that the drive home deserves, but they carry their own risks, and those risks are shaped by the same two facts that shape everything else in this book. It is dark, and you are tired. This chapter covers the practical side of getting home safely when you are not behind the wheel: traffic safety for pedestrians and cyclists, personal safety on campus and in transit and the resources most colleges provide but many evening students never use. The traffic risk is the larger one When people think about safety on the way home at night, they usually think first about crime. It is a legitimate concern, and later sections of this chapter address it. But for most people walking at night in the United States, the greater statistical danger is traffic. NHTSA's figures for 2022 show that 7,522 pedestrians were killed in traffic crashes in the United States that year.[1] The details, summarized in Table 5, describe a problem concentrated in exactly the conditions evening students encounter. Table 5. Pedestrian deaths in U.S. traffic crashes, 2022. Factor Share of pedestrian deaths Occurred in the dark 78% Occurred between 6:00 and 8:59 p.m. 26% Occurred in urban areas 85% Occurred away from intersections 75% Source: NHTSA, Traffic Safety Facts 2022 Data: Pedestrians (DOT HS 813 590, 2024). More than three-quarters of pedestrian deaths happen in the dark, and the three hours from 6:00 to 9:00 in the evening, when many evening classes are in session or just ending, account for about a quarter of the total, more than any other three-hour period. Most deaths happen in urban areas, and most happen away from intersections, where drivers are least likely to expect someone crossing. NHTSA's review of pedestrian safety countermeasures also notes that nighttime pedestrian fatalities rose much faster than daytime deaths over the decade from 2010 to 2019.[2] The reasons are not mysterious. Drivers see pedestrians much later at night, especially pedestrians in dark clothing on poorly lit roads. Drivers are also tired in the evening, and some are impaired. And pedestrians themselves, tired after a long day, are more likely to cross mid-block, to misjudge a gap in traffic or to be looking at a phone. Being seen, and seeing The single most effective thing a pedestrian can do at night is to be visible. Many people greatly overestimate how visible they are to drivers after dark. From the sidewalk, you can see a car's headlights from far away, and it is easy to assume the driver can see you just as clearly. But a person in dark clothing can be very hard to pick out against a dark background until a car is close, especially if the driver is dazzled by oncoming lights or distracted. Some simple measures make a large difference. Wear or carry something light-colored, and ideally something retroreflective: a reflective band on a bag or ankle, a reflective strip on a jacket, a small clip-on light. Reflective material shines back toward the source of light, which means that it lights up in a driver's headlights in a way that ordinary light-colored fabric does not. Keeping a reflective vest or band in your bag, to put on for the walk to the bus or across a dark campus road, is a small habit that can matter a great deal. Cross at marked crossings and intersections wherever possible, and where there are signals, use them, even if it means a longer walk. Make eye contact with drivers before stepping in front of a car, and do not assume that because you can see them, they can see you. Walk facing traffic if there is no sidewalk, so that you can see approaching vehicles. Put the phone away while crossing streets and walking through parking lots, which are full of cars reversing out of spaces and drivers looking for exits rather than pedestrians. Fatigue matters here too. The same lapses of attention that make drowsy driving dangerous make tired walking riskier, and a student who has struggled to stay awake through the last hour of class is more likely to step off a curb without looking properly. If you notice that you are tired, slow down and be deliberate about crossings. Alcohol deserves a mention. Impaired walking is a real contributor to pedestrian deaths, and a student who has a drink after class with classmates before walking to a station is taking on more risk than they may realize. Cycling at night For students who cycle to evening classes, visibility is even more critical. Many states require bicycles ridden at night to have a white front light and a red rear reflector or light, and in any case these are the minimum sensible equipment. A bright rear light, ideally one that can flash, and reflective material on the pedals, wheels or ankles make a cyclist much easier to see. A helmet is essential. Plan a route that uses bike lanes or quieter, well-lit streets, even if it is slightly longer, and ride predictably. If you are very tired after class, it may be safer to walk the bike across complex intersections or, if transit allows bikes, take part of the trip by bus. Personal safety: an honest perspective Personal safety after dark is a subject that can easily slide into fear, and fear is not useful. Most evening students will never experience a crime on their way home. Campuses are, on the whole, relatively safe places. At the same time, evening students are often moving through campuses and neighborhoods at hours when they are quieter, alone and tired, and some sensible habits can reduce the already modest risk further. The most reliable of these habits have less to do with self-defense than with planning and awareness. Know your route before you need it, including where the well-lit paths are, where the emergency phones are and where there are likely to be other people. Park in a well-lit area close to the building you will leave from, not the building you arrive at, since arriving in daylight and leaving in darkness can make a spot that seemed fine at five feel very different at ten. Walk with a classmate when you can; the simplest safety arrangement is to leave with someone going the same way. When walking alone, stay alert to your surroundings. Headphones at high volume and absorption in a phone make it harder to notice what is happening around you. Have your keys ready before you reach your car or front door, so that you are not standing at a door searching through a bag. Glance into the back seat and around the car before getting in, and lock the doors once you are inside. None of this needs to become a source of anxiety; these are simply the habits of people who move around at night regularly. Trust your instincts. If something about a situation feels wrong, whether it is a person, a place or a sense of being followed, act on the feeling without waiting for proof. Go back into a building, move toward other people, call campus security or cross to the other side of the street. The social awkwardness of being wrong is a trivial price. What your campus provides Most colleges offer safety resources designed for exactly the situations evening students face, and it is striking how many evening students never learn about them. Spend fifteen minutes in the first week of term finding out what yours provides. The details vary, but common resources include the following. Escort services. Many campuses run safety escort programs, sometimes called safe walk or safe ride, in which a campus security officer, trained student staff member or volunteer will walk or drive you from a building to your car, a transit stop or another campus location after dark. Some services run on a schedule; others respond on request by phone or app. Escort programs exist to be used, and requesting one is not an admission of fear. It is a sensible choice, particularly for students leaving late from a part of campus that is quiet at night. Emergency phones. Many campuses have emergency call stations, often marked by blue lights, that connect directly to campus police or security. Know where they are on your usual route. Campus police or security. Save the campus police or security number, both emergency and non-emergency, in your phone. For life-threatening emergencies, call 911; but for situations like feeling uneasy about someone in a parking lot, a campus security line may be the fastest way to get someone on the scene. Safety apps and alerts. Many colleges use apps or text systems that let students contact security, share their location with a friend or security staff during a walk and receive alerts about emergencies. Sign up for emergency notifications as soon as you enroll. Shuttles. Some campuses run evening shuttles between buildings, parking areas and nearby transit stations, which can remove the least comfortable walks from your route entirely. It is also worth knowing what information your college is required to share. Under a federal law known as the Clery Act, colleges that take part in federal student aid programs must publish an annual security report, including crime statistics for the campus and certain nearby areas and a description of their safety policies; issue timely warnings about crimes that pose a continuing threat; and, for those with campus police or security departments, maintain a daily crime log available to the public.[3] Evening students can use these documents to understand where and when incidents have occurred on their campus and what the college does about them. Riding transit at night For students who use buses or trains, the ride home has its own logistics. Service is often less frequent in the evening, so a missed connection can mean a long wait. Check schedules in advance, and use real-time arrival apps where available so that you can wait inside a building rather than at an exposed stop. Where possible, choose stops that are well lit and busy, even if it means a slightly longer walk. On the vehicle, sit near the driver or in a car with other passengers. If someone makes you uncomfortable, move, and tell the driver or transit staff if needed. Many transit systems have text or app-based reporting for harassment and safety concerns; find out whether yours does. Fatigue is a hazard on transit too, though of a different kind. Falling asleep and missing your stop is a common experience for tired evening students, and in some places it can leave you at the end of a line late at night, far from home. Setting a phone alarm for a few minutes before your stop, or a location-based reminder if your phone supports one, is a simple safeguard. The walk from the final stop to your door deserves the same attention as the walk on campus. Know the route, keep it well lit where possible and consider asking someone at home to watch for you or to walk to meet you on nights when you feel uneasy. The practical kit A handful of small items, kept permanently in a bag or car, handle most of the minor emergencies that can turn a late journey into a stressful one. A phone charger or small battery pack matters more than people expect: a phone with no charge at 10 p.m. cannot summon an escort, check a bus time, share a location or call for help. A small flashlight, or at least familiarity with the phone's light, helps in dark parking lots and on unlit paths. A reflective band or vest takes up almost no space. Some cash or a transit card with credit on it covers the night when the card reader fails or a rideshare is needed. For drivers, a membership in a roadside assistance program, or at least the number of a towing service, a basic emergency kit and a habit of not letting the fuel tank run low before class nights, prevent the particular misery of a breakdown on a dark road after a long day. It is also worth having a short list of numbers stored in your phone and, ideally, written on a card in your wallet in case the phone is lost or dead: campus security, a backup ride, a taxi company and someone at home. If something does happen Despite every precaution, things sometimes go wrong. If you are in immediate danger or witness a crime in progress, call 911, or use a campus emergency phone, and move toward light and other people. If you are able, note details such as descriptions, vehicles and the direction someone went, but your safety comes before any of that. After an incident, even one that ends without physical harm, it is common to feel shaken for days or weeks, and evening students often try to push through without telling anyone, because they have work the next day and class the next week. Reporting to campus police or security is your choice in most situations, but doing so can help the college warn others and, where relevant, investigate. Colleges also have staff whose job is to support students after harassment, assault or other crimes, whether or not the student makes a formal report. These may include counseling services, victim advocates and the office responsible for handling reports of sexual harassment and violence. They can help with practical issues too, such as changing class sections or arranging an escort for the rest of the term. For anyone affected by sexual violence, the National Sexual Assault Hotline, run by RAINN, offers free, confidential support at any hour at 1-800-656-4673. For anyone whose distress becomes overwhelming, including thoughts of self-harm, the 988 Suicide and Crisis Lifeline is available by call or text at 988. Letting someone know One habit ties all of this together: let someone know when you leave and when you arrive. A brief text to a partner, family member or friend saying you are leaving campus, followed by another when you get home, creates a simple safety net. If something goes wrong, someone will know sooner. Many phones now offer location-sharing features that do this automatically for a set period; used with care and consent, they can reduce both risk and worry for the people waiting at home. Planning the journey as part of the evening In the backward plan from Chapter 2, the trip home was one of the fixed points, and this chapter adds detail to it. For a driver, the plan is about alertness. For a pedestrian, cyclist or transit rider, it is about visibility, awareness and logistics: knowing the route, the schedule and the resources available and deciding in advance how you will handle a night when you are more tired than usual or a situation feels wrong. None of this should make the evening journey feel frightening. Most nights, it will be uneventful: a walk to the car, a bus ride, a text to say you are home. The point of preparing is to keep it that way. Chapter 8: Landing the Evening The front door closes behind you at 10:05. You have been awake for more than sixteen hours. You are tired in the bone-deep way that comes from a full day of work followed by three hours of concentration. And yet, as many evening students discover, you are not ready to sleep. Your mind is still running through the lecture, the problem set due next week, the email from your manager, the permission slip your daughter needs signed. Your body feels exhausted, but your brain feels switched on. This is one of the most common complaints of evening students, and it is the part of the evening they are least likely to plan for. The class has a start and end time. The drive home has a route. But the hour between walking in the door and falling asleep is usually left to chance, and chance tends to fill it with a phone, a snack, a television and a bedtime that slides steadily later. This chapter is about that hour: why it is hard, and how to use it well. Why you feel wired Several things combine to make falling asleep after a late class harder than it should be. The first is simple mental activation. Sustained concentration on new and challenging material is stimulating, and the brain does not switch off the moment the lecture ends. Many students replay what they learned, worry about what they did not understand or start mentally planning the work to come. This kind of cognitive arousal is one of the main contributors to difficulty falling asleep. The second is light. Chapter 1 explained that bright light in the evening signals the circadian clock that it is still daytime, suppresses melatonin and can push the clock later. Most students spend the evening in brightly lit classrooms, then drive home through lit streets, then walk into a lit house and pick up a lit phone. By the time they try to sleep, their bodies have received a strong and continuous daytime signal. The third is whatever caffeine is still circulating. As Chapter 4 showed, a coffee at five in the afternoon can leave the equivalent of a small cup still active at eleven. Students who drink caffeine at the class break are often carrying more than that. The fourth, for many, is the transition itself. Arriving home, particularly to a household with children, a partner or other responsibilities, often means stepping straight from one role into another: from student to parent, from student to partner, from student to the person who notices that the dishes are not done. The emotional shift can be as stimulating as the lecture. None of these is a sign that something is wrong. They are the predictable effects of the schedule. The answer is to build a deliberate transition, a wind-down period that takes each of them in turn and gives your mind and body the signals they need to shift toward sleep. The wind-down window In the backward plan from Chapter 2, the wind-down was a period of roughly thirty to sixty minutes before lights out. What goes in that window matters less than that it is consistent, calm and different from what came before. Over time, a consistent routine becomes a signal in itself: the brain learns that these steps lead to sleep. Dim the lights. As soon as you get home on a class night, lower the lighting. Use lamps rather than overhead lights, and warmer, dimmer bulbs where possible. This is one of the simplest and most effective changes, and it directly counters the bright-light signal of the evening. Manage screens. The evidence here is worth knowing in some detail. In a carefully controlled laboratory study published in 2015, Anne-Marie Chang, Charles Czeisler and colleagues at Brigham and Women's Hospital compared reading on a light-emitting e-reader with reading a printed book in the hours before bedtime. When participants read on the light-emitting device, they took longer to fall asleep, felt less sleepy in the evening, secreted less melatonin, showed a later timing of their circadian clock and were less alert the next morning.[1] The authors noted that a national survey had found that 90 percent of Americans used some kind of electronic device at least a few nights a week within an hour of bedtime. Screens also affect sleep through what is on them. Email, news, social media and messages are designed to be engaging, and they often raise exactly the kind of mental activation that makes sleep harder. A realistic goal for most students is not to abolish screens in the evening but to put a boundary around them: no work email after class, no social media in bed and the phone charging somewhere other than the bedside for the last half hour before lights out. Night-mode settings that warm the screen color may reduce some of the light's effect, but they do not address the stimulation of the content. Close the loops. Much of the mental activity that keeps evening students awake is about unfinished business. A simple, well-tested technique is to write it down. In a 2018 study, Michael Scullin and colleagues at Baylor University asked young adults to spend five minutes before bed either writing a to-do list of tasks they needed to complete in the coming days or writing about tasks they had already completed. Those who wrote to-do lists fell asleep significantly faster, and the more specific their lists, the faster they fell asleep.[2] For an evening student, this can become a fixed part of the wind-down: five minutes with a notebook, writing down what needs to happen tomorrow and when. The act of writing it down seems to allow the mind to stop holding it. Do something calm. Reading a printed book or magazine, taking a warm shower, gentle stretching, listening to quiet music or a podcast that is interesting but not gripping, a few minutes of slow breathing or a brief conversation with someone at home can all serve. What matters is that it is calm, pleasant and not work. Keep food light. If you are hungry, have a small snack, not a meal, as discussed in Chapter 3. A piece of fruit, a small yogurt, a slice of toast or a handful of nuts will settle hunger without loading your stomach before sleep. The bedroom itself The room you sleep in matters too, and for evening students it often needs a little more attention than for people who go to bed earlier in a quieter house. Sleep specialists commonly recommend a bedroom that is dark, quiet and comfortably cool, and each of these can be harder to achieve at eleven at night in a busy household or an apartment building. Darkness is the easiest to fix. Blackout curtains or blinds shut out streetlights, and an eye mask costs very little and works almost anywhere. Cover or turn away bright standby lights on chargers and electronics. If you need to get up in the night, a small, dim night light is better than switching on the overhead light. Noise is harder, particularly for students who come home to a household that is still awake: a partner watching television, older children, roommates. Earplugs help many people, and a fan or a white-noise machine can mask intermittent sounds such as voices, traffic or doors, which tend to be more disruptive than steady ones. It is also reasonable to ask the people you live with for some quiet after a certain hour on class nights; the next chapter discusses how to have that conversation. Temperature varies more between individuals, but most people sleep better in a room that is slightly cooler than they would choose for sitting around in the evening. If you share a bed with a partner who runs warmer or cooler, separate blankets can solve more arguments than a thermostat. For students whose partners go to bed hours earlier and are woken when they come in, a few practical arrangements help: getting ready for bed in another room, keeping clothes for the morning laid out elsewhere and using a small light rather than the main one. Some couples find that on class nights, the late partner sleeping in another room altogether is the kindest arrangement for both. There is no need to treat this as a sign of a problem in the relationship; it is logistics. Alcohol is not a sleep aid Many adults use a drink in the evening to unwind, and after a long day and a demanding class, a glass of wine or a beer can seem like a reasonable reward. It is worth understanding what alcohol actually does to sleep. A 2013 review of the research by Irshaad Ebrahim, Colin Shapiro and colleagues found that alcohol shortens the time it takes to fall asleep and, at higher doses, increases deep sleep in the first half of the night, but that it increases sleep disruption in the second half of the night and, at moderate and high doses, reduces the total proportion of dreaming, or REM, sleep.[3] The authors concluded that alcohol should not be used as a sleep aid. In other words, alcohol helps you fall asleep faster and then makes the rest of your night worse. For an evening student already short of sleep, that is a poor trade. If you enjoy a drink, the best place for it is on an evening without class, well before bed and never before driving. When sleep will not come Even with a good wind-down, some nights you will lie awake. Sleep specialists generally advise against lying in bed for long periods trying to force sleep, because the bed can gradually become associated with frustration and wakefulness. If you find yourself awake for what feels like more than twenty minutes or so, get up, go to another room with dim light and do something quiet and boring until you feel sleepy, then return to bed. Do not check the time repeatedly; clock-watching tends to increase anxiety about sleep. Occasional nights like this are normal. If they become the pattern, if you regularly lie awake for long periods, wake frequently or wake too early and cannot get back to sleep, and it has been going on for weeks, talk to a clinician. Chronic insomnia is common, and the treatment recommended as first-line by major medical bodies is not medication but cognitive behavioral therapy for insomnia, a structured program that addresses the thoughts and habits that keep insomnia going. It is available through sleep clinics, some therapists and, increasingly, through validated digital programs. A caution about sleep aids Some evening students turn to sleep aids, either over-the-counter products or prescriptions, to force sleep after a late class. This deserves caution for a reason that connects directly to the rest of this book: some sleep medications can leave people impaired the next morning, including while driving. In 2013, the U.S. Food and Drug Administration required lower recommended doses for certain insomnia drugs containing zolpidem, because of evidence that blood levels in some patients, particularly women, remained high enough the next morning to impair activities requiring alertness, such as driving.[4] Many over-the-counter sleep aids contain sedating antihistamines, which can also cause next-day drowsiness. Melatonin is widely sold as a dietary supplement in the United States, which means it is not regulated in the same way as a medicine, and the dose and timing that might be appropriate vary considerably between people and purposes. If you are considering any sleep aid, including melatonin, discuss it with a clinician or pharmacist, especially if you drive early in the morning. The morning after Landing the evening well includes the next morning. On the day after a late class, a few habits help keep the circadian clock anchored and the sleep debt from compounding. Keep a regular wake time. It is tempting to sleep in after a late night if your schedule allows, and occasionally that is reasonable. But a large shift in wake time pushes the clock later, making the next night's sleep harder. If you need more sleep, a modest adjustment of an hour or less is usually better than a large one, and an early-afternoon short nap is often better still. Get morning light. Outdoor light in the first hour or two after waking is the most powerful signal the clock receives to stay on an earlier schedule. A short walk, a coffee by a window, or getting off the bus a stop early can all provide it. This is the counterweight to the bright evening light of class. Use caffeine in the morning. The morning after a late class is exactly when caffeine is most useful and least costly. It can help with the grogginess of a short night, and it will be long cleared by the following evening. Be kind to yourself. The morning after a class night is often the hardest part of the week. It helps to schedule less demanding tasks for it where you can, and to accept that you may not be at your best. Three wind-downs What a good wind-down looks like depends heavily on who is waiting at home. Three illustrative examples show how the same principles adapt to different households. Consider first a student who lives alone. She arrives home at 10:05 and her anchor is 11:30. She switches on two lamps rather than the ceiling light, plugs her phone in on the kitchen counter and puts the kettle on for a caffeine-free herbal tea. While it steeps, she writes a short list of what needs to happen tomorrow, including the one reading she wants to review at lunch. She takes a shower, reads a novel in bed for twenty minutes and turns out the light at 11:05. Nothing in the sequence is special, which is the point; after a few weeks, it runs almost automatically. Now consider a father of two young children whose partner has handled bedtime. He gets home at 9:50. The children are asleep, but his partner has been waiting to talk, and he wants to hear about the evening. They sit together in the kitchen with the lights low for twenty minutes, deliberately keeping the conversation to the day rather than to the family budget or next week's scheduling problems, which they have agreed to save for the weekend. He writes his to-do list while his partner gets ready for bed, then reads for fifteen minutes. The conversation is part of his wind-down rather than an obstacle to it, because they have agreed what kind of conversation it is. Finally, consider a student who shares an apartment with two roommates who are still up and watching a film when she gets home at 10:30, with an anchor of midnight. She says hello, eats a yogurt in the kitchen rather than joining them, and goes to her room. She uses a fan for background noise and wears an eye mask because light shows under the door. Her wind-down is shorter than she would like, but it still has the same elements: dim light, the list, a calm activity and a consistent lights-out. She has asked her roommates to keep the volume down after midnight on weeknights, and on most nights they do. In each case the routine is modest, and none of them follows it perfectly every week. What they share is that the last hour has a shape, decided in advance, rather than being whatever happens to fill it. The evening, reconsidered Most evening students think of their class night as ending when class ends. In practice, the night ends when you fall asleep, and the last hour matters as much as any other. A student who arrives home, dims the lights, writes tomorrow's list, eats a small snack if needed, reads for twenty minutes and turns off the light at a consistent time is giving themselves a very different night from one who scrolls on the couch until 12:30 with a bowl of leftovers and a glass of wine. The difference is not willpower. It is having decided in advance what the last hour will look like. The next chapter turns to the people who share that last hour: the families, partners and employers who surround every evening student. Chapter 9: The People at Home and at Work Every evening student lives inside a web of other people's schedules. A class from six to nine is not only three hours of the student's time; it is three hours in which someone else is putting the children to bed, or the dog is not walked, or a parent is not visited, or a shift cannot be covered. The morning after is not only the student's hard morning; it is the morning their manager expects them at their desk at eight and their partner expects them to take the children to school. Wellness advice that treats the student as an isolated individual, free to set their own bedtime and meal schedule, misses most of what makes these habits hard to keep. The Lumina Foundation reports that roughly four in five college students work, about 30 percent of them full time, and that close to one in five has children or other dependents.[1] Among evening students those proportions are, by the nature of the schedule, often higher. For them, the question is not only what to do, but how to arrange the rest of life so that doing it is possible. This chapter is about those arrangements: at home, at work and with the college itself. The conversation at home The single most useful thing many evening students can do for their health during a term is to have one clear conversation with the people they live with before it starts. Not a vague announcement that "school is going to be busy," but a specific discussion of what class nights will look like and what each person needs. It helps to bring the backward plan from Chapter 2 to that conversation. If you can say, "On Tuesdays and Thursdays I'll be home at about ten, and I need to be asleep by 11:30, so I need the time between ten and eleven to be quiet," the people you live with have something concrete to work with. They can see what you need and where they fit. Vague requests tend to produce vague cooperation; specific ones are much easier to honor. Several topics are worth covering. Who handles dinner, children's bedtimes, homework help, pets and household tasks on class nights? What is the plan when a child is sick on a class night? Who can pick you up if you are too tired to drive, as discussed in Chapter 6? What does the household need from you on the nights you are home, and on weekends, so that the load does not simply shift permanently onto someone else? How will you stay connected, since a partner who has handled every Tuesday and Thursday evening alone for fifteen weeks may feel something closer to loneliness than resentment? That last question is easy to overlook. An evening program changes the rhythm of a household in ways that affect everyone. Partners may feel they have become single parents two nights a week. Children may miss bedtime stories. Older parents who rely on a student for company or care may feel neglected. Acknowledging these costs openly, and agreeing on ways to protect time together, such as a weekly meal, a regular Saturday morning, or a phone call from the car before class, helps prevent the slow build-up of friction that can undermine both the relationship and the student's ability to keep going. Children and the evening student Student parents face some of the hardest scheduling problems in higher education. Child care that covers evening hours is often difficult to find and expensive, and the evening class may fall exactly across dinner, bath and bedtime, the most labor-intensive part of a young child's day. Some colleges operate campus child care centers, and some of those offer evening care or drop-in options for students; a federal program called Child Care Access Means Parents in School helps fund campus-based child care for low-income student parents at participating institutions. It is worth asking your college's student services office what is available, since these programs are often underused simply because students do not know they exist. Community organizations, family members, other student parents and neighbors who can trade care are other possibilities. When children are old enough to understand, involving them in the plan can help. A child who knows that Mom has class on Tuesdays, that Grandma will do bedtime and that Mom will be there for breakfast often copes better than one for whom the absence is unexplained. Some student parents find that sharing a little of what they are learning, or letting children see them study, turns the class into something the family is doing together rather than something taking a parent away. Many adult students cite their children as a reason for returning to school; letting children see that effort can be a quiet form of teaching. Student parents also need to be realistic about their own sleep. A parent of a baby or toddler who wakes in the night is carrying a sleep debt before any class begins, and the advice in this book about caffeine timing, drowsy driving and protecting sleep applies with extra force. If night wakings are shared with a partner, it is reasonable to ask that the class-night parent be spared them on those nights, and to repay the favor on other nights. The conversation at work For students who work, the workplace can either support or undermine every habit in this book. The morning after a late class is the pinch point of the week, and whether an employer allows even a small amount of flexibility around it can make the difference between a sustainable schedule and an exhausting one. It is worth considering carefully whether and how to raise your studies at work. Many employers are supportive, and some offer formal tuition assistance, flexible scheduling or study leave; others may be indifferent or, occasionally, see studies as a sign that an employee plans to leave. You know your own workplace best. But where it is reasonably safe to do so, a straightforward conversation with a manager can produce practical help: a later start on the morning after class, not being scheduled for late shifts on class days, the ability to leave on time on class afternoons rather than being asked to stay for a late meeting, or a quiet space to eat before leaving. As with the conversation at home, specific requests work better than general ones. "Could I start at 9:00 rather than 8:00 on Wednesdays, and make up the hour on Friday?" is easier for a manager to say yes to than "I'm going to be really busy this term." Framing the request around your work performance, such as being sharper on the morning after class, can also help, and it has the virtue of being true. For students who work shifts, particularly rotating or night shifts, the challenges are greater. Shift work itself disrupts the circadian clock, and combining it with evening classes can leave very little room for regular sleep. If this describes your situation, prioritize a consistent sleep period as much as your rota allows, be especially vigilant about drowsy driving and consider discussing your schedule with a clinician, because the health effects of combining shift work with study can accumulate quietly over a term. The college as a partner Colleges that serve large numbers of evening students increasingly recognize that those students need services at times they can actually use them. Advising, tutoring, counseling, financial aid, the library and health services may offer evening hours, online appointments or both. Many evening students assume these services are only for daytime students and never ask. Ask. Several college resources are particularly relevant to the themes of this book. Counseling services can help with the stress, anxiety and low mood that often accompany a heavy load. Health services may be able to assess persistent fatigue or sleep problems. Disability services can arrange accommodations for health conditions that affect energy, alertness or scheduling. Basic needs programs, including food pantries, emergency grants and help with public benefits, address the practical pressures that undermine wellness, as Chapter 3 discussed. Campus safety offices provide escort services and safety information, as Chapter 7 described. Instructors, too, are partners. Most evening instructors understand the lives of their students and will respond reasonably to a respectful conversation about a problem: an occasional late arrival because of a shift change, a missed class because of a sick child or a request for recordings or slides. Telling an instructor early that you are juggling work and family, before a crisis, tends to produce more flexibility than explaining it after one. Knowing your limit Perhaps the most important decision an evening student makes about wellness happens before a term begins: how many courses to take. The pressure to finish quickly is real. Tuition costs, the desire for a better job and the sense that life is on hold until the degree is done all push toward a heavier load. But the arithmetic of the backward plan applies at the level of the term as well as the evening. Each additional evening class is another night with late caffeine temptations, a displaced dinner, a late drive and a short sleep, and another morning after. There is no universal right number. A student with a flexible job, supportive family and short commute may manage three evening classes comfortably; another with a long commute, young children and a physically demanding job may find that one is the limit. What matters is making the decision deliberately, with an honest look at the whole week, rather than by default. Finishing a program a term or two later, in good health, is a better outcome than burning out halfway through and withdrawing. If you find partway through a term that the load is unsustainable, talk to an adviser before the withdrawal deadline. Dropping one course to protect the others, and your health, is a legitimate choice, not a failure. When the plan meets a bad week Every term contains at least one week when everything goes wrong at once: a child with a fever on a class night, a mandatory late meeting, a parent in the hospital, a car in the shop, an exam the next morning. The habits in this book tend to collapse in exactly those weeks, and that is when they matter most, because a week of crisis is also a week of short sleep and high stress, often with more driving than usual. It helps to decide in advance what your fallback rules are, so that you are not making them up at eleven at night. A reasonable set might look like this. Sleep is protected before study: if something has to give, it is the reading, not the night. Driving is only done alert: if you are too tired, you use a backup ride, a nap or a later departure, whatever the inconvenience. Missing one class is recoverable: a quick message to the instructor and a classmate's notes are almost always better than attending exhausted and driving home impaired. And food is simplified rather than skipped: in a crisis week, a supermarket sandwich and a piece of fruit before class is a perfectly good dinner. It also helps to have agreed with the people around you what happens in a crisis week. Who covers child care if your usual arrangement falls through? Can a classmate share notes? Is there someone at work who can cover the morning after class if you have been up at the hospital? These arrangements are easier to set up in week one, when everyone is calm, than in week nine, when they are needed. What supporters can do This book is written mainly for students, but many of its readers will be the people around them: partners and parents, advisers and instructors, managers and college administrators. For them, the evidence in the earlier chapters points toward some concrete forms of help. Partners and family members can take on class-night tasks without keeping score, protect the quiet hour after the student gets home, offer rides on exhausting nights without judgment and keep the conversation on class nights light, saving difficult household decisions for another time. Employers can recognize the morning after class as the hardest part of the week and allow flexibility around it where the work permits, avoid scheduling late meetings or overtime on class afternoons and make tuition assistance and study-leave policies easy to find and use. Many of these cost little and are repaid in a more rested, more loyal employee. Instructors of evening classes can build in a real break, vary activities through a long session so that students are not asked to listen passively for three hours, post materials so that a moment of drowsiness does not mean lost content, and end on time, since for a student with a long drive or a late bus, ten extra minutes can mean a missed connection or a later, riskier trip home. Colleges can match their services to the hours evening students keep. That means food available on campus in the evening, not only vending machines; counseling, advising and health appointments at times working students can attend; safety escort services and shuttles that run until after the last class lets out rather than stopping before it; well-lit parking areas and paths; and orientation materials that tell evening students, specifically, what is available to them. None of these is exotic. They simply reflect the fact that a large share of students now attend at night. Stress, mood and when to get help Combining work, family and study is stressful, and some degree of stress is part of the bargain. But chronic short sleep, irregular meals and constant pressure can combine to wear down mood and resilience over a term, and evening students are often reluctant to seek help, both because of time and because they feel that struggling is simply the price of their choice. Watch for signs that stress has become something more: persistent low mood or loss of interest in things you normally enjoy, feeling hopeless, irritability that is damaging relationships, withdrawal from people, significant changes in appetite, reliance on alcohol or other substances to cope or an inability to function at work or in class. These deserve attention from a professional, whether through college counseling, your primary care provider or an employee assistance program if your employer offers one. If you ever have thoughts of harming yourself, or feel that you cannot go on, reach out immediately. In the United States, you can call or text 988 to reach the 988 Suicide and Crisis Lifeline at any hour. If you are in immediate danger, call 911. Borrowing strength Evening students often describe themselves as doing this alone: going to class alone, driving home alone, studying alone after everyone else is asleep. In practice, most of them are carried by a network of people who make it possible. Recognizing that network, asking clearly for what you need from it and thanking the people in it are not only courtesies. They are part of what makes the habits in this book sustainable. The same goes for fellow evening students. The person in the next seat is probably dealing with a version of your problems. Classmates can share rides, compare notes, form study groups that meet online at convenient times, walk together to the parking lot and simply provide the reassurance that someone else understands. The first night of class is a good time to find them. Conclusion It is tempting to read a book like this as a list: eat dinner earlier, stop caffeine by mid-afternoon, nap for twenty minutes rather than an hour, wear something reflective, dim the lights when you get home. The list is real, and every item on it is worth doing. But looked at together, the chapters point to something more useful than a list, and it is worth drawing out, because it changes how an evening student should think about the whole term rather than a single night. One decision, not many The first conclusion is that the evening student's apparently separate choices are really one choice seen from different angles. Dinner timing, caffeine timing, how you use the class break, how you get home and what you do in the last hour before bed all look like separate problems with separate solutions. They are not. Each one either protects or erodes the same thing: the night's sleep. A late coffee costs sleep through caffeine still in the blood. A late, heavy meal costs it through reflux and discomfort. A bright, screen-filled last hour costs it through a delayed clock and an activated mind. And lost sleep, in turn, makes every other choice harder the next day: you reach for more caffeine, you eat less carefully, you drive home more tired. This is why the book began with a single rule, to plan backward from the time you intend to be asleep, and why that rule turns out to be more than a scheduling trick. It is a way of seeing that the evening is a system, and that the system has one output that matters more than the others. When you choose the anchor first, most of the other decisions stop being willpower contests at the moment of temptation and become arithmetic done in advance. It also explains a pattern that many students notice once they start: changing one habit tends to make others easier. The student who moves caffeine earlier falls asleep faster, wakes less exhausted, needs less caffeine the next afternoon and finds the drive home less of a struggle. The student who eats a proper meal before class stops arriving home ravenous and so stops eating at eleven. The effects compound in both directions, which is why a modest change made consistently tends to be worth more than a dramatic one abandoned after a week. Not all compromises are equal The second conclusion concerns what to do when the plan fails, which it will. Evening students live with little slack, and no one keeps every habit every week. What the chapters make clear is that the costs of different compromises are not remotely equal, and that it pays to know the difference before the difficult night arrives. Most compromises are recoverable. A dinner eaten late, a coffee drunk after the cut-off, a night of scrolling, a skipped wind-down: each costs some sleep, some comfort, perhaps some learning. A few bad nights are absorbed by a few good ones. The body's systems for regulating sleep and appetite are resilient, and a student who slips and then returns to the plan loses little in the long run. One compromise is different in kind. Driving while drowsy carries a risk that cannot be recovered from if it goes wrong. The research in Chapter 6 on the prevalence of drowsy crashes, their severity and people's poor ability to judge their own impairment means that the ordinary logic of "just this once" does not apply. A thousand uneventful drives do not make the next one safe. This asymmetry suggests a simple ordering of priorities for a hard week. Be flexible about food. Be reasonably disciplined about caffeine, since it is the easiest lever and the quickest to repay. Protect sleep as much as the week allows. And treat the rule about driving only when alert as absolute, arranging backup rides, naps or delayed departures in advance, so that on the night you need them, you are following a plan rather than making a hard decision while exhausted. The term is the unit The third conclusion is that the evening, which is where this book has focused, is not really the right unit for thinking about an evening student's health. The term is. Sleep debt accumulates across nights. The Van Dongen study discussed in Chapter 2 found that people restricted to six hours a night kept getting worse over two weeks, while feeling only slightly sleepier. A student who averages six hours through a fifteen-week term is not experiencing fifteen weeks of the same mild tiredness; they are likely experiencing a slow decline in attention, memory and mood that they may not notice until it shows up in grades, relationships or a near miss on the road. The late-eating research, too, concerns patterns rather than single meals. The consequences of the evening schedule are cumulative, and so the most powerful decisions about it are the ones made at the level of the term. That shifts attention from the individual night to a handful of structural choices, most of which are made before the first class. How many evening courses to take at once. Which nights to take them on, and whether the morning after each can be protected at work. How long the commute is, and whether there is a backup way home. Who will cover the household on class nights, and whether that arrangement is fair and sustainable for them. Which college services exist at hours you can use. These decisions are less vivid than choosing between coffee and water at the break, but they set the conditions in which every nightly choice is made. A student who gets them right will find the nightly habits far easier; a student who gets them wrong may find that no amount of nightly discipline is enough. This is why the chapter on families, workplaces and colleges is not an afterthought. The evening student's wellness is a shared project whether or not anyone acknowledges it. The partner who does bedtime, the manager who allows a 9:00 start on Wednesdays, the instructor who ends on time and the college that runs its escort service until after the last class all contribute to whether the student sleeps, eats and gets home safely. Making those contributions explicit, and asking for them clearly, is part of the work. A different measure of success Evening students tend to measure success by endurance: getting through the class, getting through the week, getting through the term. The capacity to endure is real and admirable, and many evening students have far more of it than their daytime peers. But endurance is a poor guide to health, because it rewards pushing past exactly the signals, fatigue, hunger, drowsiness at the wheel, that the body uses to protect itself. A better measure is sustainability. Can you keep this schedule up for the length of the program without your health, your relationships or your safety paying for it? If the answer is yes, the details of this book are refinements. If the answer is no, the most important thing you can change is not your caffeine cut-off but your schedule, and there is no shame in changing it. A degree finished a term later, by someone who is well rested, still married, still close to their children and never had the crash they came close to, is a better outcome by any measure that matters. The people in the parking lot at a quarter to six, eating granola bars and finishing phone calls, have already shown that they have the determination to do something hard. What they need is not more determination but a way of arranging their evenings, and their terms, that lets that determination last. That is what these habits are for: not to make night classes easy, which they will never be, but to make them survivable, healthy and safe, one well-planned evening at a time. Hashtags: #HealthyHabits #NightClasses #EveningStudents #StudentWellness #NonTraditionalStudents #AdultLearners #EveningClassWellness #HealthyStudentLifestyle #SleepHygiene #SleepHealth #CircadianRhythm #SleepManagement #CaffeineManagement #HealthyMealPlanning #MealTiming #HealthyEatingHabits #StudentNutrition #EnergyManagement #FatigueManagement #DrowsyDrivingPrevention #CampusSafety #LateNightSafety #StressManagement #WorkStudyBalance #WorkLifeStudyBalance #TimeManagement #StudentMentalHealth #HealthyStudyHabits #EveningRoutine #LifelongLearning