Download the Book (PDF): Introduction As the sequel to Prediction Machines, Power and Prediction tackles the "between times": the difficult transitional phase in which AI is adopted as a point solution before it disrupts system-level architecture. Understanding this adoption lag is crucial for students of IT management. But tracing the historical analogies of electrification and early computing to anticipate how modern AI will be integrated, and on what timeline, requires a highly structured analytical approach. This guide provides that structure. It is written explicitly to explain Power and Prediction, organizing the authors' historical and economic research into clear academic models. It breaks down, carefully and with examples, the difference between point solutions, application solutions and system solutions. With targeted case study analysis, it equips you to examine critically the systemic hurdles of enterprise AI adoption in advanced capstone projects. The book and its authors Power and Prediction: The Disruptive Economics of Artificial Intelligence was published by Harvard Business Review Press in 2022. Its three authors, Ajay Agrawal, Joshua Gans and Avi Goldfarb, are economists at the University of Toronto's Rotman School of Management. Agrawal founded the Creative Destruction Lab, a program that helps science-based start-ups grow, and all three have spent years watching entrepreneurs try, and often fail, to turn advances in machine learning into businesses. That vantage point matters. The book is not written from a laboratory, where the question is whether a model works, but from the market, where the question is whether anyone can build an organization around it. The authors' earlier book, Prediction Machines (2018), made a simple claim that proved influential. Strip away the science fiction, they argued, and the current wave of artificial intelligence is one thing: a technology that makes prediction cheap. Prediction here means using information you have to generate information you do not have, whether that is the next word in a sentence, the chance a loan will default, or whether a shadow on an X-ray is a tumor. When the cost of something falls, economics tells us two things happen. We use more of it, including in places we never thought of as prediction problems. And the value of its complements rises, while the value of its substitutes falls. For prediction, the key complement is judgment: knowing what outcomes matter and how much, which the machine does not supply. Prediction Machines was, in effect, a book about the individual decision. It showed managers how to find the predictions buried inside tasks and how to weigh the value of improving them. Power and Prediction starts from an awkward fact that the first book did not fully explain. By the early 2020s, prediction technology had improved dramatically, yet most organizations had little to show for it. The authors' answer is that the value of AI rarely comes from improving one decision in isolation. It comes from redesigning the system of interlocking decisions, rules and procedures that the decision sits inside. That redesign is slow, expensive and politically fraught, because it moves power from some people to others. The period in which the technology is clearly capable but the systems that would exploit it have not yet been built is what the authors call the Between Times. Why the argument matters now The book appeared a few months before generative AI became a mass consumer product at the end of 2022. It would be easy to assume that the arrival of large language models made its argument obsolete. The opposite has happened. Since 2023, organizations have poured money into generative AI tools, and most surveys report widespread use by individual employees. Yet the evidence of organization-wide transformation remains thin. Industry reports describe a pattern of promising pilots that never reach production, a condition consultants now call "pilot purgatory". Economists continue to debate why aggregate productivity has not visibly accelerated. The shape of this experience is almost exactly what Power and Prediction anticipated: fast adoption of tools that slot into existing workflows, slow and uneven change in the workflows themselves. For a student of IT management, this makes the book unusually practical. Enterprise technology projects fail far more often from organizational causes than technical ones, and the authors give a vocabulary for naming those causes precisely. Why does a highly accurate model deliver little value? Because its predictions feed a decision that a rule has already fixed. Why does a department resist a tool that would clearly help the firm? Because the tool relocates judgment, and with it authority, away from that department. Why is it so hard to change one process? Because the rule that governs it is the glue holding several other processes in place. These are the questions a capstone project on AI adoption must answer, and the book supplies the models for answering them. The controlling idea of this guide One idea runs through everything that follows. AI's large economic effects arrive only when organizations redesign the systems around cheap prediction, and because such redesign reallocates judgment, it reallocates power; the Between Times last as long as the people who hold that power, and the rules that protect it, remain in place. Put more simply: the obstacle to AI value is rarely the algorithm. It is the system, and behind the system, the distribution of power. Holding that idea steady helps you read the book well. Each of its major concepts, from rules and hidden uncertainty to decoupling and the discovery canvas, is a piece of the explanation for why the system is hard to change and what changing it would require. When you evaluate a real organization's AI program, the same idea gives you a test: is this a new tool inserted into an old system, or a new system built around the tool? And who stands to lose if the system changes? How this guide is organized The guide follows the logic of the authors' argument rather than retelling the book page by page. It begins with the foundation inherited from Prediction Machines, restating the economics of prediction and judgment from scratch so that no prior reading is required. It then turns to history, the story of how American factories took roughly four decades to redesign themselves around electric power, and to the productivity paradox that economists observed when computers spread without showing up in the statistics. That history is the source of the book's central distinction among point, application and system solutions, which receives its own careful treatment, including a worked example that follows one industry through all three levels. The middle of the guide examines why systems resist change. Here the authors' analysis of rules is essential: rules are decisions made in advance to avoid the cost of deciding, they hide uncertainty that better prediction could expose, and they bind the parts of an organization together in ways that make piecemeal change dangerous. From there the guide takes up power, the book's most distinctive contribution, explaining how system change shifts economic power between firms, within firms, and between jobs, and how AI separates the prediction component of a decision from its judgment component so that judgment can move to new hands. The later part of the guide is constructive. It sets out the authors' tools for imagining new systems, including their blank-slate approach and the AI Systems Discovery Canvas, and their argument that well-designed AI systems can be more reliable and less discriminatory than the human systems they replace, precisely because they can be audited. The final chapter brings the framework to the present, testing it against the adoption data, productivity studies and enterprise surveys available by 2026, including claims about agentic AI and about failure rates in corporate pilots, some of which deserve more skepticism than they have received. Throughout, the guide distinguishes between what the authors argue and what it adds. Where it summarizes the book, it attributes the ideas to Agrawal, Gans and Goldfarb. Where it brings in later evidence or offers critique, it says so. Each chapter closes with key takeaways and review questions designed for seminar discussion and exam preparation, and the back matter provides a glossary of the terms that recur most often and a short list of further reading for students who want to go deeper. Chapter 1: AI as Prediction Technology Every argument in Power and Prediction rests on a foundation laid in the authors' earlier book, Prediction Machines. The foundation is deliberately narrow. Agrawal, Gans and Goldfarb do not ask what intelligence is, whether machines can think, or when they will surpass people. They ask a question economists are trained to ask about any technology: what has it made cheaper? Their answer is prediction. Everything else in their analysis follows from taking that answer seriously and tracing its consequences through organizations and markets. This chapter rebuilds that foundation from the ground up. It explains what the authors mean by prediction, why a fall in its cost matters, how prediction fits into the anatomy of a decision, and why the most important complement to prediction, human judgment, becomes more valuable rather than less. It closes with an example the authors use to show that even a company as sophisticated as Amazon faces limits on what cheap prediction alone can do. What the authors mean by prediction In everyday speech, prediction means forecasting the future. The authors use a broader definition: prediction is the process of filling in missing information. It takes data you have and produces information you do not have. Sometimes the missing information does lie in the future, as when a retailer predicts next week's demand. But often it lies in the present or the past. When a bank's system flags a card transaction as probably fraudulent, it is predicting a fact about something that has already happened. When a phone's software turns speech into text, it is predicting which words were spoken. When a radiology model labels an image, it is predicting what a trained expert would say is there. This broad definition is the key that unlocks the authors' view of modern AI. Machine learning, the family of techniques behind almost every commercial AI system, works by finding statistical patterns in large quantities of example data and using those patterns to fill in the blanks for new cases. Translation, image recognition, recommendation, fraud detection, and the generation of text by large language models are all, in this framing, prediction problems. A language model predicts which token is likely to come next given everything before it; the fluency of its output is the accumulated result of many such predictions. Seen this way, much of what looks like general intelligence is better understood as a very capable prediction engine. The framing is not meant to belittle the technology. Its purpose is analytical. Once you see AI as a prediction technology, you can apply a large body of economic reasoning about what happens when an input becomes cheap. The economics of a falling cost The authors draw an analogy that runs through both books. In the second half of the twentieth century, computing became dramatically cheaper. Computers are, at bottom, arithmetic machines, and as the cost of arithmetic fell, two things happened. First, arithmetic was used more heavily in tasks that had always depended on it, such as accounting, census tabulation and ballistics. Second, and more importantly, it began to be used for tasks that nobody had previously thought of as arithmetic problems. Photography became digital, which meant turning images into numbers. Music became files. Communication, entertainment and eventually retail were reorganized around the manipulation of numbers. The same logic applies to prediction. As its cost falls, organizations use more prediction where they already relied on it, in demand forecasting, credit scoring and inventory management. They also begin to reframe other problems as prediction problems. Driving is the authors' favorite illustration. For decades, engineers tried to make vehicles drive themselves by writing rules for every situation a car might face, an approach that failed outside controlled environments. The breakthrough came when the task was reframed as a prediction problem: given what the sensors detect, what would a good human driver do next? Once driving became prediction, the rapidly falling cost of prediction could be brought to bear on it. Economics adds a second consequence. When the price of a good falls, the value of its substitutes falls and the value of its complements rises. When coffee becomes cheap, tea sellers suffer and cream and sugar sellers benefit. For prediction, the most obvious substitute is human prediction: the forecasting done by analysts, underwriters, dispatchers and diagnosticians. As machines take over more of this work, the market value of human prediction as such declines. The complements are more interesting. The authors identify data, which is needed to train and run predictive models; action, the ability to do something with a prediction; and above all judgment. The anatomy of a decision To see why judgment matters so much, the authors break a decision into parts. A decision begins with data, which feeds a prediction about the relevant state of the world. The decision-maker then applies judgment, which means assessing the payoff, or the relative value, of each possible outcome given each possible action. Combining prediction and judgment produces an action, which leads to an outcome. The outcome in turn generates new data, which can be used to improve future predictions. An example makes the structure concrete. Suppose you are deciding whether to carry an umbrella. The data are the sky's appearance and the weather report. The prediction is the probability of rain. The judgment is how much you mind getting wet compared with how much you mind carrying an umbrella you do not need. Someone attending a job interview in an expensive suit and someone walking to the gym in shorts might receive exactly the same forecast and reasonably make different choices. The prediction is the same; the judgment differs. This distinction between prediction and judgment is the most important analytical tool the authors provide, and students should practice applying it. Prediction answers the question "what is likely to happen?" Judgment answers the question "how much do I care about each possibility?" A model can tell a doctor that a patient has a 12 percent chance of a particular complication. It cannot, by itself, tell the doctor whether that risk is worth an invasive test, because that depends on the patient's values, the test's risks and costs, and the consequences of missing the complication. Judgment is where values, goals and trade-offs live. In Prediction Machines, the authors turned this anatomy into a planning tool they called the AI Canvas, a simple template that asks managers to specify, for a given decision, the prediction required, the judgment involved, the action taken, the outcome sought, and the data needed for training, input and feedback. The exercise forces managers to break a vague ambition such as "use AI in underwriting" into precise components. In Power and Prediction, as later chapters show, they extend this idea from the single decision to the whole organization. Why judgment becomes more valuable Because prediction and judgment are complements, better prediction makes judgment more valuable. The reason is subtle. When predictions are poor, decision-makers often cannot act on fine distinctions, so careful thought about the relative value of outcomes has little payoff. When predictions become sharp, the question of what to do with them becomes pressing. If a fraud model can identify suspicious transactions with high accuracy, someone must decide how many false alarms are tolerable, how to treat customers who are wrongly flagged, and when to block a payment outright rather than query it. None of these questions arose when fraud could not be detected in time to act. This has direct implications for people. Work that consists mainly of prediction is exposed to substitution. Work that consists of judgment, or of acting on predictions in ways machines cannot, gains importance. But as later chapters explain, the more profound consequence is organizational rather than individual. Once prediction is provided by a machine, it no longer has to be made by the same person who exercises judgment. Prediction and judgment, which have always lived together inside one human mind, can be separated. That separation, which the authors call decoupling, allows judgment to move to different people, and moving judgment moves power. The authors also stress a limitation that students should not overlook. Machine predictions are built on correlations in data. They are excellent at telling you what tends to go with what, but they do not by themselves reveal what causes what. A model may predict that customers who receive a certain discount spend more, but that could be because the firm already targets discounts at big spenders. When a decision requires knowing the effect of an intervention, the authors point out that randomized experiments remain the most reliable way to find out. Mistaking a correlation-based prediction for a causal estimate is one of the most common and expensive errors in applied AI. The limits of cheap prediction: Amazon's thought experiment Prediction Machines included a thought experiment about Amazon that became widely quoted, and Power and Prediction returns to it with an important twist. Amazon currently operates what the authors describe as a shop-then-ship model: customers browse, choose and order, and the company then delivers. Amazon's recommendation engine predicts what each customer might want, but the prediction is only a suggestion on a web page. Now imagine that the recommendation engine becomes accurate enough. At some point, the authors reasoned, it could become profitable to reverse the sequence and ship first: to send customers products the company predicts they will want, and let them keep what they like and return the rest. Amazon did in fact obtain a patent on anticipatory shipping, which gave the idea a certain plausibility. The twist in Power and Prediction is to ask why this has not happened. The answer is not that the predictions are too poor, although they may be. It is that the shift from shop-then-ship to ship-then-shop requires changes far beyond the recommendation engine. The business would need a returns system that could collect unwanted items cheaply and restock them efficiently. The authors note that returns are costly enough that it is often cheaper for retailers to dispose of returned goods than to process them. A ship-then-shop model would also require new packaging, new logistics for pickups, new pricing and new relationships with customers, who would need to accept boxes of goods they did not order. In the authors' terms, a better prediction is a point solution; ship-then-shop is a system solution, and the system does not yet exist. This example is the hinge between the two books. Prediction Machines asked how much a better prediction is worth. Power and Prediction asks what else must change before a better prediction can be worth anything at all. The answer, very often, is a great deal. The value of a prediction depends on the decision it informs, and the value of that decision depends on the other decisions and processes around it. When those surrounding elements are designed on the assumption that the prediction is unavailable, improving the prediction may achieve almost nothing until they are redesigned. Data, feedback and the flywheel Data deserves separate attention because it is the complement most often misunderstood. The authors distinguish three roles that data plays. Training data is the historical record from which a model learns its patterns. Input data is the information fed to the model when it makes a particular prediction, such as the details of a transaction being screened. Feedback data is information about what actually happened after the prediction was made, which allows the model to be checked and improved. Feedback data is where the organizational stakes become visible. A model that predicts which students will struggle with a topic can improve only if the system reports, soon and reliably, whether those students did struggle. A model that predicts which loan applicants will repay learns nothing about the applicants it rejected, because the bank never observes whether they would have repaid. Whether useful feedback exists is rarely a technical question. It depends on how the surrounding processes are designed: how often outcomes are measured, who records them, and whether anyone has an incentive to record them accurately. Feedback also creates a dynamic that matters for competition. A system that makes predictions, observes outcomes and learns from them improves with use. Better predictions attract more users, more users generate more feedback, and more feedback produces better predictions. The authors return to this loop when they discuss power, because it can turn a small early lead into a durable advantage. For now the essential point is that the value of prediction is tied to the flow of data around it, and that flow is a property of the system, not of the model. From decisions to systems The shift in emphasis from individual decisions to systems is what gives the new book its distinctive character. A system, in the authors' usage, is a set of interdependent decisions, rules and procedures that together produce an organization's output. A hospital is a system. So is an insurance company, a school, an airport and a supply chain. In each, many decisions are made by many people, and they are coordinated partly through rules and procedures that fix some choices in advance so that others can rely on them. The authors' central contention is that the large economic gains from AI will come from designing new systems, not from inserting prediction into old ones. This is why they are pessimistic about the short run and optimistic about the long run. In the short run, most organizations will do the easy thing, which is to use AI to improve particular tasks inside systems that remain unchanged. The gains will be real but modest. In the long run, new organizations, and some existing ones, will build systems that assume cheap prediction from the start, and those systems will outcompete the old ones, sometimes dramatically. The intervening period, when the technology is ready but the systems are not, is the Between Times. The next chapter examines why history suggests that period can be long. Key Takeaways · The authors define AI narrowly and usefully as a technology that lowers the cost of prediction, meaning the generation of missing information from available data. · When prediction becomes cheaper, it is used more widely, human prediction loses value as a substitute, and complements such as data, action and especially judgment gain value. · A decision can be analyzed as data, prediction, judgment, action and outcome; prediction says what is likely, judgment says how much each outcome matters. · Machine predictions rest on correlation; decisions about the effect of an intervention still require causal evidence such as randomized experiments. · The Amazon ship-then-shop thought experiment shows that a better prediction often creates little value until the surrounding system, such as returns logistics, is redesigned. Review Questions 1. Explain why the authors treat language translation and fraud detection as prediction problems even though neither involves forecasting the future. 1. Using the economics of complements and substitutes, explain why cheaper prediction may raise the value of some jobs and lower the value of others. 2. Choose a decision in an organization you know and break it into data, prediction, judgment, action and outcome. Which element is currently the bottleneck? 3. Why has Amazon not adopted a ship-then-shop model, according to the authors, and what does this reveal about the limits of prediction alone? 4. Give an example in which relying on a correlation-based prediction to estimate the effect of an intervention would lead a manager astray. Chapter 2: The Between Times and the Lessons of Electrification Agrawal, Gans and Goldfarb organize their book around a historical claim: transformative technologies change the economy in two stages, and there is typically a long, frustrating interval between them. In the first stage, the new technology is used to do old things more cheaply. In the second, people discover how to do new things, and to do old things in entirely new ways, by redesigning the systems in which the technology operates. The authors call the interval between the two stages the Between Times, and they argue that AI in the early 2020s was squarely inside it. The claim is not original to them, and they do not present it as such. It draws on a rich body of economic history, most famously an essay by the Stanford economist Paul David, and on a long-running debate about why new technologies often fail to show up in productivity statistics. This chapter explains that history, because the authors' distinctions among types of AI solution are modeled directly on it. The productivity paradox In 1987, the Nobel laureate Robert Solow wrote a book review in the New York Times containing a line that has been quoted ever since: you can see the computer age everywhere but in the productivity statistics. By the late 1980s, American businesses had spent heavily on computers for more than two decades. Mainframes ran payrolls, minicomputers ran factories, and personal computers were appearing on office desks. Yet measured labor productivity growth in the United States had slowed markedly since the early 1970s. If computers were so powerful, where were the gains? Economists offered several explanations. Perhaps the gains were real but mismeasured, because official statistics struggled to capture improvements in quality and convenience. Perhaps computers were simply not yet a large enough share of the capital stock to move the aggregate numbers. Perhaps firms were wasting money on computers they did not know how to use. Each explanation had some truth. But the most influential answer came from history. Paul David and the dynamo In 1990, Paul David published a short paper in the American Economic Review titled "The Dynamo and the Computer: An Historical Perspective on the Modern Productivity Paradox". His argument was that the puzzle Solow identified was not new. Electricity, David showed, had followed a similar path a century earlier. Thomas Edison's first commercial central power stations opened in 1882. Yet the productivity gains from electrification in American manufacturing did not appear clearly until the 1920s, roughly four decades later. For most of that period, observers could see electricity everywhere, in lighting, in streetcars, in new factories, and yet the broad economic payoff seemed stubbornly delayed. David's explanation centered on how factories were designed. In the steam age, a factory was built around a single large engine. The engine turned a main shaft, called a line shaft, which ran the length of the building. Belts and pulleys carried power from that shaft to each machine. This arrangement imposed a rigid logic on the whole building. Because power losses from friction grew with distance, machines had to be clustered close to the shaft. Factories were often built several stories high, with shafts on each floor, to keep machines near the engine. The layout was dictated by the needs of power transmission, not by the flow of materials or the convenience of workers. When the engine or the main shaft stopped, the whole factory stopped with it. When electric motors first arrived, many factory owners did the obvious thing. They replaced the steam engine with a large electric motor and kept everything else the same: the same shafts, the same belts, the same multistory buildings. This saved some money on fuel, but it changed little else. The historian of technology Warren Devine, whose work David drew upon, traced the gradual transition that followed. Some factories adopted what engineers called group drive, in which several smaller motors each powered a shorter shaft serving a group of machines. Eventually many adopted unit drive, in which each machine had its own electric motor. Unit drive was the breakthrough, but not mainly because it saved energy. Once each machine had its own motor, the tyranny of the line shaft disappeared. Machines no longer had to be arranged around a power source. They could be placed wherever the production process required. Factories could be built on a single story, which was cheaper to construct and easier to move materials through. Without the forest of overhead shafts and belts, buildings could admit more natural light, and overhead cranes could run freely. Production could be organized around the sequence of work. A breakdown in one machine no longer halted the whole plant. And a factory could be expanded simply by adding machines, without redesigning its power system. These gains required a different factory, and different factories required new investment, new management practices, new skills among workers and engineers, and often new firms. Existing factories with decades of useful life left in their buildings had little reason to tear them down. The full transition therefore waited on the replacement of the old capital stock and on a generation of managers who thought about production in a new way. David estimated that electric motors did not account for around half of mechanical drive capacity in American manufacturing until about 1919. Only then did the productivity surge arrive. Table 1 summarizes the stages of this transition as they are commonly described in the economic history literature on which the authors draw. Table 1. Stages of factory electrification. Stage Power arrangement Factory layout Main source of value Steam era One engine, one line shaft Multistory, built around shaft Mechanized production Direct substitution One large electric motor replaces engine Unchanged Lower fuel and power cost Group drive Several motors, shorter shafts Partly freed from main shaft Less friction, some flexibility Unit drive and redesign A motor in each machine Single story, arranged by workflow Throughput, reliability, flexibility The three entrepreneurs Agrawal, Gans and Goldfarb turn this history into a parable. They imagine three entrepreneurs at the dawn of electrification, each offering a different kind of business built on the new technology. The parable is a teaching device rather than a historical account of three real people, and it should be read as such. The first entrepreneur offers the simplest proposition: electricity as a cheaper source of power. Customers can replace their steam engines with electric motors and keep their factories as they are. This entrepreneur's value proposition is cost saving, and it is easy to sell because it requires little change on the customer's part. It is also limited, because the savings are bounded by the cost of the power being replaced, and power was only one part of a factory's costs. The second entrepreneur offers something more ambitious: machines with their own electric motors built in. This removes the dependence of each machine on a central shaft. The value proposition now includes reliability, since one failure no longer stops everything, and some flexibility. But the customer's factory, still organized around the old logic, cannot fully use that flexibility. The third entrepreneur sells neither power nor machines, but a new kind of factory. By starting from the question of how production should be organized once power can be delivered anywhere, this entrepreneur captures the full value of electricity: the single-story building, the workflow layout, the independence of each station. This proposition is by far the most valuable. It is also the hardest to sell, because it requires the customer to abandon a working factory and much of the knowledge and routine that went with it. The authors label these three offerings the point solution, the application solution and the system solution. Each is a real business opportunity, and each succeeded in its time. The lesson is not that point solutions are worthless. It is that the largest value, and the largest disruption, came from the system solution, and it came late. The authors add an observation that is easy to miss. What made the system solution possible was decoupling. Under steam, the location of a machine was tied to the location of the power source. Electricity broke that link. The machine and the power source became independent, and that independence opened up the design space for the whole factory. The authors argue that AI has a similar decoupling at its heart: it separates prediction from judgment, which were previously bound together in the same human mind. Later chapters develop this parallel in detail. AI in the Between Times The authors apply the electrification pattern directly to AI. By the time they wrote, machine learning had advanced remarkably, yet the economic effects looked small. They note that productivity growth in advanced economies had been weak for years despite the technology's evident capability. In their reading, this is not evidence that AI is overhyped. It is evidence that the economy is in the first stage, where AI is used as a point solution inside systems designed for a world without cheap prediction. This interpretation is consistent with a body of work by other economists. In a paper published in the American Economic Journal: Macroeconomics in 2021, Erik Brynjolfsson, Daniel Rock and Chad Syverson proposed what they called the productivity J-curve. Their argument is that general-purpose technologies, those with wide applications across the economy, require large investments in complementary intangible assets: new business processes, new skills, new organizational designs and new data. These investments are costly and largely unmeasured in official statistics. In the early years, firms spend resources building them, so measured productivity can actually appear to fall. Later, as the intangible investments start to pay off, measured productivity rises faster than the underlying inputs would suggest. The shape of the curve over time resembles the letter J. The J-curve and the Between Times are two descriptions of the same phenomenon, one from the viewpoint of national statistics and the other from the viewpoint of an individual business. The J-curve says the economy must invest in intangibles before it harvests the gains. The Between Times says those intangibles are, above all, redesigned systems. The second episode: computing in the 1990s Solow's paradox did not last forever. In the second half of the 1990s, American labor productivity growth accelerated sharply, and a large share of the acceleration was traced to industries that produced or intensively used information technology. The pattern mattered as much as the timing. The gains were concentrated in firms that had changed how they worked, not merely what equipment they bought. Research by Erik Brynjolfsson and Lorin Hitt, among others, found that the returns to computer investment varied enormously across firms, and that the firms earning high returns tended to combine computers with a cluster of organizational practices: more decentralized decision-making, greater use of teams, more investment in training, and redesigned processes. Computers dropped into an unchanged hierarchy produced modest gains. Computers combined with a reorganized firm produced large ones. The complementary organizational investments were expensive, took years, and did not appear on any balance sheet. Managers of the period had their own language for the same insight. In 1990, Michael Hammer published an article in the Harvard Business Review whose title urged managers not to automate but to obliterate: rather than using computers to speed up existing paperwork, firms should ask whether the paperwork needed to exist at all. The business process reengineering movement that followed had a mixed record, partly because it was often used as a label for layoffs. But its central idea anticipated the authors' distinction. Automating a step in an existing process is a point solution. Redesigning the process around what the technology makes possible is a system solution. The computing episode is useful to students for a second reason. It shows that the lag between capability and payoff is not a curiosity of the distant past, and that it can end. Retailers such as Walmart built supply chains around the real-time sharing of sales data with suppliers, and in doing so reorganized relationships across an entire industry. The value came not from any one computer but from a system that assumed cheap information processing at every link. That is the kind of change the authors expect cheap prediction to bring. What the analogy does and does not show Students should use the electrification analogy carefully. It is persuasive, but it is an analogy, and analogies have limits. The guide offers three observations for critical discussion. First, the time scale may differ. Electricity required heavy physical infrastructure: generating stations, transmission lines and new buildings. Much of AI runs on existing computing infrastructure and is delivered as software, which can spread far faster. On the other hand, the organizational and regulatory changes AI requires may be just as slow as the physical ones electricity required, because they involve people, institutions and law. The authors' emphasis on rules and power suggests they believe the organizational constraint is binding. Second, the analogy can be read too deterministically. Electrification eventually transformed manufacturing, but the path was not inevitable, and many firms that adopted point solutions did perfectly well for decades. The claim that system solutions will eventually dominate is a strong prediction about AI, not a proven fact. It depends on the size of the gains from redesign, which differ by industry. Third, the analogy highlights a practical point that is often forgotten in technology strategy: the organizations that captured the value of electrification were frequently not the ones that had been most successful under steam. New factories were often built by new firms or in new places. The authors make this link explicitly when they turn to disruption and power. For a student evaluating an incumbent firm's AI strategy, the history poses an uncomfortable question: is the firm building the new factory, or merely buying a bigger motor? Key Takeaways · Robert Solow's 1987 observation that computers were visible everywhere except in productivity statistics framed a long debate about delayed technology payoffs. · Paul David's 1990 paper showed that electrification followed a similar path, with productivity gains arriving roughly four decades after the first central power stations, once factories were redesigned. · The move from line-shaft steam power through group drive to unit drive freed factory layout from power transmission, enabling single-story plants organized by workflow. · The authors' parable of three entrepreneurs maps electrification onto point, application and system solutions, and identifies decoupling as the source of system-level value. · The productivity J-curve of Brynjolfsson, Rock and Syverson offers a complementary macroeconomic explanation: intangible investments in new systems precede measured gains. Review Questions 1. Describe how the line shaft shaped the design of steam-powered factories, and explain why replacing the steam engine with an electric motor changed so little. 1. Why was unit drive valuable for reasons other than energy efficiency? 2. In the authors' parable, why is the third entrepreneur's offering both the most valuable and the hardest to sell? 3. Explain the productivity J-curve and how it relates to the idea of the Between Times. 4. Identify one way in which the electrification analogy may overstate and one way it may understate the time AI will take to transform an industry you know. Chapter 3: Point, Application and System Solutions The distinction among point, application and system solutions is the analytical core of Power and Prediction. It is also the part of the book most often summarized loosely, as if the three categories were simply small, medium and large AI projects. They are not. The categories are defined by a specific criterion, and a student who grasps that criterion can classify almost any AI initiative and predict how hard it will be to implement and how much value it can create. This chapter sets out the definitions, explains the criterion that separates them, discusses why task-level thinking pushes organizations toward the first category, and then follows one hypothetical industry through all three levels in a worked example. Definitions and the independence criterion Agrawal, Gans and Goldfarb define the three categories in terms of decisions and procedures. A point solution uses AI to improve an existing decision, and it can be adopted on its own, without changing anything else in the system. An application solution uses AI to enable a new decision, or a new way of making a decision, and it too can be adopted on its own. A system solution uses AI to improve existing decisions or enable new ones, but its value depends on changing other decisions and procedures in the organization. The criterion that separates the first two from the third is independence. Point and application solutions can be dropped into an existing organization. The rest of the organization does not have to know or care that they are there. A system solution cannot work that way. The value of the improved decision is tied to changes elsewhere, and unless those changes are made, the value is not realized. The authors return to this point repeatedly: when the benefit of a better decision depends on other substantive changes, what is needed is a system solution. The difference between point and application solutions is subtler and less consequential. A point solution does the same thing better: a more accurate forecast feeding the same ordering decision. An application solution does something that was not being done before: a decision that was previously not made at all, or was made in a fundamentally different way. In the electrification parable, the point solution was cheaper power for an unchanged factory, and the application solution was a machine with its own motor, which allowed a new operating mode (running machines independently) without requiring the factory to be rebuilt. Both could be bought and installed without rethinking the whole enterprise. Table 2 sets out the three categories side by side. Table 2. The three types of AI solution. Type What AI does Depends on other changes? Typical value Ease of adoption Point solution Improves an existing decision No Incremental, often cost saving High Application solution Enables a new decision or method No Moderate, new capability Moderate System solution Improves or creates decisions within redesigned processes Yes Potentially transformative Low, often resisted Two features of the table deserve emphasis. First, the value column is a tendency, not a rule. Some point solutions are very valuable, especially where the decision they improve is large and frequent. Second, the adoption column captures the central tension of the book. The solutions easiest to adopt are the least likely to transform an organization, and the most transformative are the hardest to adopt. That inverse relationship is why the Between Times exist. Why organizations default to point solutions The authors argue that most organizations approach AI through what they call task-level thinking. The standard method, promoted by consultants and business schools, is to break jobs into tasks, identify the tasks where machines can outperform humans, and deploy AI there. This method has virtues. It is concrete, measurable and relatively low risk. It fits the budgeting and project management practices of large firms. It produces business cases that finance departments can evaluate. But task-level thinking tends to produce point solutions, because it takes the existing arrangement of tasks as given. It asks how to do a task better, not whether the task should exist or how the tasks should be arranged. It also tends to frame AI's value as labor cost reduction, since the natural metric for a task done by a machine is the wage of the person who used to do it. The authors contrast this with system-level thinking, which asks how the organization would be designed if cheap, accurate prediction were available from the start. They point to firms such as Amazon, Google, Netflix, Meta and Apple as examples of organizations whose core businesses were built around prediction at the system level, and whose value came from creating things customers wanted rather than merely cutting costs. There are further reasons for the default. Point solutions can be bought from vendors, while system solutions must usually be built. Point solutions fit within a single department's budget and authority, while system solutions cross departmental lines. Point solutions threaten few people, while system solutions, as later chapters explain, reallocate power. And the benefits of point solutions appear quickly, while those of system solutions may take years and require worse performance in the short run. The authors give a telling example from health care. AI has been applied to diagnosis, surgical assistance, patient monitoring, treatment personalization and drug discovery, and many individual applications perform impressively. Yet the authors observe that the benefits to health care as a whole have been modest. Regulation explains part of this. But the larger explanation, in their view, is that these tools have been deployed as point solutions inside a system whose training, delivery, payment, privacy and liability arrangements were designed for a world without them. A diagnostic model that is superhuman at reading scans does not change much if the workflow, the reimbursement rules and the division of professional responsibility remain as they were. Recognizing a system solution How can a manager or analyst tell whether a proposed AI initiative is a point or a system solution? The independence criterion suggests a practical test. Take the initiative and ask what else must change for it to deliver its promised value. List the other decisions, rules, roles, contracts, metrics and technical systems that would have to be altered. If the list is empty or trivial, the initiative is a point or application solution. If the list is long, and especially if it crosses departmental or organizational boundaries, the initiative is a system solution, whatever its sponsors call it. This test often reveals that an initiative presented as a simple tool is in fact a system change in disguise. A hospital that buys a model predicting which patients are likely to deteriorate may assume it is buying a point solution. But if the model's alerts are to be useful, someone must respond to them, which means staffing rules, escalation protocols and perhaps the allocation of beds must change. If none of these change, the alerts may be ignored, and the model will deliver nothing. The reverse error is also possible. Some initiatives described as transformational are, on inspection, point solutions with ambitious marketing. The test also clarifies why AI projects so often stall between pilot and production. A pilot is typically run in isolation, where the surrounding system is held constant or informally adjusted by an enthusiastic team. The pilot succeeds. Scaling it then requires the formal changes the pilot avoided, and those changes meet resistance. The project has discovered that it was a system solution all along. Point solutions as stepping stones and as traps It would be a misreading of the authors to conclude that point solutions are a mistake. They serve several useful purposes. They generate returns while larger changes are considered. They build the data infrastructure, technical skills and organizational familiarity that any system solution will later need. And they reveal where the binding constraints lie: a forecasting tool that improves accuracy but not results shows, by its very disappointment, which rule or procedure is holding the value back. The danger is that point solutions can also become traps. Each one is optimized for the existing system, and each creates vendors, budgets, teams and performance metrics that have a stake in that system's survival. An organization that accumulates dozens of point solutions may find that its AI portfolio has made system change harder, not easier, because every tool now assumes the old arrangement of decisions. The authors' historical parallel is apt. A steam-age factory that installed a large electric motor on its line shaft had spent money on electricity, but it had also reinforced the shaft. The guide suggests a simple discipline for students evaluating an AI portfolio. For each point solution, ask two questions. Does it produce data, skills or evidence that a future system redesign would use? And does it lock in any rule, contract or process that such a redesign would need to remove? A point solution that scores well on the first and poorly on the second is a stepping stone. One that scores the other way round is a trap, however attractive its immediate return. A worked example: one grocery chain, three solutions To make the distinctions concrete, consider a hypothetical regional grocery chain with about a hundred stores. The chain's biggest operational problem is fresh food. Produce, dairy, bakery and prepared meals spoil quickly. Ordering too much means waste; ordering too little means empty shelves and lost customers. Today, each store manager places orders twice a week, using a spreadsheet of past sales and personal experience, within a fixed delivery schedule set by the central distribution center. Prices are set centrally and change weekly with the promotional flyer. Markdowns on items nearing their date are applied by staff at a fixed time each evening according to a simple rule. This example is invented for teaching purposes, but each element is typical of the industry. The point solution. The chain buys a machine learning demand forecasting tool. It ingests sales history, weather forecasts, local events and promotional plans, and produces a forecast for each product in each store. Store managers see the forecast in their ordering screen and use it in place of their spreadsheets. Nothing else changes: orders are still placed twice a week, deliveries still arrive on the same schedule, prices and markdowns still follow the same rules. The forecast is more accurate, so waste and stockouts fall somewhat. The benefit is real and measurable, and the project is easy to approve because it replaces a spreadsheet and changes no one's role. But the benefit is capped. A better forecast cannot help much when orders can only be placed twice a week, because demand for fresh items shifts daily. The application solution. The chain goes further and adds an automated markdown engine. Instead of staff applying a fixed discount at a fixed hour, the system predicts, for each item, how likely it is to sell before it expires at various prices, and recommends dynamic markdowns through electronic shelf labels throughout the day. This is a new way of making a decision that was previously governed by a rule. It can be adopted independently: ordering, delivery and central pricing are untouched. The value is larger than the forecast alone, because it attacks waste at a second point. But it still operates inside a system whose basic rhythms were designed for a world of poor prediction. The system solution. Finally, imagine the chain redesigns its fresh food operation around cheap prediction. Forecasts are updated continuously and drive daily, rather than twice-weekly, replenishment. That requires the distribution center to move from fixed delivery routes to dynamic ones, which changes the work of warehouse staff and drivers. Supplier contracts are renegotiated so that farms and bakeries receive rolling forecasts and commit to flexible volumes. Central pricing is replaced by a system that coordinates markdowns, promotions and orders together, so the chain stops ordering products it will later have to discount. Store staffing schedules are set by predicted demand. The store manager's role changes from placing orders to handling exceptions, local merchandising and customer service. Surplus food predicted days in advance is routed to discount channels or food banks rather than discovered at closing time. Every one of these changes depends on the others. Daily replenishment without dynamic routing is impossible. Flexible supplier contracts without reliable rolling forecasts are unacceptable to suppliers. Coordinated pricing requires that the merchandising team give up its weekly authority over promotions. The value of the system is far larger than the sum of the forecasting and markdown tools, but no single part can deliver it alone. That interdependence is exactly what defines a system solution. Table 3 compares the three versions of the project. Table 3. Three solutions in a hypothetical grocery chain. Dimension Point solution Application solution System solution AI function Demand forecast Dynamic markdowns Forecast-driven operations Other changes needed None None Logistics, contracts, pricing, roles Who is affected Store managers Store staff Warehouse, suppliers, merchandising, stores Likely resistance Low Low to moderate High Value ceiling Limited by ordering rules Limited by fixed rhythms Set by redesigned system Several lessons follow. First, the system solution requires changes in departments that gain little from the forecasting tool itself: the distribution center, the merchandising team and the procurement function. Each has its own metrics, which the redesign may disrupt. Second, the system solution changes who holds authority. The merchandising team that once controlled weekly promotions loses power to a coordinated system, and store managers lose discretion over orders. Third, a newcomer building a fresh food business from scratch, with no existing contracts or routes, could design the system solution far more easily than the incumbent chain could. This is precisely the pattern the authors expect: point solutions favor incumbents, and system solutions open the door to disruption. Key Takeaways · Point solutions improve existing decisions and application solutions enable new ones; both can be adopted without changing the rest of the organization. · System solutions derive their value from changes to other, interdependent decisions and procedures, and the criterion that defines them is lack of independence. · Task-level thinking, the dominant approach to AI adoption, tends to produce point solutions focused on labor cost reduction. · A practical test for any initiative is to list everything else that must change for it to deliver value; a long, cross-departmental list signals a system solution. · Many pilots stall at scale because they were system solutions presented as point solutions. Review Questions 1. State the criterion that distinguishes system solutions from point and application solutions, and explain why it matters more than project size. 1. Why does task-level thinking tend to frame the value of AI as cost reduction? 2. Explain why the authors believe AI's benefits to health care have been limited, and identify the elements of the health care system that would need to change. 3. In the grocery example, which departments would resist the system solution, and why? 4. Choose an AI tool your organization or university uses. Apply the "what else must change" test and classify it as a point, application or system solution. Why might a new entrant find it easier than an incumbent to implement the system solution in the grocery example? Hashtags: #TheAITransition #PowerAndPrediction #ArtificialIntelligenceEconomics #CheapPrediction #PredictionMachines #PredictionAndJudgment #BetweenTimes #PointSolutions #ApplicationSolutions #SystemSolutions #EnterpriseAIAdoption #SystemRedesign #AITransformation #Decoupling #OrganizationalPower #JudgmentReallocation #ProductivityParadox #ProductivityJCurve #GeneralPurposeTechnology #AIComplementarities #TaskLevelThinking #AISystemsDiscovery #EnterpriseAI #AIAdoptionStrategy #FutureOfAITransformation

Download the Book (PDF): Introduction In the floor cavities of a sandstone building on Macquarie Street in Sydney, archaeologists working in the early 1980s found the small possessions of people who had nowhere private to keep them. The Hyde Park Barracks had been built between 1817 and 1819 to house male convicts assigned to government labour gangs, and later sheltered orphaned Irish girls, newly arrived immigrant women and destitute elderly women. Over nearly seven decades, objects slipped or were pushed between the floorboards: scraps of cloth, buttons, clay pipes, pins, playing pieces, pages of letters. Rats carried more of it into their nests. What survived is now counted among the largest collections of nineteenth-century institutional material anywhere. None of it was ever meant to be seen. Much of it was deliberately hidden. That contrast, between a building designed to organise and discipline human labour and the tiny things its inmates hid inside it, is the subject of this book. Every regime that has forced people to work has left material traces of two kinds. There are the traces of the system itself: the walls, fences, watchtowers, leg irons, identity badges, ration bowls, ballast bars and quarry faces through which labour was extracted and bodies were controlled. And there are the traces of the people inside it: the beads worn against regulations, the engraved coins passed to a lover at a prison gate, the scratched name on a barrack wall, the wedding ring kept hidden until the last moment. Both kinds of evidence often lie in the same soil, sometimes in the same layer. Archaeology can recover both. What it chooses to do with them is the question that matters. The argument The book's argument is simple to state and demanding to practise. The material remains of forced labour are double-voiced: they record the regime's design for extracting work while erasing personhood, and they record the persistence of the persons it tried to erase. Every stage of archaeological practice, from the first survey of a landscape to the final label in a museum case, can amplify one voice or the other. Good practice is not neutral between them. It is judged by whether it restores the coerced person to view without reproducing, in a new medium, the logic of the system that coerced them. This is not a sentimental claim. It has practical consequences for which survey methods are chosen, how human remains are approached, what gets conserved and what is allowed to decay, how objects are arranged in a gallery, and who is consulted before any of it happens. A trench cut carelessly through a mass grave repeats a desecration. A museum wall of anonymous shoes, however moving, can turn murdered individuals into a quantity again. A heritage listing that celebrates a mine's engineering and omits the men conscripted to dig it re-performs their disappearance. In each case the archaeologist, conservator or curator is not only describing the past but acting within its unfinished consequences. What the book covers The field examined here is broad, so the book works through a deliberately mixed set of cases rather than a single region. Three families of coercive regime recur. The first is the penal colony and the convict labour system, above all Britain's transportation of convicts to Australia between 1788 and 1868, but also France's penal settlements in Guiana and New Caledonia and colonial prisons such as the Cellular Jail in the Andaman Islands. The second is chattel slavery in the Atlantic world, whose plantations, burial grounds, quarters and shipwrecks have been among the most intensively studied sites in historical archaeology since the 1970s. The third is the forced labour of the twentieth century's wars and dictatorships: the camps of Nazi Germany and occupied Europe, including those on the British Channel Island of Alderney; Japan's wartime mobilisation of Korean, Chinese and Allied prisoner labour; and the Soviet Gulag. These systems differed profoundly. Their legal foundations, their economic logic, the degree to which they intended their workers to die, and the kind of afterlife they have had in public memory are all distinct, and the book tries never to collapse them into a single category of generic suffering. A convict transported for theft who later received a ticket of leave and farmed his own land was not in the position of an enslaved child born into hereditary bondage, and neither was in the position of a Soviet prisoner of war worked to death on a fortification in the Channel Islands. What they share, and what justifies treating them together, is a material problem. In each case, the people whose labour built the site were denied control over their own record. The paperwork that survives was, overwhelmingly, written by those who owned, sentenced or guarded them. The physical remains are often the only evidence produced partly on the workers' own terms. How the book is organised The first chapter sets out what coercion leaves behind in general terms, and why the archive alone is not enough. The second turns to landscapes: how the layout of a camp, a plantation or a penal station encodes a theory of control, and how non-invasive survey methods, from wartime aerial photographs to airborne laser scanning and ground-penetrating radar, have transformed the ability to read those layouts without disturbing what lies beneath. The third chapter addresses excavation itself, especially where human remains are present, and the forensic, legal and religious constraints that govern digging at sites of atrocity. The next two chapters divide the finds. Chapter 4 examines the objects of the regime: restraints, marks of ownership and identification, the architecture and tools of work, and the equipment of feeding and punishment. Chapter 5 examines the objects of the coerced: the things people made, hid, adapted, traded and treasured, and the difficult question of how far such objects can be read as resistance. Chapter 6 turns to conservation, the least discussed and perhaps most consequential stage, where decisions about whether to stabilise, restore or let decay shape what future generations can know. Chapter 7 addresses display, and the ethics of placing instruments of oppression and possessions of the oppressed behind glass. Chapter 8 widens the lens to ask who has authority over these sites and their interpretation: descendant communities, survivor organisations, states, international heritage bodies. The conclusion draws out what follows from all of this for practitioners and for anyone who visits such places. A note on language and limits Words matter in this field, and they are contested. The book uses "enslaved people" rather than "slaves" where it can, because the phrase keeps the person in front of the condition, while recognising that historical sources and some descendant communities use other terms. It uses "forced labour" as an umbrella for work extracted under threat of punishment from people who had not freely offered it, following the broad sense of the International Labour Organization's definition, and it uses more specific terms, such as convict labour or chattel slavery, where the distinction matters. It refers to Nazi camps by function as far as possible, because a forced labour camp at an aircraft factory, a concentration camp and an extermination centre were different institutions even when they were linked. The book is also honest about the limits of its evidence. Archaeology does not replace survivor testimony, descendant memory or documentary history; it sits alongside them, and its findings are often ambiguous. A bead in a quarter may be a charm, an ornament, a trade good or something dropped by a visitor. A worn vertebra may reflect heavy labour or simply age. Much of this book is concerned with how practitioners handle that uncertainty responsibly: saying what the evidence supports, refusing to inflate it into a story it cannot carry, and remembering that the people behind the objects were more complicated than any interpretation of them. The things in the floor at Hyde Park Barracks were hidden because their owners had no right to privacy. Recovering them, and deciding how to show them, is a small restitution of that right. It is also a responsibility. The chapters that follow are an attempt to describe what that responsibility asks. Chapter 1: What Coercion Leaves Behind Every system of forced labour is, among other things, a building project. People who are compelled to work must be moved, housed, fed at the lowest tolerable cost, guarded, counted, and prevented from leaving. They must be given tools, or denied them. Their work must be organised in space: a cane field divided into pieces, a quarry face into allotted sections, a factory floor into stations, a railway embankment into daily lengths. And the whole arrangement must be made visible to those who run it and, as far as possible, invisible or illegible to everyone else. All of this leaves marks in the ground. The archaeology of forced labour begins from the recognition that these marks are evidence of a particular kind. They are not simply the residue of daily life that any settlement produces. They are the physical form of a relationship of domination, and they carry the imprint of both parties to it. A fence line records the intention to confine; a gap dug beneath it records an attempt to leave. A standardised ration bowl records a bureaucracy's calculation of the minimum needed to keep a worker productive; a name scratched into its base records someone insisting that the bowl was theirs. Why the archive is not enough Coercive regimes are, almost without exception, prolific producers of documents. The British convict system generated some of the most detailed records of individual bodies ever assembled before the twentieth century: indents listing each transported convict's height, complexion, eye colour, scars and tattoos, trade, offence and sentence; conduct registers recording every infraction and punishment; musters and assignment lists. Slaveholders kept plantation ledgers, inventories that listed people alongside livestock and tools, and probate records that valued them. The Nazi state and its industrial partners kept labour allocation files, camp registers, transport lists and invoices for prisoner labour hired out to firms. Japan's wartime mobilisation produced recruitment quotas and company payroll records. The Gulag administration kept statistics on output and mortality. These records are indispensable, and much of the best archaeology of forced labour is done by researchers who have mastered them. But they share three structural problems that no amount of archival skill can fully overcome. The first is authorship. The documents were written by the people who ran the system, for purposes of control, profit, or legal cover. They record what the administration needed to know and nothing else. A convict indent tells us the precise length of a man's scar and nothing about how he got it or what he thought about his sentence. A plantation inventory tells us an enslaved woman's assigned value and nothing about the songs she sang or the beads she wore. When enslaved or imprisoned people do appear as active subjects in these records, it is most often as the objects of punishment, which gives a distorted picture of their lives as a sequence of infractions. The second problem is silence by design. Some regimes deliberately avoided recording what they did. The operators of the Operation Reinhard extermination centres at Bełżec, Sobibor and Treblinka in occupied Poland, where small numbers of Jewish prisoners were forced to labour in the killing process itself, dismantled the camps in 1943 and planted over their sites. The German occupiers of Alderney appear to have destroyed many records before the island's liberation in 1945. Plantation records rarely described the internal organisation of the quarters, because the owners did not care how enslaved people arranged their own space so long as they appeared for work. Where there is no document, the ground is the record. The third problem is survival. Archives burn, are pulped, are lost in moves, or are held in countries whose governments restrict access. Records of Soviet forced labour were closed for decades and have become harder to consult again in recent years. Records of wartime Korean labour in Japanese mines and factories are scattered across company, municipal and national archives, many incomplete. Archaeology does not solve these problems, but it approaches them from a different angle. Material remains are not free of bias: what survives in the ground depends on soil chemistry, on later disturbance, and on who had the means to own durable things. But they were not produced primarily as a record. They are the unintended by-products of lives lived and work done, and they were shaped by the coerced as well as the coercers. That makes them a check on the archive and, sometimes, the only surviving voice of the people the archive reduced to entries. Three families of regime It helps to be precise about the systems under discussion, because their material signatures differ as much as their histories. The differences can be summarised along a few lines: the legal basis on which labour was compelled, how the work was organised, what the typical material evidence looks like, and where it has been studied, as Table 1 sets out. Table 1. Five coercive labour systems compared by legal basis, organisation and typical material evidence. System Legal basis of coercion Organisation of labour Typical material evidence Examples of studied sites British convict system (1788–1868) Criminal sentence; fixed term Government gangs, private assignment, probation stations Barracks, penitentiaries, road and bridge works, irons, marked equipment Hyde Park Barracks; Port Arthur; Cascades and Ross Female Factories; Darlington, Maria Island Atlantic chattel slavery (16th–19th c.) Hereditary property in persons Gang and task systems on plantations; urban and domestic labour Quarters, subfloor pits, burial grounds, restraints, ship ballast, local ceramics Newton Plantation, Barbados; New York African Burial Ground; Monticello; São José wreck Nazi forced and slave labour (1933–1945) Racial and political decree; wartime occupation Camps attached to factories, construction sites and SS enterprises Barrack footprints, fences, personal effects, industrial debris, mass graves Tempelhof, Berlin; Treblinka I; Alderney camps; Mittelbau-Dora Japanese war labour (1939–1945) Colonial conscription and recruitment; prisoner-of-war status Mines, factories, railway construction Mine workings, dormitories, railway cuttings, camp sites Burma–Thailand Railway; Hashima; Sado mines Soviet Gulag (1930–1950s) Administrative and criminal sentence Camp complexes tied to extraction and construction Camp perimeters, barracks, mines, burial areas Perm-36; camps of Kolyma and the far north The table flattens a great deal. Convict transportation, for example, was not one system but several, changing sharply after the introduction of the probation system in Van Diemen's Land in the 1840s. Chattel slavery in Virginia's tobacco economy differed materially from slavery on Barbadian sugar estates or in Brazilian cities. Nazi Germany operated tens of thousands of camps of different kinds, from small barracks housing foreign civilian workers near a village factory to vast complexes such as Auschwitz, where labour and murder were interwoven. What the comparison does show is that each system has a recognisable material grammar, and that the archaeologist's first task is to learn it. The two kinds of trace Within each system, the material record can be sorted, roughly, into two categories. The distinction is imperfect, but it organises much of what follows. The first category is the apparatus of the regime: everything built, issued or imposed in order to extract labour and maintain control. It includes architecture and site layout, perimeter and surveillance features, tools and equipment of work, restraints and instruments of punishment, identification marks and badges, and the standardised material of provisioning, such as rations and uniforms. These objects are often durable, because they were made of stone, brick and iron by institutions with capital. They are also often well documented, because the institution ordered and paid for them. Their archaeological value lies in showing how the system actually operated as against how its regulations said it should: whether the prescribed separation between groups of prisoners was maintained, whether rations were as described, whether a fence was where the plan says it was. The second category is the material life of the coerced: what people made, acquired, kept, modified, hid and discarded for their own purposes. It includes personal ornaments, religious and protective objects, food remains that go beyond the ration, tools and objects improvised from scrap, items traded illicitly, marks and inscriptions, and the arrangement of space within quarters. These objects tend to be small, fragile and undocumented. Their value lies precisely in their independence from the regime's record-keeping. The categories overlap constantly. An issued bowl becomes personal when someone scratches their initials into it. A regulation button reappears in a burial as an ornament. A tool of labour is turned into a weapon of escape. A tin can issued as a container becomes a cooking pot, a lamp, a cup. Much of the most interesting interpretive work in the field lies at this seam, where people took the material of their confinement and bent it to their own use. Why this matters now The archaeology of forced labour has grown from a marginal specialism into a substantial field over the past half-century, and several forces have driven that growth. In the Americas, the civil rights movement and the rise of social history in the 1960s and 1970s pushed archaeologists to study the sites of enslaved people rather than only the mansions of their owners. Charles Fairbanks's excavation of slave cabins at Kingsley Plantation in Florida in 1968, and his later work on the Georgia coast, are often cited as a starting point, and Jerome Handler and Frederick Lange's study of the Newton Plantation cemetery in Barbados, published in 1978, established what careful archaeology of enslaved communities could reveal. The rediscovery of the African Burial Ground in Lower Manhattan in 1991, during construction of a federal office building, became a turning point, because the descendant community's demand to control the research reshaped how such work is done. In Europe, the archaeology of the Nazi period was long resisted, partly from a sense that the period was sufficiently documented, partly from reluctance to disturb sites of mass death, and partly because archaeology in Germany had itself been compromised by the regime. That resistance began to give way from the 1990s. Andrzej Kola's survey of Bełżec in the late 1990s, the Sobibor excavations begun in 2007 by Yoram Haimi and Wojciech Mazurek, Caroline Sturdy Colls's work at Treblinka from 2007, and the excavation of forced labour camps at Tempelhof airfield in Berlin by Reinhard Bernbeck, Susan Pollock and their colleagues from 2012 demonstrated that archaeology could add substantially to what was known, even about the most studied events of the twentieth century. In Australia, the conservation movement of the 1970s and 1980s and the approach of the bicentenary of British settlement in 1988 brought convict sites into the heritage system. Large excavations at the Hyde Park Barracks and elsewhere were followed by the inscription of eleven Australian Convict Sites on the UNESCO World Heritage List in 2010. A further force is time itself. The last survivors of the Second World War's forced labour systems are dying. Direct descendants of people enslaved in the United States are now several generations removed from emancipation. As living memory fades, material evidence takes on a heavier burden, and sites become the main points of contact between the public and these histories. This is also why the sites are increasingly contested, and why decisions about excavation, conservation and display have become political in ways that earlier practitioners could sometimes avoid. The final force is denial. Every one of the regimes discussed here has its deniers or minimisers: those who claim convict life was a fair bargain, that slavery was benign, that the Holocaust did not happen or was exaggerated, that wartime labour in Japan's mines was voluntary employment. Physical evidence cannot by itself refute an ideologically committed denier, and archaeologists have learned that their findings are sometimes twisted by the people they might expect to rebut. But evidence in the ground, carefully recorded and publicly presented, sets limits on what can credibly be claimed. The foundations of the gas chambers at Sobibor, located in 2014, are not a matter of interpretation. This is the context in which the rest of the book should be read. Archaeologists working at these sites are not recovering a closed past. They are producing evidence that enters living disputes about responsibility, compensation, national identity and memory. That is why the ethical dimension is not an addition to the method but part of it. Beyond the familiar cases The field's best-known cases come from a small number of countries, above all the United States, Australia, Germany and Poland, and that concentration shapes its assumptions. It is worth remembering how much wider the phenomenon was. France operated penal colonies in French Guiana from the 1850s until the mid-twentieth century and in New Caledonia in the second half of the nineteenth century. Convicts sentenced to transportation, and later repeat offenders sentenced to relegation, were shipped across the world and put to work clearing forest, building roads and constructing the settlements that held them. The Camp de la Transportation at Saint-Laurent-du-Maroni in Guiana, where convicts were received and sorted, survives in part and has become a heritage site, and the remains of the penitentiary islands off the coast, including the notorious Devil's Island, are among the region's most visited places. Much of what has been written about them has come from memoirs and from the colony's own records; archaeological study has been far more limited than at Australian convict sites. Britain's colonial authorities in India built the Cellular Jail at Port Blair in the Andaman Islands, completed in 1906, to hold political prisoners in individual cells arranged in radiating wings, a design related to the separate system that shaped the Separate Prison at Port Arthur. Prisoners were set to labour such as oil pressing and coir work. The jail is now a national memorial in India, visited as a monument to the independence movement, and its meaning is framed almost entirely by that national story. In each of these cases, the physical form of the institution circulated between empires. A radial prison plan devised in Philadelphia and London was reproduced in Tasmania and the Andamans; a barrack design used for soldiers was adapted for prisoners; a regime of classification by conduct was borrowed from one colony by another. Part of the value of a comparative archaeology is that it can trace these borrowings, and show that forced labour systems were not isolated national aberrations but connected parts of a wider history of punishment, empire and labour. Reading doubled evidence One consequence runs through every later chapter. Because the evidence is double-voiced, interpretation needs a discipline that ordinary settlement archaeology sometimes lacks: the habit of asking, of every feature and object, whose intention it primarily records and whose intention it might also record. Take a simple example. At convict sites in New South Wales and Van Diemen's Land, government property was routinely stamped with the broad arrow, a mark of the British Crown's ownership that appeared on tools, buildings materials, clothing and equipment. A fragment of a spade with a broad arrow is, first, a record of the state's claim on the object and, by extension, on the labour of the person using it. But the same mark appears on objects found far from any government establishment, in contexts suggesting they were stolen, traded or kept. The mark of ownership then becomes a record of its own subversion. Or take the ration. At the Tempelhof forced labour camps, excavators found fragments of porcelain tableware bearing printed slogans and insignia of the regime. Such crockery tells us about the regime's attempt to saturate even eating with its symbols. Comparing the distributions of tableware and other finds across different barrack areas, however, can tell us something the regulations only hint at: how differently groups of workers were supplied and treated, and how far the hierarchy of the Nazi racial order, which ranked western European workers above those from Poland and the Soviet Union, reached down to the level of the soup plate. Reading doubled evidence also means resisting two temptations. One is to see resistance everywhere, turning every personal object into heroic defiance. People in forced labour systems did resist, often at great cost, but they also simply lived, and a clay pipe may be nothing more than evidence that someone smoked. The other temptation is to see only the regime, treating the coerced as passive material on which the system acted. That reading, however well meant, repeats the archive's reduction of people to the objects of power. The discipline lies between: taking the apparatus seriously as evidence of what was done, and taking the personal seriously as evidence of who it was done to, without forcing either into a story the material cannot support. The next chapter begins at the largest scale, with the landscapes in which these systems were built. Chapter 2: Landscapes of Control Before a single object is lifted from the ground, a site of forced labour can be read as a document. Its layout records decisions: where the workers slept relative to where their overseers slept, how far they had to walk to the place of work, which lines of sight were kept open and which were blocked, where the boundary ran and how it was guarded, where the dead were put. These decisions were made by people with a theory of how to extract labour from unwilling bodies, and the theory is legible in the plan. So, often, is the pushback: the paths worn where no path was intended, the gardens planted at the margins, the gaps under the wire. This chapter is about reading that landscape, and about the methods that now allow archaeologists to do so at a scale and with a gentleness that were not available a generation ago. Architecture as a theory of labour The clearest cases are those where a regime built from scratch according to an explicit plan. The Separate Prison at Port Arthur in Tasmania, built in the late 1840s and early 1850s and modelled on London's Pentonville, is an architecture of silence. Its wings radiate from a central hall from which a single officer could observe every corridor. Prisoners were confined to individual cells, forbidden to speak, and hooded when moved so that they could not recognise one another; in the chapel, each man stood in a separate stall that allowed him to see the chaplain but not his neighbours. The building is an argument, in stone, that isolation and silence would reform the criminal mind. It is also, now, the best evidence of how that argument was implemented, because the buildings survive even where the regulations are vague. The female factories of Van Diemen's Land and New South Wales, institutions where convict women were housed, put to work, and held between assignments, show a different logic. Eleanor Conlin Casella's excavations at the Ross Female Factory in Tasmania in the 1990s examined how the institution's division of women into classes, from those eligible for assignment to those undergoing punishment, was expressed in the separation of yards and wards. Her finds, which came to include small illicit goods traded across those boundaries, showed that the physical separation designed to sort women by conduct was also the setting for an internal economy that the administration did not control. On plantations, the logic of the plan was surveillance and proximity combined with social distance. At many Chesapeake and Carolina plantations, rows of quarters were placed within sight of the owner's house or the overseer's, so that the movements of enslaved people could be observed, while the house itself was positioned and ornamented to display the owner's mastery. At Monticello, Thomas Jefferson's plantation in Virginia, the row of workshops and dwellings known as Mulberry Row ran along the slope just below the main house, integrating enslaved artisans and domestic workers into the landscape of the mansion while keeping them in a clearly subordinate position. Archaeology along Mulberry Row since the 1950s, and more intensively since the 1980s, has recovered the footprints of buildings that had vanished above ground, turning an idealised landscape of the Enlightenment gentleman back into what it was: a working plantation. The forced labour camps of the twentieth century were often built from standardised, prefabricated components, and their regularity is itself evidence. Barracks built to a small number of stock designs, arranged in grids, fenced, and overlooked by towers, were erected across occupied Europe next to factories, quarries, airfields and construction sites. The standardisation made the camps cheap and quick to build and to dismantle, which is one reason so many vanished almost without trace after 1945. It also means that when an archaeologist finds a characteristic pattern of foundation posts or a regular sequence of rectangular footprints, the layout of a vanished camp can often be reconstructed from fragments. Landscapes of work are just as eloquent as landscapes of confinement. The limestone quarry on Robben Island, where political prisoners including Nelson Mandela were set to dig in glaring white rock, was a place of punishment through labour as much as of production. The Konyu cutting on the Burma–Thailand Railway, known to Allied prisoners as Hellfire Pass, was cut through rock largely by hand in 1943 by Australian, British and other prisoners and by Asian labourers during the period of intensified work the prisoners called the "Speedo". The cutting survives as a scar in the hillside, and walking through it communicates something about the scale of the labour demanded that no document can. The tunnel systems at Mittelbau-Dora in Germany's Harz mountains, where prisoners were forced to excavate and then work in underground factories producing V-2 rockets, are both workplace and tomb. Linear landscapes Not every site of forced labour is a bounded compound. Some of the largest are lines: roads, railways, canals and fortification belts, along which camps were strung like beads, each housing the workers for one section of the project and moving on as the work advanced. The Burma–Thailand Railway is the most notorious example. Built by the Japanese army in 1942 and 1943 to supply its forces in Burma, it ran for roughly 415 kilometres through mountainous jungle between Ban Pong in Thailand and Thanbyuzayat in Burma. Tens of thousands of Allied prisoners of war and a much larger number of Asian labourers recruited or coerced from across the region worked on it, housed in a chain of camps along the route. The death toll among the prisoners of war is usually put at around twelve to thirteen thousand; among the Asian labourers, whose deaths were far less carefully recorded, the numbers are uncertain but are generally estimated in the tens of thousands, far exceeding the prisoner deaths. Archaeologically, the railway is a long ribbon of features: cuttings, embankments, bridge footings, camp sites, cemeteries, and the scattered debris of work. Most of the camps were built of bamboo and thatch and have left little above ground, and many sections of the line have been absorbed into jungle, reservoirs or later railways. The linear form has consequences for research and memory. It is difficult to survey a site four hundred kilometres long, and so attention tends to concentrate at a few accessible points, above all Hellfire Pass and the bridge over the Khwae Yai at Kanchanaburi. The Asian labourers, whose camps were often separate from those of the Allied prisoners and whose experience was less documented, are correspondingly harder to find. A landscape approach that followed the whole line, rather than its most famous points, would almost certainly change the balance of the story. Road-building by convict gangs in Australia and by forced labourers and prisoners of war in German-occupied Finnish Lapland produced similar linear landscapes, with stockades, camps and quarries set at intervals along the route. In each case the road survives as infrastructure, still used, while the camps that built it have faded into the verge. The archaeologist's task is to restore the connection between the two. Rival geographies A landscape designed for control is never only that. The historian Stephanie Camp, writing about enslaved women in the American South, used the phrase "rival geography" to describe the alternative ways enslaved people knew and used the spaces around them: the paths through woods and swamps used to visit family on other plantations, to attend secret gatherings, to trade, or to hide for a time. Such geographies rarely leave dramatic features, but they can sometimes be traced in the distribution of artefacts away from sanctioned areas, in the location of provision grounds and gardens that enslaved people worked for their own subsistence, and in the placement of burial grounds at the edges of estates. At the Newton Plantation in Barbados, the enslaved people's burial ground lay in an uncultivated area away from the planter's domain. Burials there, investigated by Jerome Handler and Frederick Lange in the 1970s, showed practices that owed more to West African traditions than to the planter's church. The cemetery's location was, in part, a consequence of the owners' indifference; it was also a space that the enslaved community controlled. In twentieth-century camps, rival geographies appear at a smaller scale: the gap in a fence, the hiding place beneath a barrack floor, the garden plot, the area behind a latrine where forbidden conversations happened. Survivor testimony is often the key to recognising them, because an anomaly in the ground may mean nothing until someone who was there explains it. Seeing without digging The great methodological change in this field over the past three decades has been the rise of non-invasive survey. It matters here more than in most branches of archaeology, for three reasons. Many sites contain human remains that should not be disturbed, either because of religious law, the wishes of descendants, or simple respect. Many sites are large, overgrown or partly built over, so excavation can reach only a fraction of them. And many are contested, so that evidence gathered without disturbing the ground can be less controversial than evidence obtained by digging. The main tools, which are compared in the next chapter alongside excavation methods, work in complementary ways. Historical aerial photography is the foundation of much twentieth-century work. Allied and German reconnaissance aircraft photographed much of occupied Europe during the Second World War, and collections of these images, held in archives in Britain, the United States and Germany, record camps and work sites while they were in use or shortly after they were abandoned. Comparing wartime images with modern ones shows what has been lost, where structures stood, and where the ground has been disturbed. At Tempelhof, wartime air photographs helped locate the barracks of the forced labour camps that the excavators then investigated. Airborne laser scanning, generally known as LiDAR, sends pulses of light from an aircraft or drone and records their return. By filtering out the returns from vegetation, it produces a model of the ground surface that reveals subtle earthworks, foundations, pits and paths even beneath forest canopy. At Treblinka, where the site of the extermination camp and the adjoining penal labour camp known as Treblinka I were planted with trees after the war, Caroline Sturdy Colls's team combined LiDAR with walkover survey and archival research, and their work suggested that the camp had been considerably larger than the boundary marked by the memorial. On Alderney, similar methods have been used to map the remains of the island's four German labour and concentration camps, Helgoland, Norderney, Borkum and Sylt, and the fortifications their prisoners built. Ground-penetrating radar sends radio waves into the ground and records reflections from buried interfaces, such as the edges of pits, walls, floors and voids. It can suggest the size and depth of buried features, including graves, without breaking the surface, although its results depend heavily on soil conditions and are notoriously easy to over-interpret. Magnetometry detects small variations in the earth's magnetic field caused by buried iron, burnt material and disturbed soil, and is well suited to locating hearths, cremation deposits and metal-rich debris. Electrical resistance survey measures how easily current passes through the ground, which distinguishes, for example, compacted foundations from loose fill. None of these methods produces a picture of what is underground. Each produces a map of physical contrasts, which must be interpreted. A rectangular anomaly on a radar plot may be a grave, a cellar, a refuse pit, a bomb crater, or a natural feature. The strength of modern practice lies in integration: combining several survey methods, overlaying their results in a geographic information system with historical maps, aerial photographs and survivor accounts, and testing the interpretation with small, targeted interventions where these are acceptable. Landscapes of absence Some of the most important landscapes in this field are defined by what is missing. The Operation Reinhard camps were deliberately erased; so were many smaller forced labour sites whose operators, or whose post-war governments, had no wish to preserve them. After 1945, the barracks of countless labour camps in Germany were reused as housing for refugees and displaced persons, then demolished, and their sites built over. In the Soviet Union, the Gulag's camps were closed and allowed to decay in remote places where no one had reason to preserve them, and in some cases the state later discouraged attention to them. On many former plantations, the quarters were demolished while the main house was preserved and restored, producing a heritage landscape that shows the owners and hides the owned. Reading such landscapes requires attention to negative evidence: the unexplained clearing in a forest, the patch of different vegetation, the depression that marks a filled pit, the concrete footing without a building. It also requires attention to later human action, because the erasure was often itself a historical act. When the Nazi authorities dismantled Treblinka and built a farmhouse on the site to disguise it, they created a landscape that is evidence of a cover-up. When a post-war municipality built a housing estate over a labour camp, it made a decision about memory. These layers are part of the site's history and deserve recording as much as the wartime features beneath them. The landscape at Tempelhof illustrates the layering particularly well. The airfield had been the site of one of the earliest Nazi concentration camps, the Columbia-Haus, in the 1930s. During the war, aircraft manufacturers, including Weser Flugzeugbau, built labour camps for foreign workers and prisoners of war on the field's margins. After the war, the airport became a symbol of freedom through its role in the Berlin Airlift of 1948 and 1949, and later served as a commercial airport until its closure in 2008. The excavations from 2012 onward confronted a site where a celebrated story of liberty sat directly on top of a largely forgotten story of coercion, and where the wartime layer was literally beneath the post-war runways and fields. What landscape survey cannot do Landscape-scale methods have made it possible to document sites with minimal disturbance, and in some settings they are the only acceptable approach. They have limits that practitioners must be honest about. They are better at finding structures than people. A survey can locate a barrack's footprint but cannot tell who slept in it. They depend on physical contrasts that some features do not produce: a shallow grave in similar soil may be invisible to radar; a wooden structure that has rotted completely may leave no trace detectable from the surface. They generate large quantities of data that can be misread, and an anomaly announced prematurely as a mass grave can do serious damage if it turns out to be something else. Treblinka and other Holocaust sites have attracted denialists who seize on any gap between a survey interpretation and a later finding. Above all, landscape survey tends to reproduce the view from above, which was also the perspective of those who designed and ran these sites. The aerial photograph is literally a view from the reconnaissance aircraft; the site plan is the view of the architect and the commandant. The lived landscape of the prisoner, the enslaved worker or the convict was experienced on foot, at ground level, in cold and heat, under guard. Archaeologists working at these sites have increasingly tried to bring those experiences back into the interpretation, through walking survey, attention to what could and could not be seen from inside a barrack or a cell, and the integration of testimony. But the tension remains, and it is a first example of the book's larger theme: the methods available to archaeology are not neutral, and their default perspective is often that of power. The next step, when survey alone is insufficient and excavation is justified, carries heavier obligations still. Chapter 3: Excavating Under Moral Pressure Excavation is destruction. Every archaeologist learns this early: once a layer has been dug, it cannot be dug again, and the only thing that survives is the record made while digging. At most sites, this is an accepted trade-off between knowledge and preservation. At sites of forced labour, it acquires a moral weight that changes the whole calculation. The ground may contain the bodies of people who died because of the labour extracted from them. It may be sacred to a religious community or to descendants. It may be evidence in an unfinished legal or political dispute. And it may be the last place where anything of certain individuals survives. The question is not only how to dig well, but whether to dig at all, and for whom. The case for and against digging For much of the twentieth century, many specialists believed that the sites of Nazi atrocity did not need excavation. The crimes were documented by the perpetrators' own records, by the trials at Nuremberg and elsewhere, and by survivor testimony. Digging seemed at best redundant and at worst a desecration. Similar arguments were made, in different forms, about plantation burial grounds and convict cemeteries: that the dead should be left in peace, and that the documents told the story well enough. The argument has shifted, for reasons that the first two chapters have already indicated. The documents are partial, the sites were often deliberately erased, and denial feeds on gaps. Reinhard Bernbeck and Susan Pollock, reflecting on their work at Tempelhof, argued that archaeology's full potential is realised only when excavation is undertaken, because inconspicuous objects, studied systematically and in their spatial context, are a fundamental means of understanding both the conditions of suffering and the transgressive actions of camp inhabitants. Survey can locate a barrack; only excavation can recover the rosary, the wedding ring or the scrap of improvised jewellery that tells us who lived there. The case against digging remains strong wherever human remains are involved, and the resolution in most current practice is a hierarchy of intervention. Non-invasive methods come first and may be all that is done. Where excavation is justified, it is targeted at structures and activity areas rather than burials, designed to answer specific questions, and planned in consultation with the communities most concerned. Where human remains are encountered unexpectedly, work stops and the relevant authorities and communities are consulted before anything further happens. Jewish law and the Holocaust sites The most developed framework for excavation at sites of mass death has emerged from work at Holocaust sites, because Jewish religious law, Halacha, places strict limits on the disturbance of the dead. Graves are to be left undisturbed, and remains that are exposed should be reburied as quickly as possible. In practice, this has meant close cooperation between archaeologists and rabbinic authorities, above all the office of the Chief Rabbi of Poland, whose representatives have monitored work at several sites. At Treblinka, Caroline Sturdy Colls's team deliberately avoided the mass graves during their limited excavations, in order to comply with Halacha. Their excavations targeted structures such as the gas chambers, whose foundations were identified in part through distinctive ceramic tiles made by a Polish firm, consistent with survivor accounts that the chambers had been disguised as a bathhouse. Human remains that were encountered were reburied under rabbinic supervision. The earlier work at Bełżec shows the difficulty of these judgments. Between 1997 and 1999, a team led by Andrzej Kola of Nicolaus Copernicus University in Toruń used thousands of boreholes to map the extent of the burial pits ahead of the construction of a new memorial. The survey identified thirty-three pits interpreted as mass graves and gave the memorial's designers a basis for protecting them. But the technique itself, which involved drilling through burial deposits and bringing samples to the surface, has been criticised as incompatible with the principle of non-disturbance. The later preference for radar, LiDAR and other non-invasive methods at Holocaust sites is partly a response to that experience. At Sobibor, where Yoram Haimi and Wojciech Mazurek began excavations in 2007, joined later by Ivar Schute, the team worked over several seasons to locate the camp's structures, eventually identifying the foundations of the gas chambers in 2014. Among the thousands of personal objects recovered were a wedding ring inscribed in Hebrew with the traditional marriage formula and a pendant later linked to a young girl deported from Frankfurt. These finds came from the areas where victims undressed and walked to their deaths, and from a well used by the Jewish prisoners forced to labour in the camp's killing area. The excavation had to balance the scientific value of finding and documenting these areas against the certainty that parts of the site held human remains. Descendant authority and the African Burial Ground The most consequential case in the Americas began with a legal requirement and ended in a new model of practice. In 1991, archaeological testing required by federal law ahead of construction of a federal office building at 290 Broadway in Lower Manhattan revealed that part of an eighteenth-century burial ground for free and enslaved Africans survived beneath the city's streets. Colonial maps had marked it as the "Negros Burial Ground". Over the following months, the remains of more than four hundred individuals were excavated, under intense time pressure and with little consultation. The response from New York's African American community was forceful. Activists, clergy and politicians protested that the ancestors were being treated as an obstacle to construction, that the excavation was being run by firms with no expertise in African diaspora history, and that the descendant community had no say. Congress intervened, excavation of further burials was halted, and responsibility for the scientific study passed to a team based at Howard University, a historically Black institution, under the direction of the biological anthropologist Michael Blakey. The Howard team built its research design around questions posed by the descendant community: where did these people come from, what was their physical quality of life, what biocultural transformations did they undergo, and how did they resist enslavement. Blakey described the descendant community as the project's "ethical client", whose interests had priority over those of the government agency that funded the work or the scientific community that would read its reports. The studies documented skeletal evidence of hard, repetitive labour among both men and women, of nutritional stress and disease, and of the very high proportion of children among the dead, close to forty per cent. They also documented cultural continuities, including dental modification practised in parts of West and Central Africa, and burial goods such as beads. In 2003 the remains were returned to the site in carved wooden coffins and reinterred, and the burial ground is now a national monument. The African Burial Ground did not settle every question. Some descendants and scholars would have preferred no excavation at all; others valued the knowledge the research produced. What it established was a principle that has since shaped work at other burial grounds of enslaved people: those with a genealogical or cultural relationship to the dead have a right to shape whether, how and why they are studied. At the Anson Street burial ground in Charleston, South Carolina, found during construction in 2013, researchers working with a community group of Gullah Geechee descendants developed a genetic study whose design, sampling and interpretation were negotiated with the community before any laboratory work began; the results, published in 2023, traced the ancestries of the individuals to several regions of Africa. The forensic standard Archaeology at sites of coercion increasingly borrows the standards of forensic practice, developed in the investigation of recent crimes and human rights abuses. Since the founding of the Argentine Forensic Anthropology Team in 1984 to investigate the disappeared of Argentina's military dictatorship, forensic archaeologists have worked in the aftermath of conflicts from the Balkans to Guatemala to Iraq. Their methods are designed to produce evidence that can withstand hostile scrutiny in court. Several elements of that standard carry over to historical sites, even where no prosecution is possible. Recording is exhaustive. Every find is located precisely in three dimensions, every layer is photographed and described, and the relationship of every object to every other is documented. This is good archaeological practice anywhere, but at a site where the evidence may be contested, it must be done in a way that another specialist, or a sceptical reviewer, could check. Chain of custody is maintained. The movement of each find, from the ground to the laboratory to the archive, is logged, so that no one can later claim that material was introduced or altered. Interpretation is disciplined. Forensic reports distinguish between what the evidence shows, what it is consistent with, and what it cannot establish. At sites vulnerable to denial, an overstated claim can do more damage than an understated one, because a single error becomes a pretext for dismissing everything else. The review of the Nazi occupation of Alderney, commissioned by the United Kingdom's special envoy on post-Holocaust issues, Lord Pickles, and published in May 2024, illustrates this discipline. A panel of thirteen historians, archaeologists and other specialists, coordinated by the Cambridge archaeologist Gilly Carr, re-examined the evidence for the number of people who died on the island. The panel estimated that between 7,608 and 7,812 forced and slave labourers had been sent to Alderney, and that between 641 and 1,027 people had died there, well above the figure of 389 that had been widely cited. It also concluded that claims of tens of thousands of deaths, or of Alderney as a "mini-Auschwitz", were not supported by the evidence. The review's credibility rested on its readiness to say both things: that the toll was higher than the official figure, and lower than the most extreme claims. Archaeologists working on the island, including those who had mapped the camps and the fortifications their prisoners built, contributed to that assessment. Digging a labour camp Excavating a twentieth-century forced labour camp is technically different from excavating an ancient settlement, and the differences shape what can be learned. The deposits are shallow and compressed. A camp that was occupied for three or four years, then demolished and perhaps built over, may leave a layer only a few centimetres thick, lying directly beneath modern surfaces. Distinguishing the wartime material from later debris requires close attention to stratigraphy and to the manufacturing dates of objects, many of which carry makers' marks, patent numbers or datable designs. The quantity of material is large and repetitive. The Tempelhof excavations produced a database of some ninety thousand artefacts, a great many of them mass-produced: nails, glass, porcelain fragments, fittings, wire. Faced with such volumes, older practice sometimes recorded only the "interesting" pieces and discarded the rest. The Tempelhof team took the opposite approach, treating even the most inconspicuous finds as data and analysing their distribution statistically across different parts of the camps. The results allowed them to compare how different groups of workers lived, which would have been impossible if only the striking objects had been kept. The ground can be dangerous. Sites of wartime labour may contain unexploded ordnance, contaminated industrial waste, asbestos from demolished buildings and other hazards. Excavation at Tempelhof and at other German sites has had to be coordinated with specialists in explosive clearance. Similar precautions apply at military sites in the Channel Islands and elsewhere. The excavation is also a public event. Digging at a site of Nazi forced labour in the middle of a modern city attracts attention, and the Tempelhof team made a point of opening the excavation to visitors, explaining the work and the history it was uncovering. That openness is part of the ethical practice of such projects: the knowledge is produced in public, for a public that has a stake in it, rather than behind a fence. It also exposes the excavators to challenge, which is a healthy thing at a site where the past is contested. Finally, the objects themselves may carry obligations. Personal possessions recovered from a labour or concentration camp are, in a sense, the property of people who were robbed of everything. Where an owner or a family can be identified, some projects have returned objects or offered to do so. Where they cannot, the objects are held in trust, usually by a memorial museum, rather than treated as ordinary archaeological collections. Comparing methods Different techniques answer different questions at different costs to the site, and the choice among them is always a compromise. Table 2 sets out the main options side by side. Table 2. Principal survey and excavation techniques at sites of forced labour. Technique What it detects Level of disturbance Main limitation Archival aerial photography Structures, fences, pits and disturbance at the time of photography None Coverage and resolution vary; features may be obscured Airborne laser scanning (LiDAR) Surface earthworks, foundations and paths, including under trees None Cannot see features with no surface expression Ground-penetrating radar Buried interfaces, voids, walls, pits and graves None Highly dependent on soil; easily over-interpreted Magnetometry and resistance survey Burnt deposits, iron debris, foundations, disturbed soil None Detects contrasts, not identities; modern metal interferes Coring and augering Depth and content of buried deposits Low but direct; perforates deposits Unacceptable where graves may be pierced Targeted excavation Stratigraphy, artefacts in context, structures, remains High and irreversible Samples only small areas; ethical limits near burials The table understates one factor: time. Non-invasive survey can cover a large site in days, while careful excavation of a single barrack may take a season. At sites facing development, there is often no choice about whether to disturb the ground, only about how well it will be recorded. Much of the archaeology of forced labour in Germany, as in the United States, has taken place in this setting, triggered by construction rather than by research. Bodies of evidence Where human remains are studied, with appropriate consent, they can speak about the physical cost of coerced labour in ways that no document can. Bioarchaeologists read the skeleton for signs of heavy work, including changes where muscles and tendons attach to bone, degeneration of the spine and joints, and certain kinds of fracture. They read teeth and bones for episodes of childhood malnutrition or illness, which leave lines and defects in tooth enamel. Chemical analysis of the isotopes of strontium and oxygen in tooth enamel can indicate the geological region where a person grew up, and so distinguish people born in Africa from those born in the Americas. A study of individuals from the Newton Plantation cemetery in Barbados, published in 2009, used such analysis to identify several people who had probably been born in Africa and forcibly transported, confirming what dental modification had already suggested for some of them. These methods must be used with care. The skeletal markers associated with heavy labour are influenced by age, sex, genetics and body size, and a single skeleton rarely proves a particular occupation. Comparisons between populations are more reliable than readings of individuals, and even they depend on the composition of the sample. The ethical difficulty is also persistent: the bodies of enslaved people were exploited in life, sometimes dissected after death by medical schools, and displayed in museums as racial specimens. Any study that treats them primarily as data risks continuing that history. The best current work, like the Howard University project and the Anson Street study, tries to make the research serve the questions and interests of the communities connected to the dead. The same principle underlies the chapters that follow. What is recovered from the ground carries obligations. The next chapter turns to the first category of finds: the objects through which the regime imposed itself. Hashtags: #TheArchaeologyOfForcedLabor #ForcedLaborArchaeology #MaterialRemains #CoerciveRegimes #DoubleVoicedEvidence #ConvictLabor #AtlanticChattelSlavery #NaziForcedLabor #JapaneseWartimeLabor #SovietGulag #LandscapesOfControl #MaterialCultureOfCoercion #MaterialLifeOfTheCoerced #RivalGeographies #EthicsOfExcavation #NonInvasiveArchaeology #HistoricalAerialPhotography #LiDAR #GroundPenetratingRadar #ForensicArchaeology #DescendantCommunityAuthority #HumanRemainsEthics #AfricanBurialGround #HeritageAndMemory #FutureOfForcedLaborArchaeology

Download the Book (PDF): Introduction Data is the lifeblood of the modern enterprise, and Martin Kleppmann's textbook is the gold standard for understanding how large-scale data systems actually work. But at over 600 pages, it is heavily focused on the complex computer science of replication, partitioning, and distributed transactions. Expecting an IT management or business student to synthesize this immense volume of highly technical material into a concise strategic framework is a massive challenge. This book is your strategic shortcut. It is explicitly written to explain Designing Data-Intensive Applications by Martin Kleppmann, bridging the gap between hardcore software engineering and high-level enterprise architecture. We isolate the core concepts of scalability, consistency, and fault tolerance, translating them into the exact vocabulary you need for your exams. By focusing on the structural logic of data systems, this guide ensures your technological analyses are both accurate and academically rigorous. The book and its author Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems was published by O'Reilly Media in 2017. Its author, Martin Kleppmann, came to the subject from industry before turning to research. He co-founded the startup Rapportive, which LinkedIn acquired, and he then worked at LinkedIn on large-scale data infrastructure, including the stream-processing ecosystem around Apache Kafka and Apache Samza. He later moved into academic research on distributed systems, and he is now based at the University of Cambridge. That mixture of backgrounds shows on every page. The book is precise enough for researchers and practical enough for working engineers, and it cites hundreds of papers, blog posts and incident reports so that readers can trace each claim to its source. The book became something unusual for a technical title: a reference that working engineers, interviewers and university lecturers all treat as common ground. Its lasting appeal comes from a decision Kleppmann made early. Rather than teaching any single product, he explains the principles that products share. Databases, caches, search indexes, message queues and batch processors come and go; the ideas underneath them, such as how to copy data between machines or how to keep concurrent writes from corrupting each other, change far more slowly. A reader who understands those ideas can evaluate a new product in an afternoon, because the product is almost always a new combination of old trade-offs. In March 2026 O'Reilly published a second edition, written by Kleppmann together with Chris Riccomini, an engineer who also worked on data infrastructure at LinkedIn and co-created Apache Samza. The second edition updates the material for a world dominated by cloud services, data warehouses and data lakes, and it revises the organization and terminology in places. It grows to fourteen chapters, opens with a new chapter on trade-offs in data systems architecture, renames the chapter on partitioning to use the now-dominant term sharding, and closes with a chapter devoted to the ethics of data systems. This guide is based mainly on the first edition, whose three-part structure remains the clearest way to learn the subject and is the version most course reading lists still cite. Where the field has moved since 2017, the guide says so and marks the addition as its own. The first edition is organized in three parts. The first, Foundations of Data Systems, covers the ideas that apply even to a single machine: what reliability, scalability and maintainability mean; how data models shape what an application can express; how storage engines lay bytes on disk; and how data is encoded so that it can survive changes to the software that reads it. The second, Distributed Data, asks what happens when data lives on many machines: replication, partitioning, transactions, the unreliability of networks and clocks, and the hard problems of consistency and consensus. The third, Derived Data, looks at systems built by transforming data from one form into another, through batch processing and stream processing, and ends with Kleppmann's view of where data systems are heading, including a serious discussion of ethics and privacy. Why this matters to managers A student of IT management might reasonably ask why any of this is their business. Nobody expects a chief information officer to implement a consensus algorithm. But the choices those algorithms force are business choices. Whether a retailer's checkout should refuse orders during a network failure or accept them and risk overselling is not a question an engineer can answer alone; it depends on what an oversold order costs compared to a lost one. Whether a bank can let customers in two regions update the same account simultaneously depends on regulatory exposure, not just latency. Whether an analytics team should build on nightly batch jobs or real-time streams depends on how fast decisions need to be made and how much the organization is willing to spend on operational complexity. Kleppmann's book is, underneath its technical surface, a catalogue of such trade-offs. Each technique buys something and costs something. Replication buys availability and read capacity but introduces lag and the possibility of conflict. Partitioning buys scale but makes some queries expensive and transactions harder. Strong consistency buys simplicity for application developers but costs latency and availability during failures. The manager who can name these trade-offs, ask which one a proposed architecture has made, and connect that choice to business requirements is doing exactly the work that enterprise architecture exists to do. The controlling idea of this guide This guide argues one thing throughout: every data architecture is a bundle of explicit trade-offs among reliability, scalability and maintainability, and good architectural judgment consists of knowing which guarantees the business actually needs and what each guarantee costs. There is no best database, no correct consistency model and no universally right processing style. There are only systems whose guarantees match their purpose and systems whose guarantees do not. Holding that idea steady makes the technical material far easier to absorb. Every mechanism in the chapters that follow can be read with three questions in mind. What problem does this solve? What does it give up to solve it? When would a business prefer the opposite choice? A student who can answer those three questions for leader-based replication, hash partitioning, snapshot isolation or event sourcing understands the topic at the level an examiner, or an employer, is looking for. How the guide is organized The chapters follow the order of Kleppmann's argument, because that order is itself instructive: it builds from one machine to many, and from storing data to transforming it. The opening chapters establish the vocabulary of system goals, then examine how data is modeled, stored and encoded. The middle of the guide moves into distributed data, where most of the difficult ideas live: replication and its anomalies, partitioning and its hot spots, transactions and their isolation levels, the many ways a distributed system can fail, and the guarantees, such as linearizability and consensus, that engineers build to cope. The later chapters turn to derived data, treating batch and stream processing together because they share a single underlying model, and then to Kleppmann's closing arguments about the design of future systems and the responsibilities of the people who build them. Each chapter explains the relevant concepts in plain terms, then draws out the strategic lens: what decision a manager or architect faces and how to reason about it. Real incidents and published designs anchor the discussion. GitHub's October 2018 outage, in which a 43-second network interruption led to more than a day of degraded service, illustrates why replication and failover are business risks rather than purely technical details. Amazon's 2007 Dynamo paper and Google's 2012 Spanner paper show two of the world's largest companies making opposite choices on consistency, each for sound reasons. The leap second of June 2012, which disrupted websites and airline check-in systems, shows how something as apparently simple as time can undermine a distributed system. Every chapter closes with key takeaways and review questions. The takeaways condense the argument into statements worth remembering; the questions test whether the reasoning, not just the vocabulary, has landed. A short glossary, a list of further reading and notes on sources follow the conclusion. A note on faithfulness. This guide explains Kleppmann's book; it does not replace it. His arguments are paraphrased and attributed, and where the guide adds analysis, modern context or criticism of its own, it says so. Readers who find a topic here that matters to their work should go to the original, which rewards careful reading far more than any summary can. Chapter 1: Reliable, Scalable and Maintainable Systems Kleppmann opens his book by making a distinction that frames everything after it. Many modern applications are not compute-intensive, limited by how fast a processor can crunch numbers. They are data-intensive: their hardest problems are the amount of data, its complexity and the speed at which it changes. A social network, an online store or a banking app does relatively simple computation on each request. What makes it difficult is that millions of such requests arrive, each touching shared data that must be stored, retrieved, indexed, cached, searched and processed in the background. Such applications are rarely built on one piece of software. They are assembled from standard building blocks: a database to store data, a cache to speed up expensive reads, a search index to support keyword queries, a message queue or stream processor to handle work asynchronously, and batch jobs to crunch accumulated data periodically. Kleppmann's point is that once an application stitches these components together, the application developer has become a data system designer. The guarantees the whole system offers, such as whether a cache is correctly invalidated when the database changes, are no longer provided by any single product. They are the responsibility of whoever composed the pieces. To reason about such composite systems, Kleppmann proposes three concerns that matter in almost every case: reliability, scalability and maintainability. They are the vocabulary for the rest of the book, and for most exam questions on the subject. Reliability: working correctly when things go wrong Informally, a reliable system continues to work correctly even when things go wrong. Kleppmann sharpens this by distinguishing a fault from a failure. A fault is one component deviating from its specification: a disk dies, a process crashes, a network link drops packets. A failure is the system as a whole ceasing to provide the required service to the user. Because the probability of faults can never be reduced to zero, the goal is to design fault-tolerant (or resilient) systems, in which faults do not cascade into failures. Kleppmann notes, somewhat provocatively, that it can make sense to increase the rate of faults deliberately, by killing processes at random, so that fault-tolerance machinery is constantly exercised. Netflix's Chaos Monkey is the best-known example of this practice. He groups faults into three kinds. Hardware faults are the traditional concern. Hard disks have a mean time to failure often quoted at 10 to 50 years, which sounds reassuring until one remembers that a data center with ten thousand disks should then expect, on average, around one disk failure per day. The classic response is redundancy: RAID arrays, dual power supplies, backup generators. Kleppmann observes a shift here. As data volumes and computing demands have grown, applications run on far more machines, and cloud platforms are designed to prioritize flexibility over the reliability of any single machine, so virtual machines can disappear without warning. The consequence is a move toward systems that tolerate the loss of entire machines through software, which also brings an operational benefit: a machine can be patched and rebooted one node at a time without taking the whole system down. Software errors are more insidious because they are correlated. A disk failure on one machine says little about the disk on the next machine, but a bug in a shared piece of software can hit every node at once. Kleppmann gives the example of the leap second on 30 June 2012, when a bug in the Linux kernel caused many applications to hang simultaneously; the incident is examined more closely in Chapter 7. Other examples include a runaway process consuming shared resources, a dependency that slows down, and cascading failures in which a small fault triggers faults elsewhere. There is no quick fix, only many small disciplines: careful thinking about assumptions, thorough testing, process isolation, allowing processes to crash and restart, and measuring and monitoring behavior in production. Human errors are, by Kleppmann's reading of the evidence, a leading cause of outages; he cites a study of large internet services that found operator configuration errors to be a larger cause of outages than hardware faults. The response is not to blame operators but to design systems that make the right thing easy and the wrong thing hard: well-designed interfaces and abstractions, sandbox environments where people can experiment safely with real data, thorough automated testing, quick and easy rollback of configuration changes, gradual rollout of new code so that a bug affects a small subset of users first, and detailed monitoring, which Kleppmann calls telemetry by analogy with the instrumentation used to track rockets. Why should a business care? Kleppmann's answer is plain: bugs in business applications cause lost productivity, legal risk if figures are reported incorrectly, and outages of e-commerce sites carry direct costs in revenue and reputation. He also acknowledges that there are situations, such as a prototype for an unproven market, where one might consciously sacrifice reliability to reduce development cost. The strategic point is that this should be a conscious decision, not an accident. Scalability: describing load and performance Scalability is the term used to describe a system's ability to cope with increased load. Kleppmann insists that it is not a one-dimensional label. Saying "X is scalable" or "Y doesn't scale" is meaningless. The useful question is: if the system grows in a particular way, what are our options for coping with that growth? Load parameters and the Twitter example To answer that, one first has to describe the current load succinctly, using load parameters. The right parameters depend on the architecture: requests per second to a web server, the ratio of reads to writes in a database, the number of simultaneously active users, the hit rate on a cache. Sometimes the average case matters most; sometimes a small number of extreme cases dominate. Kleppmann illustrates this with Twitter's home timeline, drawing on a 2012 presentation by Twitter engineers. Two operations mattered. Posting a tweet averaged around 4,600 requests per second, with peaks above 12,000. Reading a home timeline, the stream of tweets from people a user follows, ran at around 300,000 requests per second. Handling 12,000 writes per second is not especially hard in itself. The difficulty was fan-out: each user follows many people and is followed by many people. Twitter tried two approaches. In the first, posting a tweet simply inserted it into a global collection. When a user loaded their home timeline, the system looked up everyone they followed, found all of those people's tweets, and merged them in time order. This is cheap on write and expensive on read, and the read load was far higher than the write load, so the system struggled. In the second approach, Twitter maintained a cache of each user's home timeline, like a mailbox. When someone posted a tweet, the system looked up all of their followers and inserted the new tweet into each follower's cached timeline. Reading a timeline became cheap, because the result had already been computed. This worked better because reads vastly outnumbered writes, so it made sense to do more work at write time and less at read time. The second approach has its own weakness. The average tweet reached about 75 followers, so 4,600 tweets per second became about 345,000 cache writes per second. But the average hides a long tail: some accounts had more than 30 million followers, so a single tweet from such an account triggered more than 30 million writes. Delivering tweets to followers within about five seconds, Twitter's target, became a significant challenge. Twitter settled on a hybrid. Most users' tweets continued to be fanned out at write time, while tweets from a small number of accounts with very large followings were fetched separately at read time and merged in. The lesson Kleppmann draws is that the distribution of followers per user, weighted by how often those users tweet, was the key load parameter for Twitter's scalability. A different application will have a different key parameter, and finding it is the first step of any serious scalability discussion. For a student, the example teaches a general pattern: work can be shifted between write time and read time, and the right balance depends on the shape of the load. Describing performance: percentiles and tail latency Once load is described, one can ask what happens to performance when load increases. Two framings are useful. If load increases and resources stay the same, how is performance affected? And if load increases, how much must resources increase to keep performance unchanged? For a batch system, performance is usually throughput: records processed per second, or the time to run a job over a dataset of a given size. For an online system, it is usually response time: the time between a client sending a request and receiving a response. Kleppmann distinguishes response time, which is what the client sees and includes network and queueing delays, from latency, the duration a request spends waiting to be handled. Response time is not a single number but a distribution, because the same request will take different amounts of time on each attempt. The average (the arithmetic mean) is a poor summary, because it says nothing about how many users actually experienced a given delay. Kleppmann recommends percentiles. Sort all response times from fastest to slowest; the median, or 50th percentile (p50), is the halfway point. If the median is 200 milliseconds, half of requests return in less than 200 milliseconds and half take longer. Higher percentiles, such as p95, p99 and p999, show how bad the outliers are. A p99 of 1.5 seconds means that 99 out of 100 requests are faster than 1.5 seconds and one in a hundred is slower. These high percentiles are called tail latencies, and Kleppmann argues they matter commercially. He notes that Amazon describes internal response time requirements in terms of the 99.9th percentile, even though that affects only one request in a thousand, because the customers with the slowest requests are often those with the most data in their accounts, who have made many purchases and are therefore among the most valuable. Amazon's published Dynamo paper gives an example of this style of requirement: a service guaranteeing a response within 300 milliseconds for 99.9 percent of requests at a peak load of 500 requests per second. Kleppmann also reports Amazon's observation that small increases in response time measurably reduce sales. Optimizing the 99.99th percentile, by contrast, was judged too expensive for the benefit. Percentiles are often used in service level objectives (SLOs) and service level agreements (SLAs), contracts that define the expected performance and availability of a service. A typical formulation might state that the service is considered up if its median response time is under 200 milliseconds and its p99 is under one second. Two further points make tail latency especially important. First, head-of-line blocking: a server can process only a few requests in parallel, so a few slow requests can hold up everything queued behind them. This is why response times should be measured on the client side. Second, tail latency amplification: when serving one user request requires calling several backend services, the chance that at least one of them is slow grows with the number of calls. Even if only a small fraction of backend calls are slow, a large fraction of end-user requests can end up slow. Approaches to coping with load Kleppmann distinguishes scaling up (vertical scaling, moving to a more powerful machine) from scaling out (horizontal scaling, distributing load across many smaller machines). The latter is also called a shared-nothing architecture. Systems that add resources automatically when load increases are called elastic. Kleppmann's practical advice is balanced: distributing stateless services is relatively simple, but taking stateful data systems from one node to many introduces a great deal of complexity, and until fairly recently it was common wisdom to keep a database on a single node until cost or availability forced otherwise. He expects distributed data systems to become the default as tools improve, which, looking back from today, is largely what happened with cloud-managed databases. The most important point is that there is no generic, one-size-fits-all scalable architecture. A system designed for 100,000 requests per second, each 1 kilobyte in size, looks very different from one designed for three requests per minute, each 2 gigabytes in size, even though the two have the same data throughput. An architecture that scales well is built around assumptions about which operations will be common and which rare. If those assumptions are wrong, the scaling effort is wasted. For an early-stage product, Kleppmann notes, it is usually more important to be able to iterate quickly on features than to scale to some hypothetical future load. Maintainability: the cost that lasts Kleppmann reminds readers that the majority of the cost of software is not its initial development but its ongoing maintenance: fixing bugs, keeping systems operational, investigating failures, adapting to new platforms and use cases, repaying technical debt and adding features. Many people dislike working on so-called legacy systems, but every system becomes legacy eventually, so it should be designed to minimize the pain. He proposes three design principles. Operability means making it easy for operations teams to keep the system running smoothly. Good operations teams monitor health, track down problems, keep software patched, anticipate future problems such as capacity limits, establish good practices for deployment and configuration, and preserve the organization's knowledge about the system as people come and go. Data systems help by providing visibility into their internal behavior, supporting automation, avoiding dependence on individual machines, offering good documentation and predictable behavior, and providing sensible defaults while allowing administrators to override them. Simplicity means making it easy for new engineers to understand the system, by removing as much complexity as possible. Kleppmann distinguishes essential complexity, inherent in the problem the software solves, from accidental complexity, which arises only from the implementation. The most powerful tool for removing accidental complexity is abstraction: a good abstraction hides implementation detail behind a clean, understandable interface and can be reused across many applications. High-level programming languages abstract away machine code; SQL abstracts away on-disk data structures and concurrency. Much of the book is about finding good abstractions for distributed data. Evolvability means making it easy to change the system in the future, as requirements, business priorities, regulations and platforms shift. Kleppmann connects this to agile practices but notes that most discussion of agility focuses on small scales, such as a few source files. His interest is in agility at the level of a whole data system, such as refactoring Twitter's architecture from one timeline approach to another. Simplicity and good abstractions make that kind of large change far easier. The strategic lens For a manager, Chapter 1 of Kleppmann's book supplies a checklist that can be applied to any proposed architecture. On reliability: which faults does the design tolerate, which does it not, and what is the business cost of each failure mode? On scalability: what are the load parameters, which one is expected to grow, and what happens to tail latency when it does? On maintainability: how will operators see what the system is doing, how much of its complexity is accidental, and how hard will it be to change when the business changes? This guide adds one observation of its own. These three properties compete. Redundancy improves reliability but adds components to operate. Scaling out handles load but adds the complexity of distribution. Aggressive simplicity may mean accepting a single-node database with a lower ceiling. Architectural judgment is the art of choosing which property to favor, for which part of the system, at which stage of the business's life. Key Takeaways · A fault is a component deviating from specification; a failure is the whole system ceasing to serve users. Reliable design aims to stop faults becoming failures. · Scalability is not a label but a question: if load grows along a specific parameter, what options exist to cope? · Twitter's home timeline shows that work can be shifted between write time and read time, and that the distribution of followers, not the average, was the critical load parameter. · Response times should be described with percentiles, not averages; tail latencies (p99, p999) often affect the most valuable users and are amplified when requests call many services. · Maintainability rests on operability, simplicity (removing accidental complexity through good abstractions) and evolvability. Review Questions 1. Distinguish a fault from a failure, and explain why deliberately inducing faults can improve reliability. 1. Why are software errors often more damaging than hardware faults in a large system? 2. Explain the two approaches Twitter used to build home timelines, and why the company moved to a hybrid of both. 3. A service reports an average response time of 150 milliseconds. Why might this figure mislead a manager, and what should be reported instead? 4. What is tail latency amplification, and what does it imply for an architecture composed of many microservices? 5. Give an example of accidental complexity and explain how abstraction can remove it. Chapter 2: Data Models and Query Languages Kleppmann argues that data models are perhaps the most important part of developing software, because they shape not only how software is written but how its authors think about the problem. Most applications are built by layering one data model on another. An application developer models the real world as objects and data structures; those structures are expressed in a general-purpose model such as tables, JSON documents or graphs; the database engineers decide how that model is represented as bytes in memory and on disk; and hardware engineers decide how those bytes become electrical currents and magnetic fields. Each layer hides the complexity of the layer below by providing a clean model. Choosing the model at the second level, the one application developers work with every day, is a decision with long consequences, because it determines which operations are easy and which are awkward or slow. The relational model and the document challenge The relational model, proposed by Edgar Codd in 1970, organizes data into relations (tables), each an unordered collection of tuples (rows). It was a theoretical proposal at first, and many doubted it could be implemented efficiently, but by the mid-1980s relational database management systems and SQL had become the standard tools for storing and querying data with some regular structure. Kleppmann notes the remarkable durability of this dominance. Competing approaches, including the network and hierarchical models of the 1970s and the object databases and XML databases of later decades, generated excitement and then faded, while relational databases went on to power a vast range of applications well beyond their origins in business data processing. In the 2010s the challenger was NoSQL, a term Kleppmann notes was originally just a catchy hashtag for a 2009 meetup on open source, distributed, non-relational databases. The drivers he identifies are worth remembering: a need for greater scalability than relational databases could easily achieve, including very large datasets and very high write throughput; a preference for free and open-source software; specialized query operations poorly supported by the relational model; and frustration with the restrictiveness of relational schemas. He predicts, correctly as it turned out, that relational and non-relational stores would be used side by side, an idea sometimes called polyglot persistence. The object-relational mismatch Most application code is written in object-oriented languages, and translating between objects in code and rows in tables requires an awkward layer of glue. This disconnect is called the impedance mismatch, a term borrowed from electronics. Object-relational mapping frameworks reduce the boilerplate but cannot hide the differences entirely. Kleppmann illustrates this with a résumé, such as a professional networking profile. A person has one name, but may have several jobs, several periods of education and several pieces of contact information. In a traditional relational schema, the one-to-many relationships would be split into separate tables, with foreign keys pointing back to the user. Fetching a full profile requires either several queries or a multi-way join. A document model represents the same profile as a single self-contained JSON document, in which the list of positions and the list of education entries are nested inside the user record. The document has better locality: everything about the profile is in one place, retrieved with one query. The tree structure of one-to-many relationships is made explicit in the document's nesting. Many-to-one and many-to-many relationships The document model's weakness appears with relationships that are not tree-shaped. Suppose each profile lists a region and an industry. It is better to store a standardized identifier, such as an ID for "Greater Seattle Area", than a free-text string, because a standardized list gives consistent spelling, avoids ambiguity, makes updates easy, supports localization and enables better search. Storing an ID rather than the human-readable text is a form of normalization: the meaningful information is stored in one place and referred to everywhere else, so there is no duplication to keep consistent. Normalization requires many-to-one relationships (many people live in one region), and these fit relational databases naturally because joins are easy. In document databases, joins were historically weak or absent, so the application had to emulate them with multiple queries. Moreover, data tends to become more interconnected as features are added. If organizations and schools become entities with their own pages, and users can recommend one another, the résumé acquires many-to-many relationships that a pure document tree cannot express cleanly. Kleppmann places this debate in history. The hierarchical model of IBM's IMS in the 1970s, which represented data as trees of records nested inside records, much like JSON, faced exactly the same difficulty with many-to-many relationships. Two solutions were proposed: the relational model, and the network model (CODASYL), in which records could have multiple parents and were accessed by following pointer-like access paths. The network model required programmers to know and navigate these paths manually, which made queries and schema changes painful. The relational model instead let a query optimizer decide automatically how to execute a query and which indexes to use. That separation, Kleppmann argues, is why the relational model won: the hard work of choosing access paths was done once, by the optimizer, rather than repeatedly by every application developer. Which model leads to simpler code? Kleppmann's comparison is even-handed. The document model offers schema flexibility, better performance due to locality, and a closer match to application data structures for some applications. The relational model offers better support for joins and for many-to-one and many-to-many relationships. If the application's data has a document-like structure, a tree of one-to-many relationships usually loaded all at once, the document model is a good fit. Shredding such a structure into many tables can make code unnecessarily complicated. If the application uses many-to-many relationships heavily, the document model becomes less appealing: joins can be emulated in application code, but that moves complexity into the application and is usually slower than a join performed by the database. For highly interconnected data, a graph model is the most natural. Kleppmann adds a caution about locality. The performance benefit of storing a whole document together applies only when the application needs large parts of the document at the same time. The database typically loads the entire document even if only a small portion is read, which is wasteful for large documents. On updates, the entire document usually has to be rewritten; only modifications that do not change the encoded size of a document can easily be performed in place. For these reasons, he recommends keeping documents fairly small and avoiding writes that increase their size. He also notes that the idea of grouping related data for locality is not unique to document databases: Google's Spanner offers the same property within a relational model, by allowing tables to be declared as nested within a parent table, and column-family stores in the Bigtable tradition achieve something similar. A related consideration is how the two models handle a field that is shared across many records. In a document database, a value such as a company name may be copied into every profile that mentions it. If the company is renamed, every copy must be found and updated, and any copy that is missed leaves the data inconsistent. In a normalized relational schema, the name is stored once and referenced by ID, so a rename is a single update. Denormalization is not wrong, and it is often a deliberate performance choice, but it transfers the job of keeping copies consistent from the database to the application. He also observes a convergence between the models. Relational databases such as PostgreSQL and MySQL added support for JSON documents, and some document databases added join-like features. A hybrid of the two is a sensible direction for databases to take. In the years since the book appeared, that convergence has continued, and it is now common for a relational database to store semi-structured JSON columns alongside normal tables. Schema-on-read versus schema-on-write Document databases are often called schemaless, but Kleppmann rejects the term as misleading. The code that reads the data usually assumes some structure; there is an implicit schema even if the database does not enforce it. A more accurate distinction is this. Schema-on-read means the structure of the data is implicit and interpreted only when the data is read. Schema-on-write means the schema is explicit and the database ensures all written data conforms to it, which is the traditional relational approach. He compares the difference to dynamic versus static type checking in programming languages, a debate with no single right answer. The practical difference shows up when the application changes the format of its data. Suppose names were stored as a single field and the application now wants first and last names separately. With schema-on-read, new code simply starts writing documents with the new fields and includes logic to handle old documents that lack them. With schema-on-write, one would typically run a migration that alters the table, adds the column and backfills existing rows. Schema changes have a reputation for being slow and requiring downtime. Kleppmann notes that this reputation is not entirely deserved: most relational databases execute an added column in milliseconds, although some older engines copied entire tables on certain alterations, and updating every row to backfill a value can be slow on a large table. Schema-on-read is advantageous when the items in a collection do not all share the same structure, for example when there are many different kinds of objects or when the structure is determined by external systems outside the organization's control. When all records are expected to share a structure, an explicit schema is a useful mechanism for documenting and enforcing it. From a management perspective, the choice is between flexibility now and discipline later. Schema-on-read allows a team to move quickly, but it moves the burden of handling historical data formats into every piece of code that reads the data. Schema-on-write centralizes that burden in migrations, which are more visible and easier to govern. Query languages: declarative and imperative When the relational model was introduced, it came with a new way of querying data. SQL is a declarative language: the user specifies the pattern of the data wanted, what conditions the results must meet and how they should be transformed, but not how to achieve that. An imperative language, by contrast, tells the computer to perform operations in a particular order, looping through records and testing each one. Kleppmann lists the advantages of declarative queries. They are more concise and easier to work with. They hide the implementation details of the database engine, which allows the engine to be improved without changing any queries. They lend themselves to parallel execution, because they specify only the pattern of results and not the algorithm, so the database is free to use a parallel implementation. He illustrates the idea with an analogy from the web: CSS selectors declaratively describe which elements should be styled, which is far simpler and more robust than writing imperative code to walk the document tree and change styles manually. He also discusses MapReduce querying, a programming model popularized by Google and supported in a limited form by some document databases. It sits between declarative and imperative styles: the query logic is expressed as snippets of code called repeatedly by the processing framework. Its usability problems led some databases to add more declarative query languages on top of it, which illustrates a recurring theme: declarative interfaces tend to win once they are available, because they separate what the user wants from how the system provides it. Graph-like data models When many-to-many relationships are very common, it becomes more natural to model data as a graph, consisting of vertices (nodes or entities) and edges (relationships or arcs). Social graphs, where vertices are people and edges indicate who knows whom, the web graph of pages and links, and road or rail networks are typical examples. Graphs are not limited to one kind of object: a single graph can store people, locations, events, check-ins and comments together, which Kleppmann notes is how Facebook maintains its data. He describes two main flavors. In the property graph model, used by systems such as Neo4j, each vertex and edge has a unique identifier, a label describing the relationship, and a collection of key-value properties. Any vertex can connect to any other, and one can efficiently traverse the graph forward and backward. This makes graphs highly evolvable: new kinds of relationships can be added without changing a schema. Kleppmann's example of querying such a graph uses Cypher, a declarative query language created for Neo4j, to find people who emigrated from the United States to Europe. Such a query follows chains of "within" edges of variable length, from a city to a state to a country to a continent, which is very awkward in SQL because the number of joins is not known in advance; SQL can express it only with recursive common table expressions, which are far more verbose. In the triple-store model, all information is stored as three-part statements of the form subject, predicate, object, such as (Jim, likes, bananas). Triple stores are closely associated with the semantic web and the RDF data model, which aimed to let websites publish machine-readable data. Kleppmann notes that the semantic web was overhyped in the early 2000s and has not been realized as its proponents imagined, but that the triple-store model remains a useful internal data model regardless. SPARQL is the query language for RDF triple stores. He also discusses Datalog, a much older logic-based query language that underlies several later systems and builds complex queries from small reusable rules. Kleppmann is careful to distinguish graph databases from the network model of the 1970s. In CODASYL, the schema specified which record types could be nested within which, access to records required following a path, and the ordering of children was fixed. Graph databases have none of those restrictions: any vertex can link to any other, vertices can be accessed directly by identifier or through an index, and query languages are declarative. Table 1 compresses the comparison of the three main models, which the discussion above has developed in detail. Table 1. Three general-purpose data models compared. Model Natural data shape Strengths Weaknesses Typical query style Relational Regular records with many-to-one and many-to-many links Joins, optimizer, mature tooling Mismatch with nested objects SQL (declarative) Document Self-contained trees of one-to-many data Locality, flexible schema Weak joins, duplication risk Document queries, aggregation pipelines Graph Highly interconnected entities Variable-length traversals, evolvability Specialized skills and tooling Cypher, SPARQL, Datalog The strategic lens The key managerial insight from this part of Kleppmann's book is that a data model is a bet about the future shape of the business's data. A startup that stores user profiles as documents because they are tree-shaped today may find, two years later, that its most valuable features depend on relationships between users, organizations and content. At that point it faces either an expensive migration or a growing mass of application code emulating joins. Conversely, forcing highly variable data, such as product catalogs with attributes that differ by category, into rigid relational tables can create accidental complexity. Three questions, drawn from the book's reasoning, help frame the decision. How interconnected will the data become as the product grows? Is the data usually read as a whole unit or combined in many different ways? Who controls the structure of incoming data, the organization or external parties? The answers do not dictate a product, but they narrow the choice considerably, and they explain why many enterprises end up running more than one kind of database. Key Takeaways · Data models shape how developers think about a problem; choosing one determines which operations are easy and which are hard. · Document models suit self-contained, tree-shaped data with one-to-many relationships; relational models suit data with many-to-one and many-to-many relationships. · The relational model won historically because its query optimizer freed developers from navigating access paths manually. · "Schemaless" is better described as schema-on-read; schema-on-write enforces structure at the database and pushes change into migrations. · Declarative query languages hide implementation details, allowing engines to optimize and parallelize without changing queries. · Graph models, in property-graph or triple-store form, are the natural choice for highly interconnected data. Review Questions 1. Explain the impedance mismatch and show how a document model reduces it for a résumé-style profile. 1. Why do many-to-many relationships weaken the case for a document database? 2. Compare schema-on-read and schema-on-write, and describe a business situation that favors each. 3. What advantages does a declarative query language offer over an imperative one? 4. Why is a variable-length traversal, such as finding everyone who lives somewhere within Europe, easier in a graph query language than in SQL? Chapter 3: Storage Engines and Data Encoding Application developers rarely write their own storage engines, but Kleppmann argues they still need a rough idea of what happens inside one. Choosing a storage engine appropriate to the workload, and tuning it, requires understanding what it is optimized for. The same is true of encoding, the question of how in-memory data is turned into bytes that can be written to a file or sent over a network. Both topics belong to the foundations of data systems because they determine performance on a single machine and, crucially, how easily a system can evolve over time. How databases store and find data Kleppmann begins with what he calls the world's simplest database: two shell functions, one of which appends a key and value to the end of a text file, and another which scans the file for the most recent value for a key. Writing is extremely efficient, because appending to a file is about the cheapest thing a disk can do. Many real databases internally use a log in the same sense: an append-only sequence of records. Reading, however, is terrible, because each lookup scans the entire file, so the cost grows in proportion to the amount of data. To find data efficiently, databases use an index, an additional structure derived from the primary data that acts as a signpost. Indexes speed up reads but slow down writes, because every write must also update the index. This is an important trade-off, and it is why databases do not index everything by default: the application developer or administrator chooses indexes based on the application's typical query patterns. Log-structured storage and LSM-trees The simplest indexing strategy for an append-only log is an in-memory hash map from each key to the byte offset of its latest value in the file. Kleppmann notes that this is essentially what Bitcask, the default storage engine in the Riak database, does. It works well when there are relatively few distinct keys, each updated frequently. To prevent the log from growing forever, it is broken into segments, and a background process performs compaction, discarding duplicate keys and keeping only the most recent value for each, then merging segments together. Append-only design has real advantages. Sequential writes are much faster than random writes on magnetic disks, and are also favorable on solid-state drives. Concurrency and crash recovery are simpler when segment files are immutable. Merging old segments avoids fragmentation. But a hash index must fit in memory, and it cannot efficiently answer range queries, such as all keys between two values. The next step is the Sorted String Table, or SSTable: a segment file in which the key-value pairs are sorted by key. Sorting makes merging segments efficient, similar to the merge step of the mergesort algorithm, and means the in-memory index can be sparse, holding only some keys, because the position of any other key can be found by scanning a small region between two known keys. How do records get sorted, when writes arrive in arbitrary order? Incoming writes are added to an in-memory balanced tree called a memtable. When the memtable grows beyond a threshold, it is written to disk as a new SSTable. To serve a read, the engine checks the memtable, then the most recent on-disk segment, then older ones. Background merging and compaction run periodically. To protect against losing the memtable in a crash, every write is also appended to a separate log on disk, which is used only for recovery. This design is known as a Log-Structured Merge-Tree, or LSM-tree, a name taken from a 1996 paper by Patrick O'Neil and colleagues. Kleppmann notes that storage engines based on this principle include LevelDB and RocksDB, and that similar engines are used in Cassandra and HBase, both inspired by Google's Bigtable paper. The same principle, keeping a sorted index and merging it in the background, also underlies the full-text search engine Lucene, used by Elasticsearch and Solr. Optimizations include Bloom filters, memory-efficient structures that can say with certainty that a key does not exist, avoiding needless disk reads, and different compaction strategies, such as size-tiered and leveled compaction. B-trees The most widely used indexing structure is quite different. B-trees were introduced in 1970 and, in Kleppmann's words, became ubiquitous within a decade; they remain the standard index in almost all relational databases and many non-relational ones. Like SSTables, they keep key-value pairs sorted by key, which supports efficient lookups and range queries. But rather than variable-size segments written sequentially, B-trees break the database into fixed-size pages, traditionally four kilobytes, and read or write one page at a time, a design that corresponds closely to the underlying hardware. Each page contains keys and references to child pages, with the keys marking the boundaries between ranges. Looking up a key starts at the root page and follows references down the tree until reaching a leaf page containing the value. The number of references per page, the branching factor, is typically several hundred, so the tree stays shallow: Kleppmann notes that a four-level tree of four-kilobyte pages with a branching factor of 500 can store up to 256 terabytes. Updating a value means finding its leaf page and overwriting it in place. Adding a key may require splitting a full page into two and updating the parent. Because B-trees overwrite pages in place, a crash in the middle of a page split could leave the tree corrupted. To guard against this, B-tree implementations use a write-ahead log (WAL), an append-only file to which every modification is written before it is applied to the pages. Concurrency also requires care, typically using lightweight locks called latches. Comparing the two Kleppmann offers rules of thumb while cautioning that benchmarks for a specific workload are the only reliable guide. LSM-trees are typically faster for writes, whereas B-trees are thought to be faster for reads, because an LSM-tree read may have to check several structures. LSM-trees usually achieve lower write amplification, meaning one logical write results in fewer physical writes over the database's lifetime, which matters especially on solid-state drives that can tolerate only a limited number of overwrites. They also compress better and produce smaller files on disk. On the downside, compaction can interfere with ongoing reads and writes, producing unpredictable high-percentile response times, and if compaction cannot keep up with a high write rate, unmerged segments accumulate until disk space runs out. B-trees offer a strong advantage for transactional semantics, because each key exists in exactly one place in the index, which makes it straightforward to attach locks to ranges of keys. He also covers other index structures briefly: secondary indexes, which are essential for joins and can be built with either approach; clustered indexes, which store the row data within the index itself, versus heap files that store rows elsewhere; multi-column indexes, including specialized spatial indexes; full-text and fuzzy indexes; and in-memory databases, whose performance advantage, he points out, comes less from avoiding disk reads than from avoiding the overhead of encoding in-memory structures into a form that can be written to disk. Transaction processing versus analytics A second major distinction in Kleppmann's treatment of storage is between two patterns of use. In the early days of business data processing, a write to the database typically corresponded to a commercial transaction, such as a sale or a payment. The term transaction stuck, even though databases came to be used for many other kinds of data. The access pattern of an interactive application, looking up a small number of records by key and inserting or updating records based on user input, is called online transaction processing (OLTP). Databases are also increasingly used for data analytics, which has a very different access pattern. An analytic query scans over a huge number of records, reads only a few columns per record, and calculates aggregate statistics such as counts, sums or averages. A question such as total revenue per store in January, or how many more bananas than usual were sold during a promotion, is analytic. This pattern is called online analytic processing (OLAP). The differences are summarized in Table 2. Table 2. OLTP and OLAP workloads compared. Property OLTP OLAP Main read pattern Few records, fetched by key Aggregates over many records Main write pattern Random-access, low-latency writes Bulk import (ETL) or event stream Primary users Customers via a web application Internal analysts for decision support Data represents Latest state of data History of events over time Typical dataset size Gigabytes to terabytes Terabytes to petabytes Typical storage layout Row-oriented with B-tree or LSM indexes Column-oriented and compressed Data warehousing In the 1980s and 1990s, companies increasingly stopped running analytics on their OLTP systems and moved them to a separate database called a data warehouse. The reasoning is straightforward. OLTP systems are expected to be highly available and to process transactions with low latency, because they are critical to running the business. Analysts running expensive ad hoc queries on them could harm performance for customers. A data warehouse contains a read-only copy of data from the various OLTP systems in the company. Data is extracted from those systems, transformed into an analysis-friendly schema, cleaned up and loaded into the warehouse. This process is called Extract-Transform-Load (ETL). Many warehouses use a star schema, also known as dimensional modeling. At its center is a fact table, each row representing an event, such as one customer's purchase of one product at a particular time. Fact tables can become extremely large. Some columns are attributes, such as the price; others are foreign keys referencing dimension tables, which describe the who, what, where, when, how and why of the event: the product, the store, the date, the customer, the promotion. When dimension tables are further normalized into sub-dimensions, the result is called a snowflake schema. Column-oriented storage Fact tables are often over 100 columns wide, yet a typical analytic query touches only four or five of them. In most OLTP databases, storage is laid out in a row-oriented fashion: all values from one row are stored next to each other. For an analytic query, the engine must load every row, with all its columns, from disk into memory, parse it, and filter out the rows that do not meet the conditions. That is slow. Column-oriented storage stores all the values from each column together instead. If each column is stored in a separate file, a query needs to read and parse only the columns it uses. The columns must all store rows in the same order, so that the twenty-third entry in each column file belongs to the same row. Kleppmann notes that the Parquet format, based on Google's Dremel, is a columnar storage format. Column storage lends itself to compression, because the values in one column are often repetitive. A technique called bitmap encoding can represent a column with a small number of distinct values as a set of bitmaps, one per value, which can then be compressed with run-length encoding. Such bitmaps also allow certain queries to be answered with fast bitwise operations. Beyond reducing disk reads, compact columns make efficient use of processor caches and allow vectorized processing, in which the engine operates on chunks of compressed column data in tight loops. Sorting rows in a column store can further help compression and queries, and some systems even store the same data sorted in several different ways on different replicas. Writes are harder in column stores, because inserting a row in the middle of a sorted, compressed column would require rewriting every column file. The solution is the LSM-tree approach: accumulate writes in memory and merge them into the column files in bulk. Warehouses also use materialized views and data cubes, precomputed aggregates that make certain queries very fast at the cost of flexibility. This guide adds a note on context. Since the book's publication, cloud data warehouses and open table formats have made columnar analytics widely available as managed services, and the separation of storage from compute has become common. The underlying principles Kleppmann describes, column orientation, compression and the separation of analytic from transactional workloads, remain the reason those services perform as they do. Encoding and evolution Applications change over time. Kleppmann connects this to the evolvability goal from the book's first chapter. When a feature changes, the data it stores usually changes too. In large applications, however, code changes cannot happen instantaneously. Server-side applications are updated through rolling upgrades, deploying the new version to a few nodes at a time, and client-side applications depend on users choosing to install updates. Old and new versions of code, and old and new data formats, therefore coexist. This requires compatibility in both directions. Backward compatibility means newer code can read data written by older code. Forward compatibility means older code can read data written by newer code. Backward compatibility is usually not hard, because the author of new code knows the old format. Forward compatibility is trickier, because it requires older code to ignore additions made by a newer version. Formats for encoding data Programs work with data in memory as objects, structures, lists and hash tables. To write data to a file or send it over a network, it must be converted into a self-contained sequence of bytes. That translation is called encoding (also serialization or marshalling), and the reverse is decoding. Kleppmann warns against language-specific formats such as Java's built-in serialization or Python's pickle. They tie data to one programming language, often create security problems because decoding can instantiate arbitrary classes, treat versioning as an afterthought, and are frequently inefficient. Textual formats such as JSON, XML and CSV are widely supported and somewhat human-readable, but have subtle problems: ambiguity around numbers, with JSON not distinguishing integers from floating-point values and losing precision on very large numbers; no native support for binary strings; optional and complex schema support; and, in the case of CSV, no schema at all. They remain good enough for many purposes, especially for data interchange between organizations, where the difficulty of getting everyone to agree on any format outweighs most other concerns. Binary formats with schemas For internal use, binary formats with schemas offer compactness and clearer evolution rules. Kleppmann examines three. Apache Thrift, originally developed at Facebook, and Protocol Buffers, developed at Google, both require a schema that defines each field with a name, a type and a numeric field tag. The encoded data refers to fields by tag number rather than by name, which is compact. Evolution follows from the tags. A new field can be added with a new tag number; old code that does not recognize the tag simply skips it, giving forward compatibility. New code can read old data as long as each tag keeps its meaning, giving backward compatibility, provided that any newly added field is optional or has a default value. Field names can change freely, but tag numbers cannot. Removing a field is the mirror image: only optional fields can be removed, and the tag number must never be reused. Apache Avro, which began as a subproject of Hadoop, takes a different approach. Its encoding contains no tag numbers or field identifiers at all; it is simply values concatenated together. Decoding therefore requires the exact schema the data was written with. Avro resolves this by distinguishing the writer's schema from the reader's schema. The two need not be identical, only compatible; the Avro library resolves differences by matching fields by name, ignoring fields the reader does not know, and filling in default values for fields the writer did not provide. The reader must somehow know the writer's schema: in a large file, it is included once at the beginning; in a database, each record can carry a schema version number that refers to a registry. Because Avro schemas contain no tag numbers, they are friendlier to dynamically generated schemas, such as dumping the contents of a relational database whose columns change over time. Kleppmann summarizes the value of schemas: they are more compact than textual formats, they serve as valuable documentation that is guaranteed to stay current, a database of schemas lets teams check compatibility before deploying, and for statically typed languages, code can be generated from them. Modes of dataflow Finally, Kleppmann considers the ways encoded data flows between processes. Through databases, the process writing encodes data and a process reading, possibly much later, decodes it; Kleppmann notes the principle that data outlives code, since a database may contain values written years ago in older formats. A subtle hazard arises when older code reads a record containing a new field, updates it and writes it back, discarding the field it did not understand. Through services, using REST or remote procedure calls, clients and servers must agree on an API, and since servers are usually updated before clients, requests need backward compatibility and responses need forward compatibility. He criticizes RPC frameworks for pretending that a network call is like a local function call, when in fact network calls can time out, be retried, have unpredictable latency and require encoding of all parameters. Through asynchronous message passing, via a message broker, the sender does not wait for the recipient; the broker acts as a buffer, can redeliver messages to a crashed process, decouples senders from recipients, and allows one message to reach several consumers. The actor model is a related approach. The strategic lens For a manager, this chapter carries two practical lessons. First, the separation of OLTP from OLAP is not a technology fashion but a structural consequence of different access patterns; systems built to serve customers in milliseconds and systems built to scan billions of rows are optimized in opposite ways. Proposals to run heavy analytics directly on production databases deserve scrutiny. Second, encoding decisions govern the organization's ability to change. A team that chooses a format with explicit schemas and clear compatibility rules can deploy independently and roll out changes gradually. A team that relies on ad hoc JSON with no compatibility checks will eventually break a consumer it did not know existed. Key Takeaways · An index speeds reads but slows writes; choosing indexes is a deliberate trade-off based on query patterns. · LSM-trees append and merge sorted segments, favoring write throughput and compression; B-trees update fixed-size pages in place, favoring predictable reads and simple locking. · OLTP serves many small, key-based requests from users; OLAP scans large volumes for aggregates, which is why warehouses are separated from operational databases. · Column-oriented storage reads only the columns a query needs and compresses well, making it the standard for analytics. · Backward compatibility lets new code read old data; forward compatibility lets old code read new data. Both are needed during rolling upgrades. · Thrift and Protocol Buffers rely on stable field tags; Avro resolves differences between writer's and reader's schemas by field name. Review Questions 1. Explain how an LSM-tree handles a write, a read and a crash. 1. Why do B-trees need a write-ahead log? 2. Using the distinctions in Table 2, explain why a retailer should not run quarterly revenue analysis on its checkout database. 3. Why is column-oriented storage faster than row-oriented storage for analytic queries? 4. Define backward and forward compatibility and explain why rolling upgrades require both. 5. How does Avro achieve schema evolution without field tag numbers? Hashtags: #TheArchitectureOfData #DesigningDataIntensiveApplications #MartinKleppmann #DataIntensiveSystems #Reliability #Scalability #Maintainability #FaultTolerance #TailLatency #DataModels #RelationalDatabases #DocumentDatabases #GraphDatabases #SchemaOnRead #SchemaOnWrite #LSMTrees #BTrees #OLTP #OLAP #ColumnOrientedStorage #DataEncoding #Replication #Partitioning #DistributedSystems #FutureOfDataArchitecture

Download the Book (PDF): Introduction Weighing in at over a thousand pages, Ross Anderson's encyclopedic textbook is the ultimate authority on cybersecurity. It covers everything from the mathematics of cryptography to the psychology of social engineering and the physics of hardware tampering. For an IT management student, attempting to memorize this massive, highly technical volume before a final exam is overwhelming and practically impossible. This book is your strategic shortcut. It is designed specifically to explain Security Engineering, synthesizing the exhaustive textbook into focused, exam-ready modules. It isolates the core principles of threat modeling, access control and distributed systems architecture, explaining the engineering intuition without the paralyzing depth of the advanced mathematics. It cuts out the technical filler, leaving you with the operational knowledge you need to master your security modules. The book and its author Security Engineering: A Guide to Building Dependable Distributed Systems was first published by Wiley in 2001. A second edition followed in 2008 and a third in 2020. The third edition was released chapter by chapter as drafts for comment from 2019 before appearing in print, and its author later made the full text freely available online. This guide is based mainly on that third edition, which reorganizes and substantially updates the earlier material to cover smartphones, cloud services, machine learning, the cybercrime economy and the security of cars and medical devices. Where the guide refers to the book's structure or arguments, it means the third edition unless it says otherwise. Ross Anderson was one of the most influential figures in computer security. He spent most of his career at the University of Cambridge Computer Laboratory, where he became Professor of Security Engineering in 2003, and from 2021 he also held a chair at the University of Edinburgh. He was a Fellow of the Royal Society and of the Royal Academy of Engineering. His research ranged across cryptographic protocols, hardware tamper resistance, payment systems, medical privacy and information hiding, and he is widely credited with founding the field of security economics: he helped launch the Workshop on the Economics of Information Security, first held in 2002, which remains the main meeting place for that discipline. He also founded the Foundation for Information Policy Research, a think tank on technology and civil liberties, and campaigned for decades on patient privacy, surveillance and the treatment of victims of bank fraud. Anderson died in March 2024 at the age of 67. His book is now his most widely read legacy. What made the book unusual, and what still makes it valuable, is its refusal to treat security as a branch of mathematics or a product category. Anderson wrote as an engineer who had spent years investigating how real systems failed: cash machines that paid out to fraudsters, smartcards whose secrets leaked through their power consumption, hospital systems that exposed patient records to anyone with a login, and protocols that looked sound on paper but collapsed when one message was replayed. His recurring observation was that systems usually fail not because the cryptography is broken but because of poor design, misplaced trust, confused requirements, careless implementation, or incentives that reward the wrong people. The book therefore draws on computer science, economics, psychology, law and history in equal measure, and it teaches mainly through cases. The controlling idea of this guide The single idea this guide keeps in view is that security is a property of whole systems, shaped by people and incentives, and it is engineered through explicit choices about policy, mechanism, assurance and incentive rather than bought as a feature. Every chapter returns to that idea from a different direction. A cipher is only as useful as the protocol around it; a protocol only as strong as the access controls that protect its keys; access controls only as good as the humans who configure and use them; and the whole arrangement only as durable as the economic incentives that decide whether anyone will maintain it. An IT manager who grasps this can reason about a system they have never seen before, which is what examiners and employers actually test. This is also why the guide keeps the mathematics intuitive. Anderson himself makes the point that most practitioners will never design a cipher and should not try; what they need is a clear picture of what each tool promises, what it does not promise, and how it tends to go wrong in practice. The guide gives that picture in words, and it avoids formulas and code entirely. How the guide is organized Anderson divides his book into three parts. The first lays the foundations: what security engineering is, who the opponents are, how human psychology shapes security, how protocols and cryptography work, how access control is enforced, how distributed systems behave under failure and attack, and why economics explains so much of what goes wrong. The second part applies those foundations to real systems, from military classification schemes and bank bookkeeping through physical locks, smartcards, biometrics, side channels, phones, networks and electronic warfare. The third part turns to policy and management: surveillance and privacy, how secure systems are developed, how assurance is gained and maintained, and what society should do about systems that increasingly make decisions about people. This guide follows the same arc but compresses it into ten chapters. The foundations receive the most space, because they are the concepts that transfer to every other topic and they are what security modules examine most heavily. The guide opens with the framework that organizes the whole discipline and then works through the opponent, the human user, protocols and cryptography, access control, distributed systems and economics, each in its own chapter. It then gives two chapters to the applied material, grouping Anderson's many case chapters around the lessons they share: first the security policies that govern information flows in military, medical and financial systems, and then the physical, hardware and network attacks that break systems in the field. The final chapter covers the management material that closes the book: privacy and surveillance, development methods, assurance, certification and the long-term sustainability of systems that must be patched for decades. Along the way the guide uses several well-documented real incidents to show the principles at work. Some come from Anderson's own research, such as the Cambridge team's demonstration that a card payment could be completed without the correct PIN. Others, such as the Post Office Horizon scandal, the WannaCry ransomware outbreak that disrupted the National Health Service, the SolarWinds supply-chain compromise and the Log4Shell vulnerability, are used as the guide's own illustrations of the book's arguments; where an example goes beyond what Anderson wrote, the text says so. Each chapter closes with a short list of key takeaways and a set of review questions. The takeaways state the points most worth remembering; the questions are written in the style of exam essay prompts and short-answer items, and the best way to use them is to answer them aloud or on paper before checking the chapter again. A glossary at the end defines the terms that recur most often, and a short annotated reading list points to the original book and to the classic papers behind its most important ideas. A final word on method. Anderson liked to compare security engineering to medicine: a doctor learns anatomy and physiology, but becomes competent by studying many cases and learning to recognize patterns of failure. Read this guide the same way. Learn the principles, but pay close attention to the stories, because in an exam, and in practice, the ability to say why a particular system failed and what would have prevented it is worth more than any list of definitions. Chapter 1: What Security Engineering Is Security engineering, in Anderson's definition, is about building systems that remain dependable in the face of malice, error or mischance. The phrase is deliberately broad. It covers the obvious threat of a hostile attacker, but also the clerk who makes a mistake, the software update that goes wrong, and the unlucky coincidence that no designer anticipated. As a discipline, it concentrates on the tools, processes and methods needed to design, implement and test complete systems, and to adapt existing systems as their environment changes. The emphasis on complete systems is the key. A student who has taken a course in cryptography may think of security as a matter of choosing strong algorithms. A student from a networking background may think of it as firewalls and intrusion detection. Anderson's argument is that neither view is sufficient, because real failures almost always happen at the joins: between the cryptography and the protocol, between the protocol and the user, between the technical design and the organization that runs it. Security engineering is the practice of looking at those joins deliberately. The four-part framework The organizing idea of the book is a framework of four interacting elements. Anderson presents them as the things a good security engineer needs to get right, and they recur, explicitly or implicitly, in almost every chapter. Policy is what you are supposed to achieve. It states who may do what, which information must be protected, from whom, and what counts as a breach. A policy may be a formal model, a short statement of rules, or a set of legal obligations. Without a clear policy, a system cannot be said to be secure or insecure, because there is nothing to measure it against. Mechanism is what you use to achieve the policy: the ciphers, access controls, hardware tamper resistance, audit logs, locks, guards and procedures that enforce the rules. Mechanisms are where most technical effort goes, and they are what most security products sell. Assurance is the amount of reliance you can place on each mechanism. A mechanism that exists on paper but has never been tested gives little assurance; one that has been independently evaluated, formally verified or battle-tested over years gives more. Assurance answers the question of how you know the mechanisms really do what they claim, and how well they will keep doing it. Incentive is the motive that the people guarding and maintaining the system have to do their job properly, and also the motive that attackers have to try to defeat it. Anderson places incentive alongside the technical elements because experience showed him that systems frequently fail when the people responsible for security do not bear the cost of its failure. These four elements interact. A strong mechanism with a vague policy protects the wrong thing. A clear policy with weak assurance cannot be relied upon. And any of them can be undermined by incentives. Anderson notes that much of the security industry is shaped by the incentive to be seen to be doing something rather than to reduce actual risk, a pattern Bruce Schneier named "security theatre". Airport screening, password rules that force frequent changes, and elaborate compliance checklists are the classic examples: they are visible, they reassure, and they may do little to stop a determined opponent. The framework gives the student a way to ask, of any control, whether it serves the policy, whether its assurance is real, and whose incentives it serves. A useful exam habit is to analyse any case study through these four lenses in turn. What was the policy, and was it written down? Which mechanisms enforced it? What evidence existed that they worked? And who gained or lost when they failed? Almost every failure in the book can be explained by a gap in at least one of the four. Four example settings Anderson grounds the framework in four familiar settings, chosen because each has a different dominant concern. They are worth knowing well, because they show that "security" means different things in different places. A bank. Banks worry above all about the integrity of their records and the prevention of fraud. The core bookkeeping system must ensure that money cannot be created or destroyed except through authorized, logged transactions. Around it sit cash machines, card networks, online banking and mobile apps, each with its own threats. Confidentiality matters, but a bank's deepest fear is that its ledger cannot be trusted. Banks also illustrate the incentive problem: where a bank can shift the cost of fraud onto customers or merchants, it has less reason to prevent it. A military base. Here the traditional concern is confidentiality: keeping classified information from the enemy. Military systems gave rise to the classic multilevel security models, in which documents and people carry labels such as Confidential, Secret and Top Secret, and information must not flow from a higher level to a lower one. Military settings also care intensely about availability, since communications that fail in combat can be fatal, and about authenticity, since a forged order is worse than no order. A hospital. Hospitals face a sharper conflict between security goals. Patient records must be kept confidential, but they must also be available immediately to clinicians in an emergency, and they must be accurate, because an error can kill. Safety and security interact closely: a drug-dispensing system or an infusion pump must resist tampering, but it must never lock out a nurse at a critical moment. Hospitals also involve many parties with different interests, including patients, clinicians, administrators, insurers and researchers, which makes their policies more complicated than a simple hierarchy. A home. The home now contains networked devices of every kind: routers, phones, smart speakers, televisions, thermostats, cars. The householder is not a security professional, has no security staff, and often cannot update or even see the software in the devices. Threats range from remote criminals who compromise devices in bulk to intimate threats such as an abusive partner who uses shared accounts or tracking apps. The home setting shows that security has to work for ordinary people without expert help, and that many of the most important threats are not technical at all. Taken together, the four settings teach that there is no single meaning of "secure". A system is secure relative to a policy, and the policy depends on the setting, the assets and the people involved. The vocabulary of the discipline Much confusion in security arises from loose terminology, and Anderson spends time pinning down terms. The most important distinctions are these. A system can mean a product or component, a collection of components including an operating system and communications, the same plus applications, or all of that plus the people who use and run it. Anderson's view is that the last, widest meaning is usually the right one for security analysis, because many failures happen when designers draw the boundary too narrowly and leave people out. A subject is a physical person or process acting in the system; a principal is an entity that participates in a security system and can be held responsible, such as a person, a role, a piece of equipment, or a group. A person is a human being; an identity is a correspondence between the names of two principals, such as between a person and a login name. Getting these right matters: many access-control and audit failures arise because a single login is shared by several people, so the system cannot tell who actually did something. Trust and trustworthiness are different. A trusted system or component is one whose failure can break the security policy. A trustworthy one is one that will not fail. The goal of good design is to minimize what must be trusted, and to make what remains trustworthy. This distinction explains why security engineers try to keep the "trusted computing base", the set of components whose correctness the security depends on, as small as possible. Secrecy, confidentiality and privacy overlap but differ. Secrecy is the broad engineering term for mechanisms that limit who can access information. Confidentiality is an obligation to protect someone else's secrets that you hold. Privacy is the ability and right of people to protect their own personal information, extending to protection from intrusion into their personal lives. Integrity means that information has not been altered in an unauthorized way, while authenticity can mean integrity combined with freshness, or can refer to whether a message really comes from the claimed source. Availability means the system works when needed. The trio of confidentiality, integrity and availability is often taught as the "CIA triad"; Anderson treats it as a starting point rather than a complete account, since many real policies care about accountability, non-repudiation, safety and fairness as well. Finally, the book distinguishes a vulnerability (a property of the system or its environment that, with an internal or external threat, can lead to a failure), a threat (a circumstance with potential to cause harm), an error (a design flaw or deviation from intended behaviour) and a failure (the actual departure from required behaviour). Keeping these apart helps with incident analysis: a vulnerability may exist for years without failure, until a threat actor finds it. Security and dependability The book's subtitle refers to dependable distributed systems, and the choice of word is significant. Dependability is a concept from safety and reliability engineering that covers the ability of a system to deliver a service that can justifiably be trusted. Anderson treats security as one aspect of dependability, alongside reliability, safety and availability. The practical consequence is that many techniques from safety engineering, such as hazard analysis, fault trees, redundancy and careful change control, carry over into security, and that the two fields increasingly need to be managed together. The difference between them is the adversary. A safety engineer worries about random failures and honest mistakes, which follow statistical patterns. A security engineer must assume that someone is actively searching for the weakest point and will exploit it deliberately, often in ways that are rare under normal conditions. A bridge does not redesign the wind to find its weakness; an attacker does exactly that to a system. This is why security testing cannot rely only on typical use cases, and why the book keeps returning to the need to think like an opponent. Cars, medical devices and industrial control systems have made this convergence urgent. A vehicle that can be hacked remotely is a safety problem as well as a security problem, and a regulator certifying its safety must now also ask about software patching and resistance to attack. Anderson's later work, discussed in the final chapter of this guide, focused on exactly this meeting point. Why systems fail: the recurring lessons The examples in the book yield several lessons that recur so often they are worth stating at the start. First, most failures are not cryptographic. Anderson's early research on cash machine fraud found that losses came overwhelmingly from implementation errors, poor procedures and insider abuse, not from anyone breaking the encryption. That finding, published in the early 1990s as a paper on why cryptosystems fail, shaped his whole approach. Second, designers tend to protect the wrong things. It is easy to spend money guarding against dramatic, well-publicized threats while ignoring mundane ones, such as the help-desk operator who resets a password for anyone who sounds convincing. Third, security and usability are not opposites but they must be designed together. Controls that are too burdensome get bypassed, and a bypassed control is worse than none because it creates a false sense of protection. Fourth, complexity is the enemy of security. Every added feature, interface or dependency is a new place for things to go wrong, and large systems accumulate them. Fifth, environments change. A system designed for a closed network becomes exposed when connected to the internet; a threat model that was reasonable ten years ago may be dangerously out of date. Security is therefore never finished, a point the book makes emphatically in its closing chapters. These lessons are why Anderson describes security engineering as needing breadth. The practitioner needs to understand enough cryptography to avoid misusing it, enough psychology to design for real users, enough economics to see where incentives point, and enough law and management to make changes stick. From framework to practice: threat models and risk The framework tells an engineer what to think about; a threat model is how that thinking is written down for a particular system. A threat model identifies the assets worth protecting, the opponents who might want them, the ways those opponents could get at them, and the consequences if they succeed. Anderson stresses that the threat model must come before the choice of mechanisms, and that the most common design error is to solve a problem different from the one the system actually faces. A payment system designed against outside hackers, for instance, may be wide open to its own staff. A practical way to build a threat model is to work backwards from the harm. Start with the outcome that would matter most, such as money leaving an account without authority or a patient record reaching a journalist, and ask what would have to happen for it to occur. Each answer becomes a branch, and each branch can be broken down further until it reaches concrete actions an attacker could take. This structure, often called an attack tree, makes it easier to see which single controls block many branches and which branches are left entirely undefended. It also reveals the cheapest path for the attacker, which is usually the one that will be used. Risk management then asks how much to spend. Traditional approaches multiply the likelihood of an event by its impact and compare the result with the cost of controls. Anderson treats such calculations with caution. Likelihoods for deliberate attacks are hard to estimate, because an intelligent opponent changes behaviour in response to defences, and rare catastrophic events are easy to underweight. Figures produced by risk-scoring exercises can give an air of precision that the underlying data do not support. The sensible use of risk assessment is to force a structured conversation about priorities, not to generate numbers that are then treated as facts. Two further ideas complete the practical toolkit. The first is defence in depth: arranging several independent controls so that the failure of one does not expose the asset. A bank vault has walls, a door, a time lock, alarms and guards; a well-designed network has perimeter controls, internal segmentation, strong authentication, monitoring and backups. The second is resilience: accepting that some attacks will succeed and designing so that they can be detected, contained and recovered from. The shift of emphasis from pure prevention towards detection and recovery runs through the modern practice the third edition describes, and it reflects the experience of an industry that has learned that perfect prevention is not achievable. Key Takeaways · Security engineering is about building systems that stay dependable despite malice, error or mischance, and it deals with whole systems including their people. · Anderson's framework has four interacting elements: policy (what you want), mechanism (how you enforce it), assurance (how much you can rely on it) and incentive (who is motivated to make it work or to break it). · The meaning of "secure" depends on the setting; banks, military bases, hospitals and homes each prioritize different properties. · Precise vocabulary matters: trusted is not the same as trustworthy, and confidentiality, privacy and secrecy are distinct ideas. · Security is one aspect of dependability; it differs from safety chiefly because an adaptive adversary seeks out the weakest point. · Most real failures come from design, implementation, procedure and incentive problems rather than broken cryptography. Review Questions 1. Explain Anderson's four-part framework and show how a failure in incentive can defeat a strong mechanism, using an example of your choice. 1. Compare the dominant security concerns of a bank and a hospital, and explain why a single generic security policy would not serve both. 2. Distinguish between a trusted component and a trustworthy one. Why do designers try to minimize the trusted computing base? 3. What is "security theatre"? Give two examples and assess what, if anything, they contribute to real security. 4. Why does Anderson treat security as an aspect of dependability, and what makes security engineering harder than conventional safety engineering? Chapter 2: Who Is the Opponent? A security design is only as good as its picture of the adversary. Engineers who imagine a lone teenage hacker will build different defences from those who imagine a national intelligence agency, and both will be wrong if the real threat is a disgruntled employee or an organized fraud gang. Anderson devotes an early chapter of the third edition to this question, and his answer is deliberately concrete. Rather than discussing abstract "threat actors", he describes the main communities of attackers as they actually operate, grouping them into four broad classes: spies, crooks, geeks and what he calls the swamp. The purpose of this classification is practical. Each class has different goals, resources, time horizons and tolerance for risk, and each therefore attacks in different ways. Knowing which of them you face tells you what to defend and how hard. The four classes are compared in Table 1, and each is discussed in turn below. Table 1. Anderson's four classes of opponent. Class Typical actors Main goals Characteristic methods Spies State intelligence and military agencies Intelligence, sabotage, influence Bulk collection, targeted intrusion, supply-chain compromise Crooks Organized cybercriminals, fraudsters, insiders Money Phishing, malware, ransomware, card fraud Geeks Researchers, hobbyists, academics Knowledge, reputation, fixing flaws Reverse engineering, vulnerability research, disclosure The swamp Harassers, extremists, abusers, hacktivists Harm, control, attention Doxxing, stalkerware, bullying, disinformation Spies The first class consists of state actors. Anderson discusses the Five Eyes alliance of the United States, the United Kingdom, Canada, Australia and New Zealand in some detail, drawing on the documents leaked by Edward Snowden in 2013, which revealed the scale of signals intelligence collection by the US National Security Agency and Britain's GCHQ. These programs included bulk collection of communications metadata, tapping of undersea cables, the compromise of network equipment, and efforts to weaken cryptographic standards. Anderson also describes the capabilities of other states, notably China, Russia, North Korea and Iran, each with its own style and priorities: large-scale theft of intellectual property, political interference, financially motivated operations to fund a sanctioned regime, and regional sabotage. What distinguishes spies is resources and patience. A state agency can spend years on a single target, develop custom tools, buy or discover previously unknown vulnerabilities (zero-days), and compromise suppliers in order to reach their customers. The practical lesson is that no ordinary organization can expect to keep out a determined, well-resourced state attacker indefinitely. The realistic aim is to make attacks expensive, to detect intrusions quickly, and to limit what an intruder can reach. Two widely documented cases, used here as illustrations of the book's argument rather than drawn from its text, show the spy class at work. Stuxnet, discovered in 2010, was a sophisticated worm that targeted the Siemens industrial controllers running uranium enrichment centrifuges at Iran's Natanz facility. It is widely reported to have been a joint American and Israeli operation, though neither government has formally confirmed this. Stuxnet crossed an "air gap" by spreading through removable media, used several zero-day vulnerabilities, and manipulated the centrifuges while showing operators normal readings. It demonstrated that software could cause physical destruction and that isolated networks are not immune. The SolarWinds compromise, disclosed in December 2020, showed the supply-chain route. Attackers inserted malicious code into updates for SolarWinds' Orion network-management software, which was then distributed through the vendor's normal update channel. The company said that up to around 18,000 customers had installed the affected updates, although the attackers went on to exploit only a much smaller number of high-value targets, including several US government departments. In April 2021 the US and UK governments attributed the operation to Russia's foreign intelligence service, the SVR. The lesson for security engineers is that trusting a supplier's update mechanism means trusting the supplier's own security, a dependency that most organizations had never modelled. Crooks The second class is criminal, and it is by far the most common threat most organizations face. Anderson describes cybercrime as an industry with a mature supply chain. Some criminals write malware; others rent out networks of compromised machines, known as botnets; others sell stolen card data, run phishing campaigns, launder money through "money mules", or provide hosting that ignores abuse complaints. This specialization matters because it lowers the skill needed to commit crime: an attacker can buy most of the components as services. Anderson's work at the Cambridge Cybercrime Centre, which he founded to collect and share data on online crime, informs this part of the book. Its central observation is that most online crime is volume crime, run for profit by people who will attack whoever is easiest. Criminals behave economically. They go where the returns are highest and the risks lowest, and they abandon methods that stop paying. This is good news for defenders, because it means that raising the cost of attack even modestly can push criminals elsewhere, whereas a spy will often simply spend more. Ransomware has become the most damaging form of this crime. Criminals encrypt a victim's data and demand payment for the key, often also threatening to publish stolen data. The WannaCry outbreak of May 2017 is a well-documented example. WannaCry spread automatically across networks by exploiting a vulnerability in older versions of Microsoft Windows, using an exploit known as EternalBlue that had been developed by the NSA and later leaked. Microsoft had released a patch two months earlier, but many systems had not applied it. The attack spread across the world within hours and was halted largely by chance, when a researcher registered a web domain that the malware checked and that acted as a "kill switch". The US and UK governments later attributed WannaCry to North Korea, which shows that the lines between spies and crooks can blur when a state uses criminal methods for revenue or disruption. The effect on the English National Health Service is examined in Chapter 9, where it serves as a case study in patching and assurance. Here the point is that WannaCry was not a targeted attack. The NHS was hit because it ran large numbers of unpatched machines, not because anyone chose it. Most victims of crime are selected by their weakness, not their identity. Insiders form an important subgroup of crooks. Employees and contractors already hold legitimate access, understand the procedures, and know where the valuable assets lie. Anderson's early research on cash machine fraud found that insider abuse was a major source of loss. Controls against insiders, such as separation of duties, dual control, audit trails and rotation of staff, are among the oldest security mechanisms in existence, and they remain among the most effective. Geeks The third class is the researchers and enthusiasts who probe systems out of curiosity, for reputation, or to improve security. Anderson counts himself and his students among them. Geeks reverse engineer products, find vulnerabilities and publish their findings. They are the reason many flaws come to light at all. The relationship between geeks and vendors has long been tense. Firms often prefer that flaws stay hidden, while researchers argue that public disclosure is the only reliable way to force fixes. The now-standard compromise is coordinated (or responsible) disclosure: the researcher reports the flaw privately, the vendor has a fixed period to fix it, and details are published afterwards. Bug bounty programs, in which firms pay for vulnerability reports, have made much of this relationship commercial. Anderson's own group was repeatedly involved in disputes over publication, including an attempt by the UK banking industry to suppress a student's thesis on payment system weaknesses in 2010, which the university refused. Geeks matter to the defender in two ways. They are a source of warnings that should be taken seriously, and the tools and techniques they publish are soon picked up by crooks. A vulnerability announced on Tuesday may be exploited at scale by Friday. The swamp The fourth class is the most varied. Anderson uses the term "the swamp" for a range of actors whose aims are neither espionage nor straightforward profit: hacktivists who attack organizations for political reasons, extremists who use online platforms to recruit and coordinate, people who distribute child abuse material, cyberbullies, and those who harass and stalk. Many of these harms are carried out with little technical skill. A stalker may need nothing more than access to a partner's phone or a shared cloud account. The swamp is important for two reasons. First, it shows that the most serious harms to individuals often come from people who know them. Intimate partner abuse using location tracking, stalkerware, smart home devices and shared accounts is a growing area of research, and it defeats threat models that assume the attacker is a stranger without physical access. Second, the swamp creates pressure for content moderation and surveillance, which in turn raises the privacy and civil liberties questions discussed later in the book. Anderson was consistently wary of proposals, such as mandated scanning of private messages, that justify broad surveillance by reference to these harms. How attacks unfold Whatever the class of attacker, serious intrusions tend to follow a recognizable sequence, and understanding it helps defenders decide where to place controls. The sequence is not Anderson's own formulation; it is the common industry model, often called the kill chain and elaborated in public catalogues of attacker techniques such as MITRE ATT&CK. It fits the cases the book describes closely. The attacker begins with reconnaissance: learning about the target's staff, suppliers, technology and exposed services. Much of this uses public information, such as company websites, job advertisements that reveal which software a firm uses, and professional networking profiles that identify who has administrator access. Next comes initial access, most often through a phishing email, stolen or guessed credentials, an unpatched internet-facing system, or a compromised supplier. Once inside, the attacker seeks persistence, installing a way back in that survives reboots and password changes, and then privilege escalation, moving from an ordinary user account to one with administrative rights. Lateral movement follows, as the attacker spreads from the first machine to others, looking for the valuable data or the systems that control them. Finally comes action on objectives: copying data out, encrypting files for ransom, manipulating a process, or simply waiting and watching. This sequence carries three lessons. First, every stage is an opportunity for detection. An attacker who gets in through phishing can still be caught when they try to escalate privileges or move sideways, provided someone is watching the logs. Second, the time an attacker spends inside a network before discovery, often called dwell time, is a major driver of damage; the SolarWinds intruders were inside some victims for months. Third, controls that block several stages at once are especially valuable. Strong multi-factor authentication hinders initial access and lateral movement; network segmentation limits lateral movement and the reach of ransomware; good logging supports detection at every stage. Measuring crime and its costs A recurring theme of Anderson's research is that discussions of cybercrime suffer from bad numbers. Estimates of global losses vary by orders of magnitude, and many come from firms that sell security products and have an interest in large figures. Anderson and colleagues published a study in 2012, updated in 2019, that tried to measure the costs of cybercrime more carefully by separating the direct losses from criminal acts, the indirect losses such as lost confidence and reduced online commerce, and the cost of defence. Their most striking finding was that for many types of online crime the indirect costs and defence costs far exceed the money criminals actually take. A spam campaign may earn its operators a modest sum while imposing large costs on everyone who has to filter, read and delete the messages. This asymmetry has a policy implication that Anderson pressed repeatedly: it can be far more efficient to spend money on catching and prosecuting the small number of people who run the major criminal operations than on ever-larger defensive products for the millions of potential victims. A second measurement problem is under-reporting. Many victims do not report fraud, firms often prefer not to disclose breaches, and police forces have historically lacked the skills and incentives to investigate. Crime that is not measured tends not to be policed. In England and Wales, the official Crime Survey began to include fraud and computer misuse only in the mid-2010s, and once it did, these offences turned out to account for a large share of all the crime that individuals experienced. Using the classification The four classes are not rigid. States hire criminals; criminals use techniques published by researchers; hacktivists may be fronts for state agencies. But the classification disciplines threat modeling in three ways. First, it forces a designer to ask who would realistically attack this system and why. A small retailer's website faces crooks almost exclusively. A defence contractor faces spies. A dating app faces the swamp. A hospital faces all of them. Second, it links the attacker to their economics. Crooks respond to cost and will move on; spies respond to value and will persist; the swamp often attacks individuals through personal relationships that no technical perimeter can separate. Defences should be designed accordingly. Third, it counters the tendency to design for the attacker who is most interesting rather than the one who is most likely. Media coverage favours dramatic state operations, but the losses most organizations actually suffer come from ordinary fraud, phishing and ransomware, often enabled by missing patches and weak authentication. A final point from the chapter is that the opponent is not always outside the organization or even hostile. Many breaches begin with a well-meaning employee who clicks a link, misconfigures a cloud storage bucket, or emails a spreadsheet to the wrong address. Good threat models include error alongside malice, which is why the next chapter turns to the psychology of the people who use systems. Key Takeaways · Anderson groups attackers into spies, crooks, geeks and the swamp, each with different goals, resources and methods. · State actors are patient and well funded; the realistic defence is to raise costs, detect intrusions quickly and limit the damage, as Stuxnet and SolarWinds illustrate. · Most organizations mainly face criminals, who run an industrialized, specialized economy and choose victims by weakness rather than identity. · Researchers expose flaws and publish tools; coordinated disclosure balances the interests of vendors and the public. · The swamp, including stalkers and abusive partners, shows that many serious harms come from people close to the victim and defeat perimeter-based threat models. · A good threat model names the realistic opponent, reflects that opponent's economics, and includes honest error as well as malice. Review Questions 1. Outline Anderson's four classes of opponent and explain how the likely attacker should change the design of a hospital information system compared with an online shop. 1. Why is a financially motivated criminal easier to deter than a state intelligence agency? Refer to the economics of each. 2. Using the SolarWinds case, explain what a supply-chain attack is and why it is hard to defend against. 3. What is coordinated disclosure, and what tensions does it try to resolve between researchers and vendors? 4. Why do intimate threats, such as a controlling partner, undermine conventional threat models? Suggest two design choices that would help. Chapter 3: The Human Element: Psychology and Usability For decades, security professionals described users as the weakest link and treated them as a problem to be managed. Anderson's third edition takes a different view. It argues that systems fail when they are designed without regard for how people actually think and behave, and that blaming users for predictable behaviour is a design failure in disguise. The chapter on psychology and usability draws on cognitive psychology, behavioural economics and a large body of usable-security research, much of it produced after the book's first edition, to explain why people are deceived, why they choose weak passwords, and what designers can do about it. The central lesson is simple to state and hard to apply: security mechanisms must fit the capabilities, goals and context of the people who use them. A control that users cannot understand or that obstructs their main task will be misused or bypassed, and the designer, not the user, should carry the responsibility. How minds get fooled Anderson summarizes several strands of psychology that explain why deception works. The first is the study of cognitive biases and heuristics, associated with Daniel Kahneman and Amos Tversky. People rely on quick, intuitive judgements that work well most of the time but fail in predictable ways. We judge risks by how easily examples come to mind, so dramatic, well-publicized threats seem more likely than mundane ones. We are more strongly moved by possible losses than by equivalent gains. We tend to be over-optimistic about our own chances and to discount future costs against present convenience. Each of these tendencies can be exploited by an attacker and must be allowed for by a designer. The second strand concerns social psychology, especially obedience to authority and conformity. Classic experiments showed how readily people follow instructions from someone who appears to be in charge. Fraudsters exploit this constantly, posing as bank officials, police officers, IT support staff or senior executives. The same research shows the power of social proof: people do what they see others doing, which is why fake reviews and fabricated urgency work. The third strand is the psychology of persuasion, including the well-known principles described by the psychologist Robert Cialdini: reciprocity, commitment and consistency, social proof, liking, authority and scarcity. Almost every scam uses several of them together. A fraudulent message that claims to come from the bank (authority), warns of an account being closed today (scarcity and urgency), and asks the victim to confirm details that "other customers have already updated" (social proof) is combining these levers deliberately. Anderson also discusses the psychology of the victim's situation. People are more easily deceived when they are busy, tired, stressed or distracted, and attackers time their approaches accordingly. Work email is read quickly between other tasks, and a convincing message arriving at the end of a long day may get only a second's thought. Research on why people fall for scams finds that intelligence and education offer surprisingly little protection; what matters more is whether the context triggers careful thought. Phishing and social engineering Social engineering is the manipulation of people into performing actions or divulging information. It is older than computers; confidence tricksters and "pretexters" who telephone an office pretending to be someone else have operated for as long as there have been offices. Phishing is its large-scale online form: messages designed to trick recipients into revealing credentials, paying money or installing malware. Anderson explains why phishing is so effective and so hard to stop by technical means alone. Email was designed without strong authentication of the sender, so forging a plausible sender address was long trivial; later standards for checking the sending domain helped but did not eliminate the problem, because attackers simply register look-alike domains. Web addresses are hard for ordinary people to read correctly, and the visual cues that browsers provide, such as padlock icons, have been widely misunderstood. A padlock shows that the connection is encrypted, not that the site is honest, and phishing sites routinely obtain valid certificates. The more dangerous variants are targeted. Spear phishing aims at particular individuals using personal details gathered from social media or earlier breaches. Business email compromise involves criminals impersonating a senior executive or a supplier to persuade finance staff to make payments to a fraudulent account. These attacks require little technical skill and have caused very large losses, because they exploit the ordinary business processes by which organizations move money. Anderson's analysis leads to several design principles. First, training users to spot phishing has limited value on its own. It can raise awareness, but a well-crafted message will still fool a proportion of recipients, and in a large organization that proportion is enough. Phishing simulations that punish staff who click can breed resentment and discourage reporting. Second, systems should be designed so that a single deceived user cannot cause a catastrophe. Payment changes should be confirmed through a separate channel; administrative access should need more than a password that can be typed into a fake page. Third, the most effective technical countermeasures are those that remove the user's judgement from the loop. Authentication methods bound to the genuine website, such as hardware security keys and the newer passkeys, will not release credentials to a look-alike domain however convincing it appears, because the check is made by the device rather than the person. A related category is authorized push payment fraud, where the victim is tricked into sending money themselves, for example by a caller claiming that their account is compromised and that funds must be moved to a "safe account". Because the victim authorizes the transfer, banks long argued that they were not liable. This became a significant policy issue in the United Kingdom, and it illustrates the incentive argument of the book: when banks bear the loss, they invest in prevention; when customers do, they tend not to. Passwords Passwords receive extended treatment in the book because they illustrate so clearly the collision between security theory and human capability. A password is supposed to be something only the user knows. Security theory wants it to be long, random, unique to each site and never written down. Human memory cannot meet these requirements for dozens or hundreds of accounts. The result is predictable: people choose memorable passwords, reuse them across sites, make small variations when forced to change them, and write them down. Research by Anne Adams and Angela Sasse in the late 1990s, which Anderson discusses, found that users were not careless by nature but were responding rationally to demands that conflicted with their work, and that security departments had failed to explain the reasons for the rules. Anderson examines several specific problems. Guessability. Large leaks of password databases have shown that people choose from a small set of common patterns: names, dates, dictionary words with digits appended, and keyboard sequences. An attacker who tries the most common few thousand choices will break a substantial fraction of accounts. Composition rules, such as requiring a digit and a symbol, tend to produce predictable variations rather than real strength. Reuse. When one site is breached, the leaked email and password pairs are tried automatically against other sites, an attack known as credential stuffing. Reuse turns a breach at a minor website into a compromise of a person's email or bank account. Forced changes. Requiring users to change passwords every few months was long standard practice. Research showed that it leads to weaker, more predictable passwords, and official guidance changed accordingly. The US National Institute of Standards and Technology revised its digital identity guidelines in 2017 to advise against arbitrary periodic changes, and Britain's National Cyber Security Centre gave similar advice. Recovery. The weakest part of many password systems is the process for resetting a forgotten password. Security questions ask for facts, such as a mother's maiden name or a first school, that are often public or guessable. Help desks can be talked into resets. Email-based recovery makes the email account the master key to everything else. Storage. On the system side, passwords must be stored in a form that is useless if stolen. The standard technique is to store not the password but the result of a deliberately slow one-way function applied to it, with a random value called a salt so that identical passwords produce different stored values. Chapter 4 explains the idea of one-way functions intuitively; the point here is that many breaches have exposed passwords stored carelessly, sometimes in plain text. The design response Anderson and the wider research community favour is to reduce reliance on memorized secrets. Password managers let users have long, unique passwords without remembering them. Multi-factor authentication adds a second element, such as a code from a phone or a hardware token, so that a stolen password is not enough on its own. Rate-limiting and monitoring make online guessing impractical. And risk-based approaches let systems ask for extra checks only when something looks unusual, reducing friction for normal use. The help desk and the recovery channel A pattern Anderson returns to throughout the book is that attackers go around strong mechanisms rather than through them, and the human-staffed recovery channel is the most common way around. Every system needs a way to help a legitimate user who has lost a password, a phone or a card. Whoever runs that process is under pressure to be helpful and quick, and has limited means of checking identity. An attacker who can sound plausible on the telephone therefore faces a much easier task than one who must break the authentication itself. The SIM swap attack shows the pattern clearly. Many online services send one-time codes by text message as a second factor or as a way to reset a password. A criminal who has gathered some personal details about a victim calls the victim's mobile operator, claims to have lost their phone, and asks for the number to be moved to a new SIM card in the criminal's possession. If the operator's staff are persuaded, every text message, including the security codes, now goes to the criminal, who can reset the victim's email and bank passwords within minutes. The strong mechanism, a second factor, has been defeated by attacking a different organization's customer-service process that its designers did not control. Regulators and operators have responded with extra checks, and security guidance now treats text-message codes as weaker than app-based or hardware factors, but the underlying lesson is general: the security of an account is the security of its weakest recovery path. The same logic applies inside organizations. Attackers who breached several large companies in recent years did so by telephoning the internal IT help desk, impersonating an employee, and asking for a password reset or a new multi-factor enrolment. The defence is procedural as much as technical: help-desk staff need a verification method that does not depend on facts an attacker can find, the authority to refuse without penalty, and a process that makes high-risk changes, such as re-enrolling an administrator's authenticator, subject to a second approval or a delay. Security culture in organizations Individual psychology operates within an organizational setting, and Anderson's broader argument about incentives applies to staff as much as to firms. Employees respond to what they are measured and rewarded on. A call-centre worker judged on the number of calls handled per hour has every reason to skip slow identity checks. A developer judged on features shipped has little reason to spend time on security testing. Unless security is built into the way work is measured, it will be sacrificed whenever it conflicts with the targets people are actually held to. Research on organizational security, including the work of Sasse and her colleagues, describes the "compliance budget": the limited amount of effort that staff are willing to spend on security tasks that do not help their main work. Once that budget is exhausted, further rules are ignored. Security managers should therefore spend the budget wisely, requiring effort only where it buys real protection and removing rules that exist for show. A healthy security culture also depends on how mistakes are treated. In safety-critical industries such as aviation, the practice of reporting errors without blame has greatly improved learning, because people report near-misses that would otherwise stay hidden. Security teams that punish staff who click on phishing links discourage the prompt reporting that allows an attack to be contained. The most useful response to a suspicious email is often a fast report from the person who received it, and that only happens if reporting is easy and safe. Designing for real people Beyond deception and passwords, the chapter sets out principles for usable security that apply across all systems. Make the secure way the easy way. Defaults matter enormously, because most people never change them. A system that ships with encryption turned on, updates installed automatically and administrator accounts disabled will be more secure in practice than one with better features that must be configured. Align security with the user's primary task. People use systems to get work done, not to be secure. Warnings that interrupt the task are dismissed; research on browser warnings has shown that people learn to click through them when most turn out to be false alarms. Warnings should be rare, specific and actionable. Avoid blaming the user. When a system can be defeated by a predictable human error, the fault lies in the design. This matters for liability as well as engineering. Anderson spent years challenging banks that told fraud victims they must have been negligent with their PINs, when the evidence often pointed to system weaknesses. Organizations that blame users also learn less from incidents, because people hide mistakes rather than report them. Consider the full range of users. Systems must work for older people, people with disabilities, people with limited literacy or technical confidence, and people under stress. Security that works only for young technical users excludes large parts of the population and pushes them towards less safe alternatives, such as asking a relative to manage their accounts. Remember that the attacker adapts. Any cue that users are taught to rely on, such as a padlock icon or a personal greeting in an email, will be copied by attackers once it becomes common. Defences that depend on the user noticing something will degrade over time; defences that are built into the system do not. The larger point of the chapter is that psychology is not an optional extra for security engineers. The same knowledge that lets marketers persuade people to buy things lets criminals persuade them to give away money, and a designer who ignores it will build systems that fail in the field no matter how sound their technical core. Key Takeaways · Systems fail when they ignore how people think; predictable user behaviour is a design constraint, not an excuse for blame. · Cognitive biases, obedience to authority and the standard levers of persuasion explain why social engineering and phishing work. · Training alone cannot stop phishing; design so that one deceived user cannot cause catastrophe, and prefer controls such as phishing-resistant authentication that do not rely on human judgement. · Password problems stem from impossible demands on memory; guessable choices, reuse, forced changes and weak recovery are the main failure points. · Password managers, multi-factor authentication, rate-limiting and sensible defaults reduce dependence on memorized secrets. · Make the secure option the default and the easiest path, keep warnings rare and meaningful, and design for the full range of users. Review Questions 1. Using two principles of persuasion, analyse how a business email compromise attack manipulates its victim, and propose process controls that would defeat it. 1. Why did official guidance move away from forcing regular password changes? What does this reveal about the relationship between security rules and user behaviour? 2. Explain credential stuffing and describe two measures a website operator can take against it. 3. Why is a padlock icon a weak basis for teaching users to avoid phishing sites? 4. "The user is the weakest link." Critically evaluate this claim with reference to Anderson's view of usability. Hashtags: #TheArchitectureOfDefense #SecurityEngineering #RossAnderson #DependableDistributedSystems #WholeSystemSecurity #SecurityPolicy #SecurityMechanisms #SecurityAssurance #SecurityIncentives #ThreatModeling #AttackTrees #DefenseInDepth #CyberResilience #AccessControl #TrustedComputingBase #SecurityEconomics #HumanFactorsSecurity #SocialEngineering #PhishingResistance #MultiFactorAuthentication #UsableSecurity #DistributedSystemsSecurity #SecurityAndDependability #AdversarialThinking #FutureOfSecurityEngineering

Download the Book (PDF): Introduction Conway's Law holds that organizations design systems that mirror their own communication structures. For students of IT and software management, this means organizational design is just as critical as software architecture. Matthew Skelton and Manuel Pais provide the definitive model for this relationship, but their frameworks are dense. Cognitive load theory, four team types and a precise vocabulary of interaction modes can easily overwhelm a student trying to build a coherent set of study notes. This companion provides the necessary structural clarity. It is written expressly to explain Team Topologies by Matthew Skelton and Manuel Pais, arranging their operational models into an accessible academic curriculum. It defines the four fundamental team types (stream-aligned, enabling, complicated-subsystem and platform) and the three core interaction modes (collaboration, X-as-a-Service and facilitating). With clear chapter summaries and review questions, it prepares you to write analytical essays on modern socio-technical systems. The book and its authors Team Topologies: Organizing Business and Technology Teams for Fast Flow was published by IT Revolution Press in 2019. Its authors came to the subject from practice rather than from the academy. Matthew Skelton, based in the United Kingdom, had worked for years on software delivery, operations and what the industry had begun to call DevOps. Manuel Pais, a Portuguese consultant and trainer, had a similar background in continuous delivery and the working relationships between development and operations. Before the book, Skelton had built an online catalogue of "DevOps Topologies": a set of recurring patterns and anti-patterns for how development and operations teams could be arranged. The book grew out of that catalogue, and out of the authors' observation that organizations adopting modern tools and practices kept stalling for reasons that had little to do with the tools. The book arrived at a good moment. By the late 2010s, many large organizations had adopted agile methods, cloud infrastructure and microservice architectures, yet they still found delivery slow and painful. The research programme later published as Accelerate (Nicole Forsgren, Jez Humble and Gene Kim, 2018) had shown statistically that loosely coupled architectures, and teams able to change and release their systems without heavy coordination, were strong predictors of software delivery performance. What organizations lacked was a practical way to design for that loose coupling on the human side. Team Topologies offered one: a small, memorable vocabulary for describing teams and the way they relate, grounded in a few well-established ideas about communication and human cognition. The book quickly became a standard reference in software engineering management. Its terms, especially "stream-aligned team", "platform team" and "cognitive load", now appear in job descriptions, conference talks and internal strategy documents at organizations of every size. The authors founded a company around the approach, and in September 2025 IT Revolution published a second edition. That edition keeps the core model unchanged. The four team types and three interaction modes stand as they were. It adds a reflective foreword, an appendix of case studies from organizations around the world and an afterword on what the authors learned from the community that had adopted their ideas. The second edition also carries a slightly revised subtitle, Organizing Business and Technology for Fast Flow of Value, which signals where the authors have moved their emphasis: from software delivery alone towards value delivered to customers. Why the book matters for students The central claim of Team Topologies is simple to state and difficult to act on: the structure of teams, and the way those teams interact, shapes the software an organization can build and how quickly it can change it. Most management writing treats organizational design as a question of reporting lines, spans of control and budgets. Most software architecture writing treats system design as a technical question of modules, interfaces and data. Skelton and Pais argue that these are the same question approached from two sides, and that treating them separately produces systems nobody intended. For a student, the book is valuable because it joins these two conversations. It draws on an old idea, Melvin Conway's 1968 observation about committees and the systems they design. It draws on a body of research from educational psychology, John Sweller's cognitive load theory. It draws on anthropology, Robin Dunbar's work on the limits of human social groups. And it gives all of this operational form. Essays on the book therefore reward careful attention to how the pieces connect. A student who can explain why cognitive load limits the size of a team's responsibilities, why that limit in turn shapes where software boundaries should fall, and why the interaction modes exist to keep those boundaries honest, understands the book. A student who can only list the four team types does not. The controlling idea of this guide This guide is organized around a single argument, which it takes to be the heart of Team Topologies: fast flow of change is an organizational design problem before it is a technical one, and the team, limited by what its members can hold in their heads, is the unit around which both the organization and its software should be designed. Everything else in the book follows from this. Conway's Law explains why team design and system design cannot be separated. Cognitive load explains how large a team's responsibilities can safely be. The four team types describe the few distinct jobs teams need to do in an organization built for flow. The three interaction modes describe the few distinct ways those teams should relate. Fracture planes describe where to draw boundaries. Organizational sensing explains why the whole design must keep changing. Read this way, the book is not a catalogue of labels to be stuck onto an existing org chart. It is a method for designing, and repeatedly redesigning, a socio-technical system. The guide returns to this point often, because the most common misreading of Team Topologies, in practice and in student essays alike, is to treat it as a static taxonomy. How the guide is organized The guide follows the logic of the argument rather than reproducing the book page by page, although it tracks the book's own progression closely. It opens with the problem: why conventional org charts mislead, and what the older tradition of socio-technical systems theory had already discovered about the fit between work, technology and social structure. It then turns to the forces that make team design consequential. Conway's Law, and the deliberate strategy known as the reverse Conway maneuver, are examined first. Then come the team-first principles of size, stability and ownership. Cognitive load is treated at length, because it is the constraint from which most of the book's practical guidance is derived. With those foundations in place, the guide sets out the model itself. The four fundamental team types each receive careful definition, and platforms, which are the most frequently misunderstood of them, receive extended treatment, including the idea of the thinnest viable platform. The guide then examines fracture planes, the natural seams along which a system and its teams can be divided, before turning to the three interaction modes and the way interactions should change as a piece of work moves from discovery to routine. The final chapter deals with evolution and practice: how organizations sense when their design needs to change, the static topologies and anti-patterns the authors warn against, the caution offered by the widely copied "Spotify model", and the developments that have followed the first edition, including Team Interaction Modeling and the second edition's new material. Where the guide reports what Skelton and Pais argue, it says so. Where it adds context, such as later research, publicly documented company examples or criticism of the model, it marks that material as its own. Each chapter closes with key takeaways and review questions designed to test understanding rather than recall. A glossary, an annotated list of further reading and a set of notes on sources complete the volume. Chapter 1: The Problem with Org Charts Every sizeable organization has an org chart. It shows who reports to whom, which departments exist and where budget authority sits. Team Topologies opens by questioning how much this familiar document can tell us about how work actually gets done. Skelton and Pais argue that it tells us surprisingly little, and that organizations which design themselves by rearranging boxes on a chart tend to damage the very flow of work they are trying to improve. This chapter explains that argument, places it within the older tradition of socio-technical systems thinking, and introduces the idea of fast flow that runs through the whole book. What an org chart shows and what it hides An org chart is a picture of formal authority. It is useful for questions of accountability: who approves spending, who conducts performance reviews, who can hire. It is much less useful for questions of communication and flow. Work in a software organization does not travel up and down reporting lines. A customer request, a defect report or a new product idea moves sideways across the organization, passing between product managers, designers, developers, testers, security specialists, operations engineers and support staff. Many of these people report to different managers. The path a piece of work follows is therefore largely invisible on the chart. Skelton and Pais make a stronger point than this. If people communicate only along the lines an org chart shows, the organization will be slow, because every cross-departmental question has to travel up one hierarchy and down another. In practice, healthy organizations work in spite of their charts. People build informal networks and talk directly to colleagues in other departments. Management literature has long recognized this gap. The authors draw on the work of the organizational consultant Niels Pflaeging, who distinguishes between an organization's formal structure, which exists mainly for compliance and legal purposes, its informal structure of personal influence, and its value creation structure, which is how work is actually accomplished through the interplay of teams. The org chart captures only the first of these. The consequence is that decisions based on the chart alone tend to optimize locally. A manager responsible for a testing department may make testing more efficient by batching requests, standardizing forms and prioritizing a queue. Within the box on the chart, that is a sensible improvement. Across the organization, it may lengthen the time every change takes to reach customers, because work now waits longer between development and release. The chart gives no hint of this, because the delay lives in the connections between boxes, not in the boxes themselves. A related problem is that org charts encourage thinking about individuals rather than teams. Each box represents a manager and the people who report to that manager. Whether those people actually work together as a team, share goals and depend on one another, is not shown. Skelton and Pais argue that the team, not the individual and not the department, is the most effective unit for delivering and running software. An org chart, by its nature, obscures teams. Restructuring as a recurring disappointment Many organizations respond to slow delivery by reorganizing. Departments merge, split or change reporting lines. New leaders arrive and draw new charts. Team Topologies is sceptical of this habit. The authors observe that reorganizations are often undertaken for reasons unrelated to the flow of work: cost cutting, the preferences of a new executive or a desire to be seen to act. Such reorganizations tend to disrupt the working relationships and shared understanding that made existing teams effective, while leaving the underlying causes of slowness untouched. The book's alternative is not to stop changing structure. Skelton and Pais are clear that organizations should evolve their team design continually. Their point is that structural change should be driven by an understanding of how work flows and how software is built, and that it should treat teams as durable units whose members stay together long enough to become effective. A reorganization that reshuffles individuals across teams every few months throws away the most valuable asset a team has: its members' accumulated trust and knowledge of one another and of their systems. This guide adds a practical observation. Students writing about organizational change often assume that a structural change is a single event with a before and an after. Team Topologies treats structure more like software: something released in small increments, observed in use and adjusted. That shift in framing is one of the book's most important contributions, and later chapters return to it. The socio-technical tradition Although Team Topologies is written for technology leaders of the twenty-first century, its central insight has a long pedigree. The authors acknowledge the tradition of socio-technical systems thinking, and a student essay will be stronger for understanding where that tradition came from. In the years after the Second World War, researchers at the Tavistock Institute of Human Relations in London studied the British coal industry. Eric Trist and Ken Bamforth published a now classic paper in 1951 on the social and psychological consequences of the longwall method of coal mining. Before mechanization, miners had worked in small, self-organizing groups. Each group handled the whole cycle of cutting, loading and securing the coal face. Members were skilled in several tasks and responsible to one another. The mechanized longwall method broke this pattern. It introduced a long face worked by specialized shifts, each performing one fragment of the cycle, with the work of one shift depending on the completion of the previous one. The technology was more advanced, but productivity disappointed, absenteeism rose and conflict between shifts grew. Trist and Bamforth concluded that the technical system and the social system could not be designed independently. A technology that fragmented work also fragmented responsibility and relationships, and the losses on the social side cancelled the gains on the technical side. Later studies in the same programme found that some pits had arrived at a different arrangement, sometimes called composite working. There, multi-skilled groups took collective responsibility for the whole cycle while using the new equipment. These groups performed better. The lesson, developed by Trist, Fred Emery and others over the following decades, was that organizations are socio-technical systems. Their performance depends on the joint optimization of technology and social structure. In 1976 Albert Cherns set out a list of principles of socio-technical design. Among them were the ideas that work should be specified only as far as necessary, that boundaries between units should not be drawn in ways that impede the sharing of information and learning, and that design is never finished. The echoes in Team Topologies are strong. A stream-aligned team that owns a piece of software from idea to operation resembles the composite work group that owned the whole mining cycle. A delivery process that passes work from a development team to a testing team to an operations team resembles the fragmented shifts of the longwall. And the idea that organizational design must evolve continually echoes Cherns's principle of incompletion. Skelton and Pais did not invent the socio-technical perspective. What they did was translate it into a concrete vocabulary suited to modern software delivery, where the technology in question is not a coal-cutting machine but a changing, interconnected set of services. Fast flow as the organizing goal The goal that ties the book together is what the authors call fast flow: the ability to deliver a steady stream of changes to customers rapidly, safely and sustainably. The idea draws on lean thinking and on the DevOps movement, both of which emphasize the end-to-end movement of work from request to value rather than the efficiency of individual steps. Flow matters because software products are never finished. Customer needs change, competitors act, regulations shift and security threats appear. An organization's advantage depends less on any single release than on how quickly and safely it can respond. The Accelerate research, which Skelton and Pais draw on, measured delivery performance through indicators such as deployment frequency, lead time for changes, time to restore service and change failure rate. It found that high performers did well on speed and stability together, rather than trading one for the other. It also found that architectures allowing teams to test and deploy independently were among the strongest predictors of that performance. Fast flow places particular demands on organizational design. Work flows fastest when it does not have to wait for other teams. It needs few handoffs, the team doing the work has the skills and authority to complete it, and the people involved understand the system well enough to change it confidently. Each of these conditions is organizational as much as technical. Handoffs arise where team boundaries fall. Skills and authority depend on how teams are composed. Understanding depends on how much a team is asked to hold in its head. Team Topologies is, at bottom, a set of design rules for meeting these conditions. The book also insists that flow is not the same as speed at any cost. A team that delivers quickly but leaves behind an unmaintainable system, or that burns out its members, is not achieving fast flow in the authors' sense. Sustainability is part of the definition, which is one reason the book pays such close attention to the limits of human attention and to the stability of teams. An illustration: the path of a single change A simple, hypothetical example makes the argument concrete. Imagine a mid-sized retailer whose online shop is run by a technology department organized along functional lines. There is a front-end development group, a back-end development group, a database administration team, a quality assurance team, an infrastructure team and a service desk. Each group has its own manager and its own queue of work. The org chart is tidy, and each manager can show that the group is busy and productive. Now suppose the marketing department wants customers to be able to save items to a wish list. The request starts with a product manager, who writes a specification and passes it to the front-end group. The front-end developers need a new back-end service, so they raise a ticket with the back-end group, which is working on other priorities and schedules the change for its next cycle. The back-end developers need a new database table, so they raise a ticket with the database administrators. When the code is written, it waits for a testing slot with the quality assurance team. When testing passes, it waits for a release window managed by the infrastructure team. After release, a customer reports a bug to the service desk, which routes the report back through the same chain. No single group in this story is doing anything wrong. Each is responding sensibly to its own queue. Yet the elapsed time from request to working feature may be many weeks, even though the hands-on effort was a few days. Most of the time was spent waiting at the handoffs between groups. The people who understand the wish list feature best, the developers who wrote it, may never hear about the bug report directly. Nobody owns the feature from end to end. The org chart gives no warning of any of this. It shows six well-run groups. What it does not show is that a customer-facing change must cross five team boundaries before it reaches a customer, and that every boundary is a queue. Skelton and Pais would ask a different question from the one the chart invites. Instead of asking how each group can become more efficient, they would ask how teams could be arranged so that a change like this can be made by one team with minimal dependence on others. That question leads directly to the idea of the stream-aligned team, examined later in this guide. It also explains why the book pays so much attention to the interfaces between teams. Wherever a boundary cannot be avoided, the way the teams on either side interact determines whether the boundary becomes a queue. The example also shows why technology leaders cannot hand organizational design to a separate human resources function and treat it as none of their business. The decision to create a separate database administration team, for instance, is a decision about where software boundaries will fall and how changes to data will be made. Skelton and Pais argue that whoever decides team structure is, in effect, making architectural decisions, whether or not they realize it. Organizational design therefore needs technical insight, just as software architecture needs an understanding of the teams who will build and run it. The example points to a funding issue as well. Many organizations finance technology work as a series of projects, each with a start date, an end date and a budget. People are gathered for the project and dispersed when it ends. This model fits the org chart well, because it treats teams as temporary assemblies of individuals drawn from functional departments. It fits the idea of fast flow badly, because the software a project creates lives on after the project ends, and nobody is left who owns it. Skelton and Pais favour persistent teams that own software over its whole life, which in turn implies funding teams or product areas over time rather than funding projects. The book does not develop a full theory of technology funding, but it makes clear that organizational design and financial practice cannot be separated for long. From problem to model Having established the problem, the book builds its model in stages, and this guide follows the same path. The first stage explains why team design has such influence over system design, which is the subject of Conway's Law. The second establishes the team as the basic unit and examines what makes a team effective. The third introduces cognitive load as the constraint that limits what any team can own. Only then does the book introduce its four team types and three interaction modes, which are best understood as the design choices that follow once those constraints are taken seriously. For students, the key analytical move is to see the model as a response to a diagnosis. The org chart fails because it describes authority rather than flow, and individuals rather than teams. The socio-technical tradition explains why that failure matters: social and technical structures shape each other, and designing one while ignoring the other produces poor results. Team Topologies proposes to design both together, using the team as the unit and flow as the measure. Key Takeaways · Skelton and Pais argue that org charts describe formal authority but not how work flows, so decisions made from the chart alone tend to optimize locally at the expense of the whole. · Drawing on Niels Pflaeging, the book distinguishes formal structure from informal influence and from the value creation structure through which work is actually done. · The authors are sceptical of frequent reorganizations that reshuffle individuals, and favour deliberate, incremental evolution of durable teams. · The book's perspective continues the socio-technical systems tradition begun by Trist and Bamforth's 1951 study of coal mining, which showed that technical and social systems must be designed together. · Fast flow, meaning rapid, safe and sustainable delivery of change, is the goal against which the book judges every design choice. Review Questions 1. Why does an org chart give a poor picture of how work moves through a software organization? Illustrate your answer with an example of local optimization. 1. What did Trist and Bamforth observe about the longwall method, and how does their finding anticipate the argument of Team Topologies? 2. Explain the distinction between formal structure and value creation structure, and say which one the book asks leaders to design for. 3. In what sense is fast flow an organizational problem as well as a technical one? 4. Why are Skelton and Pais wary of reorganizations that frequently move individuals between teams? Chapter 2: Conway's Law and the Reverse Conway Maneuver If one idea underpins Team Topologies, it is Conway's Law. Skelton and Pais treat it as the basic reason why organizational design and software architecture cannot be separated. This chapter traces the law from its origin in 1968, examines the evidence for it, and explains the deliberate strategy the book recommends in response: shaping team structures first so that the desired architecture emerges, an approach known as the reverse Conway maneuver. The chapter closes with a publicly documented case, Uber's domain-oriented microservice architecture, that shows the relationship between teams and systems in practice. Melvin Conway's observation In 1968 the computer scientist Melvin Conway published an article titled "How Do Committees Invent?" in the magazine Datamation. Its central claim was that any organization that designs a system will produce a design whose structure copies the organization's communication structure. Conway's reasoning was straightforward. Designing a system requires dividing the design work among people or groups. The way the work is divided determines which parts of the system are designed by which groups. The interfaces between parts of the system must be negotiated between the groups responsible for them. The system's structure therefore ends up mirroring the structure of the design organization, because the organization can only produce interfaces where its own members communicate. Conway gave examples, the most often repeated of which concerns compilers. If a compiler is designed by several groups, it will tend to have as many major stages, or passes, as there are groups. The number of passes reflects the organization rather than any technical necessity. Conway also drew a sobering conclusion about large projects. Because the initial design shapes the organization that is set up to build it, and that organization then constrains every later design choice, early decisions about team structure can lock a project into an architecture that is no longer appropriate once more is learned. The observation acquired its name a few years later. Fred Brooks, in The Mythical Man-Month (1975), cited Conway's argument and called it Conway's Law. Brooks's book, a study of what went wrong in the development of IBM's OS/360 operating system, was widely read, and the name stuck. It is worth being precise about what the law says. It is not a claim that organizations deliberately design systems to look like themselves. It is a claim about constraint. Communication paths make certain designs easy and others hard. Where two groups communicate frequently and richly, the parts they build can be tightly integrated. Where two groups rarely communicate, the parts they build will meet at narrow, formal interfaces, or will not meet at all. Over time, the system settles into a shape that the communication structure can support. One further point of precision matters for students. The communication structure Conway described is not the same as the formal reporting structure. Two teams in different departments that talk every day form a close communication link, whatever the org chart says. Two teams under the same manager that never talk do not. Conway's Law therefore operates on the real pattern of communication, which is exactly the pattern that org charts fail to show, as Chapter 1 of this guide explained. This is why Skelton and Pais focus on designing team interactions rather than reporting lines. Evidence for the mirroring effect Conway's Law began as an observation from experience, and for many years it was treated as folklore. Later research gave it firmer grounding. This section summarizes that research. It is context this guide adds, and it is useful for essays that need to show the law is more than an anecdote. Researchers at Harvard Business School, including Alan MacCormack, John Rusnak and Carliss Baldwin, studied what they called the mirroring hypothesis. In a study published in Research Policy in 2012, they compared pairs of software products that performed similar functions but were developed by different kinds of organization. Some were built by tightly knit commercial teams, others by distributed open-source communities whose members rarely met. Analysing the dependency structure of the code, they found that the products built by loosely coupled, distributed organizations tended to be more modular, while those built by tightly coupled organizations tended to have more interdependencies between components. The structure of the organization was reflected in the structure of the code. Lyra Colfer and Carliss Baldwin later reviewed a large body of empirical studies across industries in a 2016 article in Industrial and Corporate Change. They found broad support for the mirroring hypothesis, while also documenting exceptions. The exceptions are instructive. In some cases, organizations that were formally separate managed to build tightly integrated products because they invested heavily in communication tools, shared knowledge and common standards. In other cases, companies deliberately broke the mirror for strategic reasons. The lesson is that mirroring is a strong tendency rather than an iron rule, and that departures from it come at a cost in coordination effort. Studies at Microsoft and elsewhere have also linked organizational structure to software quality, finding that measures of how the development organization was divided could predict where defects would cluster. The general finding across this literature is consistent with Conway: where communication is weak, integration suffers, and where the organization is fragmented, the system tends to fragment in the same places. How Skelton and Pais use the law Team Topologies does not treat Conway's Law as a curiosity. The authors treat it as a design force to be used deliberately. Their argument has several parts. First, if system architecture tends to mirror team structure, then any decision about team structure is also a decision about architecture. A manager who creates a separate database team has made it likely that the system will have a central, shared database with changes controlled by that team. A manager who creates separate front-end and back-end teams has made it likely that the system will be layered along those lines, with every user-facing feature requiring coordinated changes in both layers. Skelton and Pais conclude that organizational design requires technical expertise. The people making structural decisions must understand the architectural consequences, and technical leaders must be involved in them. Second, the authors argue that unrestricted communication is not the answer. It might seem that if communication paths shape architecture, the best course is to encourage everyone to talk to everyone. Skelton and Pais argue the opposite. If every team communicates with every other team, the system will tend towards a design in which every part depends on every other part. That is, the system becomes a monolith in which nothing can change independently. The authors suggest that organizations should be deliberate about which teams need to communicate and how much. Where teams build parts of the system that should be loosely coupled, the communication between those teams should also be limited and structured. This is a counterintuitive message for many managers, who have been taught that more collaboration is always better. It is also one of the book's most important ideas, because it underlies the interaction modes discussed later in this guide. Third, the authors note that the tools an organization uses can shape communication, and therefore architecture, in unintended ways. A single shared ticketing system or chat channel used by every team can create communication paths that encourage coupling. Separate tools for teams that should collaborate closely can create barriers where none are wanted. Tool choices, in other words, are part of organizational design. Fourth, the book observes that the constraint runs in both directions. An existing architecture also shapes the organization. If a system is a large monolith, the teams that work on it will find themselves forced into constant coordination, regardless of what the org chart says. Conway's Law therefore explains why simply drawing new team boundaries is not enough if the software does not allow those teams to work independently. The reverse Conway maneuver If team structure shapes system architecture, an organization can use that fact deliberately. It can decide what architecture it wants, and then arrange its teams so that their communication structure matches that architecture. Given time, the system will tend to take the intended shape. This strategy is known as the reverse Conway maneuver, or sometimes the inverse Conway maneuver. The term circulated among consultants at the technology firm ThoughtWorks around 2010 and was popularized through their publications, including the ThoughtWorks Technology Radar. Skelton and Pais adopted it as a central technique. The maneuver works as follows. An organization first identifies the architecture it needs in order to achieve fast flow: typically, a set of loosely coupled services or components, each aligned to a coherent area of business capability, each changeable and deployable independently. It then forms teams whose boundaries correspond to those components, gives each team ownership of its component and limits the need for communication between teams to the interfaces between components. The teams, working mostly independently, will naturally build and evolve components with clear boundaries, because that is what their communication structure supports. The reverse Conway maneuver is not magic, and the authors do not present it as such. An existing system does not reshape itself overnight because the teams have been rearranged. If the current software is tightly coupled, newly formed teams will still find themselves needing to coordinate, and some technical work will be needed to separate components. What the maneuver does is align the organizational force with the desired technical direction, so that the two push the same way rather than against each other. Without it, a team trying to split a monolith while organized to support that monolith is working against the grain. The maneuver also demands clarity about the target architecture, and that is where many attempts falter. An organization that forms teams around vaguely defined areas will produce vaguely bounded software. The book's later guidance on fracture planes, discussed in Chapter 7 of this guide, is essentially a method for finding good boundaries to design towards. Monoliths hidden in the organization Skelton and Pais extend the idea of a monolith beyond the obvious case of a single large application. They describe several kinds of hidden monolith that can undermine fast flow even when the software appears to be divided. As Table 1 summarizes, each kind creates a coupling that forces teams to coordinate. Table 1. Kinds of monolith described in Team Topologies. Kind of monolith What is coupled Effect on teams Application monolith One large deployable system All teams release together Joined-at-the-database Separate services sharing one schema Schema changes need cross-team agreement Monolithic build One build for many components A change anywhere triggers a full rebuild Coupled release Components released as one bundle Teams wait for a shared release date Monolithic model One domain model or language forced everywhere Teams cannot fit models to their context Monolithic thinking One standard technology or approach for all Teams lose freedom to choose fitting tools Monolithic workplace One uniform office layout Space does not suit team collaboration needs The last three items are the most interesting for students of organization. They show that the authors see monoliths not only in code but in habits of mind and physical arrangements. A policy that every team must use the same technology stack, however well intended, couples teams to a central decision and can prevent them from choosing what suits their domain. An open-plan office designed to maximize communication may, by Conway's reasoning, encourage communication patterns that do not match the intended architecture, while failing to give teams the space they need to work closely together. Case study: Uber's domain-oriented microservice architecture The ride-hailing company Uber provides a well-documented public example of the relationship between organizational scale and system architecture. The following summary draws on a July 2020 article on the Uber engineering blog by the engineer Adam Gluck, introducing what the company called Domain-Oriented Microservice Architecture, or DOMA. The analysis connecting it to Team Topologies is this guide's own. Uber had adopted microservices during a period of very rapid growth, as a way of letting many teams work and release independently. By the time of the article, the company had around 2,200 critical microservices. The article explains that this degree of fragmentation had created its own problems. Understanding how a request travelled through the system could require tracing calls across many services owned by many teams. A single feature could require changes to services owned by several teams, which meant coordination, negotiation and waiting. The independence that microservices were meant to provide had been partly lost, because the boundaries between services did not correspond to coherent areas of business function. Uber's response was to group related microservices into domains, each representing a coherent area of functionality, such as map search, fares or matching riders with drivers. The article describes around 70 domains, organized into layers, with each layer depending only on the layers beneath it. Each domain presented a single entry point, which Uber called a gateway, so that other teams could use the domain's capabilities without needing to understand its internal services. Uber also introduced extension mechanisms that allowed teams to add behaviour to another domain's services without modifying them directly. The article reports improvements, including a large reduction in the cost of supporting platforms and faster integration of new features. Two features of the case are relevant to Team Topologies. First, the gateway is an explicit, narrow interface between a domain and the rest of the organization. It reduces the need for other teams to communicate with the domain's owners about internal details, which is exactly the kind of limited, structured interaction the book recommends. In the book's vocabulary, the domain is offered to other teams as a service. Second, the article notes that domains did not always follow the company's reporting structure. For example, it describes one engineering organization whose services were divided across three separate domains. This is a useful reminder that the unit of architectural design and the unit of formal management need not coincide. What matters, from the Conway perspective, is that the teams responsible for each domain can work within it with minimal coordination with others. The case also illustrates a point Skelton and Pais make repeatedly: breaking a system into many small pieces does not by itself produce fast flow. If boundaries are drawn without regard to business domains and team cognition, the result can be a distributed system that is just as coupled as the monolith it replaced, and harder to understand. Good boundaries, in the book's view, are those that allow a team to own a coherent piece of the business and deliver changes to it largely on its own. Key Takeaways · Melvin Conway argued in 1968 that organizations produce systems whose structure copies their communication structure; Fred Brooks later named this Conway's Law. · Empirical work on the mirroring hypothesis, including studies by MacCormack, Rusnak and Baldwin, supports the law as a strong tendency, with costly exceptions. · Skelton and Pais conclude that decisions about team structure are architectural decisions, and that unrestricted communication between all teams tends to produce monolithic systems. · The reverse Conway maneuver shapes team boundaries to match a desired architecture, so that organizational and technical forces push in the same direction. · The book identifies hidden monoliths in databases, builds, releases, models, standards and even office layouts. · Uber's domain-oriented architecture shows how grouping services into domains with narrow gateways can restore the independence that fragmented microservices had lost. Review Questions 1. State Conway's Law in your own words and explain the reasoning Conway gave for it. 1. Why do Skelton and Pais argue that encouraging every team to communicate with every other team can harm architecture? 2. Describe the reverse Conway maneuver and identify one reason it may fail in practice. 3. Choose two of the hidden monoliths in Table 1 and explain how each constrains team independence. 4. How does Uber's use of domains and gateways illustrate the principle of limiting communication between teams to well-defined interfaces? 5. What do the exceptions documented by Colfer and Baldwin suggest about the limits of Conway's Law? Chapter 3: Team-First Thinking Having shown why team structure shapes software, Team Topologies turns to the team itself. Skelton and Pais argue that the team, rather than the individual, should be the fundamental unit of delivery in a software organization. This chapter examines what they mean by a team, why they place so much weight on team size and stability, how they think about ownership, and what they mean by a team's API. Together these ideas make up what the authors call a team-first approach, and they set the stage for the discussion of cognitive load in the next chapter. Why the team is the unit of delivery Modern software systems are too large and complex for any single person to understand in full. They must be built and run by groups. The question is what kind of group works best. Skelton and Pais argue for small, stable teams whose members work closely together towards a shared goal. Such a team, they suggest, can achieve more than the sum of its members because its members share context, cover for one another and build on one another's knowledge. The book defines a team in a demanding way. A team is not simply a set of people who report to the same manager or who happen to be assigned to the same project. It is a stable group of people working together towards a shared goal, whose members depend on one another and are collectively responsible for results. By this definition, many so-called teams in large organizations are not teams at all. A pool of developers from which individuals are allocated to projects is not a team. Nor is a department of specialists who each take tickets from a queue. The authors support their emphasis on teams by pointing to research on group performance. This guide adds that the idea has broad support. Google's internal study of team effectiveness, known as Project Aristotle and publicly described by the company in 2015 and 2016, found that who was on a team mattered less than how the team worked together. The most important factor it identified was psychological safety, a concept developed by the Harvard researcher Amy Edmondson: a shared belief that team members can take interpersonal risks, such as admitting mistakes or asking questions, without being punished. Psychological safety takes time to build, and it depends on stable relationships. This connects directly to the book's insistence on long-lived teams. Team size and Dunbar's number How big should a team be? Skelton and Pais answer with a mix of industry experience and anthropology. From experience, they note that effective software teams are typically small. They suggest a size of around five to eight people, with an upper bound of around fifteen for teams with unusually high trust. Beyond this, the members can no longer maintain close working relationships with everyone else on the team. The best-known industry expression of this idea is Amazon's two-pizza team. The phrase is associated with Jeff Bezos and refers to the principle that a team should be small enough to be fed by two pizzas. Skelton and Pais cite the practice approvingly. The point is not the pizza but the limit: a team that grows past a certain size starts to fragment into sub-groups, communication overhead rises, and the team loses the shared understanding that made it effective. The anthropological grounding comes from the work of the British anthropologist Robin Dunbar. In research published in the early 1990s, Dunbar related the size of the neocortex in primates to the size of their social groups, and predicted that humans could maintain stable social relationships with roughly 150 people. This figure became known as Dunbar's number. Dunbar and colleagues later proposed that human social networks form a series of layers, each roughly three times the size of the one inside it: around 5 people with whom we are most intimate, around 15 close friends, around 50 good friends and around 150 meaningful contacts, with further layers of acquaintances beyond. Skelton and Pais use these layers as a guide for organizational design, as Table 2 sets out. The smallest layers suggest the natural size of a team in which members can trust one another deeply. The larger layers suggest natural limits for groupings of teams, such as a department or a business unit, within which people can still know one another. Beyond about 150 people, the authors suggest, an organization should expect relationships to become more formal and should design its interactions accordingly. Table 2. Dunbar's social layers as applied to team design. Approximate size Social meaning (Dunbar) Organizational use (Skelton and Pais) 5 Closest personal relationships Core of a small, high-trust team 15 Close friends Upper limit for a single high-trust team 50 Good friends Grouping of several teams with mutual trust 150 Stable social relationships Upper limit for a unit whose members know each other 500 and above Acquaintances Larger divisions needing formal interfaces The authors draw a further practical conclusion. Because these limits are real, organizations should watch what happens when a unit crosses them. A department that grows from 40 to 60 people, or a business unit that grows beyond 150, will find that the informal trust that held it together no longer reaches everyone. That is the moment to consider splitting the unit or making its internal interactions more explicit. The same logic applies to software. When a system becomes too large for a single team, the team should not simply grow. The system should be divided, and new teams formed around the parts. This guide notes that Dunbar's number has attracted scholarly criticism. Some researchers have questioned the statistical basis of the original estimate and argued that the range of plausible group sizes is very wide. For the purposes of Team Topologies, however, the precise figures matter less than the underlying principle. Human capacity for close working relationships is limited, and organizational design should respect that limit rather than assume it away. Stable, long-lived teams The second pillar of team-first thinking is stability. Skelton and Pais argue that teams should be kept together over long periods, and that work should flow to teams rather than people being moved to work. This reverses a common practice. In many organizations, especially those that fund technology through projects, people are assembled into a project team, disbanded when the project ends and reassigned to new projects. The authors see this as wasteful. Their reasoning draws on the psychology of group formation. It takes time for a group of people to become an effective team. They must learn one another's strengths and habits, develop shared norms and build trust. The psychologist Bruce Tuckman described this process in 1965 in his well-known sequence of forming, storming, norming and performing. Skelton and Pais refer to this model, while also noting that later research suggests teams do not move through the stages in a neat order. Conflict and renegotiation can recur throughout a team's life. Either way, the time needed to become effective is lost every time a team is broken up. A team that is dissolved just as it reaches high performance throws away its most valuable asset. Stability does not mean that team membership never changes. People leave, join and move for good reasons, including their own development. The authors' point is that change should be occasional and careful, so that the team's core remains intact. They suggest that a healthy team can absorb the loss or addition of a member without losing its identity, whereas a team whose membership turns over constantly never becomes a team at all. Stable teams also require a change in how work is funded and planned. If teams are long-lived, then organizations must direct a flow of work to them over time rather than creating teams for each new initiative. This is one reason why the book aligns naturally with product-based and value-stream-based funding models, in which money is allocated to a persistent team or product area rather than to a time-limited project. Ownership and the team-first mindset The third pillar is ownership. Skelton and Pais argue that every piece of software in an organization should be owned by exactly one team. Ownership here does not mean exclusive access. Other teams may read the code, suggest changes or even contribute changes under agreed rules. It means that one team is responsible for the software's health, its evolution and its behaviour in production. The value of single-team ownership can be explained with an analogy from farming, close in spirit to one the authors use. A farmer who holds land for a single season has little incentive to care about the long-term health of the soil. A farmer who holds the same land for many years will care about it. Software is similar. A team that owns a service over the long term will care about its maintainability, test coverage and operational health, because it will live with the consequences. A service that is touched by many teams but owned by none will decay, because nobody is responsible for its long-term condition. Single-team ownership also has a Conway dimension. If a piece of software is owned by one team, its internal design reflects that team's communication, which is dense and coherent. If it is shared among several teams, its design reflects their communication, which is likely to be thinner and more fragmented. Shared ownership therefore tends to produce software with unclear internal boundaries. Ownership requires that teams have the capability to look after what they own. A team that owns a service in production needs to be able to deploy it, monitor it and fix it. This is the principle often summarized by the phrase "you build it, you run it", which the Amazon chief technology officer Werner Vogels used in a 2006 interview to describe how Amazon's teams took operational responsibility for their own services. Skelton and Pais embrace the principle, while recognizing that teams need support, in the form of platforms and specialist help, to carry it without being overwhelmed. The authors also describe the behaviour they expect from members of a team. A team-first mindset means that members put the needs of the team above their own individual preferences. They attend team events, help one another, share knowledge and accept collective responsibility. The authors are frank that some individuals, however talented, find this difficult, and that organizations may need to address such cases directly. They also argue that diversity within teams helps, because people with different backgrounds and perspectives tend to generate more ideas and catch more problems. Finally, they recommend that organizations reward teams rather than individuals where possible, so that incentives support collaboration rather than competition within the team. Stable teams bring one further benefit that is easy to overlook: they make the organization predictable to itself. When teams persist, other teams learn whom to ask about what, and the relationships between teams can be designed and improved over time. When teams are constantly re-formed, every relationship must be rebuilt from scratch, and the organization's map of who owns what is always out of date. Stability inside teams is therefore a precondition for designing the interactions between them, which is the subject of the team API below and of the interaction modes later in this guide. The team API The most distinctive idea in this part of the book is the team API. In software, an application programming interface, or API, is the defined set of ways in which other programs can interact with a component. It hides the component's internal details and exposes only what others need. Skelton and Pais apply the same idea to teams. A team's API is the set of ways in which other teams can interact with it. It includes the obvious technical elements, such as the code the team owns and the service interfaces it offers, along with their versioning and documentation. It also includes less technical elements. These are the team's working practices and principles; its roadmap and current priorities; its preferred communication channels, such as a particular chat channel or regular office hours; and the way it wants other teams to request changes. A well-designed team API makes it easy for other teams to work with the team without needing to understand its internal workings or to interrupt its members constantly. The concept is useful because it turns vague notions of good collaboration into something that can be designed and improved. A team can ask itself whether its API is clear, whether other teams know how to reach it, whether its documentation is current and whether its interfaces are stable. Other teams can give feedback on the API, just as users of a software API would. The authors suggest that teams should think explicitly about which other teams they need to interact with and how, and should design their API to support those interactions. The team API also serves the book's broader goal of limiting unnecessary communication. If every interaction with a team requires a meeting or a conversation, communication overhead grows quickly as the organization grows. If most interactions can happen through clear, self-service interfaces, teams can work together without constant coordination. This is the organizational counterpart of loose coupling in software. Designing the environment for teams Skelton and Pais close their account of team-first thinking by turning to the physical and virtual environment. They argue that office layouts and digital tools should be designed around teams. Members of the same team should be able to work closely together, with space to discuss and collaborate. Teams that need to interact should be placed near each other, and teams that do not need to interact need not be. In a remote or hybrid setting, the same principles apply to chat channels, video tools and shared documents, which should be organized to support team boundaries and the interactions between teams. This point was made before the widespread shift to remote work in 2020. In 2022 the authors published the Remote Team Interactions Workbook, which applies the Team Topologies ideas to distributed teams. The underlying message is unchanged: the environment, whether physical or digital, shapes communication, and therefore, by Conway's Law, shapes the software. Key Takeaways · Skelton and Pais treat the stable, small team as the fundamental unit of software delivery, and define a team demandingly as a group with a shared goal and collective responsibility. · Team size should typically be around five to eight people, with Dunbar's social layers of roughly 5, 15, 50 and 150 offering guidance for teams and groupings of teams. · Teams should be long-lived, with work flowing to teams rather than people being shuffled between projects, because effectiveness takes time to build. · Every piece of software should be owned by exactly one team, which is responsible for its long-term health, in the spirit of "you build it, you run it". · The team API describes how other teams interact with a team, covering code, documentation, practices, roadmap and communication preferences. · Physical and digital workspaces should be designed around team boundaries and intended interactions. Review Questions 1. How do Skelton and Pais define a team, and why do many groups in large organizations fail to meet that definition? 1. Explain how Dunbar's research informs the book's guidance on team size and on groupings of teams. What criticism has Dunbar's number attracted? 2. Why do the authors argue that work should flow to stable teams rather than people moving between projects? 3. What does single-team ownership mean, and why does it tend to improve the long-term health of software? 4. List the elements of a team API and explain how a well-designed team API reduces communication overhead. Hashtags: #TheArchitectureOfFlow #TeamTopologies #FastFlow #SocioTechnicalSystems #ConwaysLaw #ReverseConwayManeuver #CognitiveLoad #TeamFirstThinking #StreamAlignedTeams #EnablingTeams #ComplicatedSubsystemTeams #PlatformTeams #TeamInteractionModes #Collaboration #XAsAService #Facilitating #TeamAPIs #StableTeams #SingleTeamOwnership #FracturePlanes #LooseCoupling #ValueStreams #OrganizationalDesign #SoftwareArchitecture #FutureOfTeamTopologies

Download the Book (PDF): Introduction Somewhere on every campus, right now, someone is composing an apology and deleting it. A student has read the message from her roommate three times, typed "I'm sorry but," looked at the "but," and put the phone face-down on the desk. A second-year who missed his turn presenting for a group project is drafting an email to the professor that opens with a paragraph about his grandmother's health, a paragraph about his laptop, and a paragraph about the bus. Two friends who have not spoken since a party a fortnight ago are each waiting for the other to go first, and each is privately rehearsing a version of events in which they were mostly right. None of this is dramatic. That is the point. The conflicts that fill a university year are rarely the stuff of tribunals. They are dishes left in a shared sink, a joke that landed badly in a seminar, a screenshot forwarded to the wrong chat, a deadline missed without warning, a comment about someone's accent, a borrowed jacket returned with a stain, a promise to split the rent on time that was not kept. They are small, and they are constant, and they are inevitable wherever several thousand young adults are placed in close quarters, handed demanding work, and asked to cooperate with people they did not choose. What decides whether such episodes pass or fester is, surprisingly often, a single conversation: whether someone says, clearly and without evasion, that they got something wrong, and what they intend to do about it. This booklet is about that conversation. Its argument is simple to state and harder to practise. An apology is not a ritual of self-abasement, a legal admission, or a social lubricant to be dispensed as freely as "no worries." It is a repair tool with a specific structure. It works when it names the harm accurately, accepts responsibility without defence, and commits to changed behaviour, and it fails when any of those parts is missing or counterfeit. On campus, where relationships are dense, repeated, and often unequal in power, learning to do this well and in proportion is one of the most practically useful things a student can learn, and very few are ever taught it. Why the campus is a special case People apologise everywhere, so it is fair to ask why the university setting deserves its own treatment. There are four reasons, and they recur throughout the chapters that follow. The first is density. A student may live, eat, study, work, and socialise inside a few square kilometres, with the same few hundred people appearing in every one of those settings. The person you snapped at in the kitchen is in your nine o'clock lecture, is dating your lab partner, and will be at the same party on Saturday. In most of adult life we can quietly let a damaged acquaintance drift. On campus that option is often unavailable, which makes unrepaired conflict unusually expensive: you pay for it every time you see the person, and you see them constantly. The second is repetition. Campus relationships run on cycles of a term, a year, a degree. A roommate is someone you will share a bathroom with for nine months. A doctoral supervisor is someone whose signature you will need for five years and whose reference you may need for fifteen. When a relationship has a long future, the way a single breach is handled shapes everything after it. Researchers who study trust describe this in terms of the "shadow of the future": the longer a relationship is expected to continue, the more each party has to gain from repairing it and the more each has to lose from letting a grievance harden. The third is asymmetry. Students deal daily with people who hold real power over them: lecturers who mark their work, teaching assistants who grade their labs, resident advisers who enforce housing rules, employers in campus jobs, admissions and funding panels. Apologising to a friend and apologising to the person who decides your grade are different acts with different risks, and the same words can land very differently depending on which direction they travel. The fourth is that students are, for the most part, still learning how to do this. Many are living without family around them for the first time. Many are negotiating shared space with a non-relative for the first time. Many arrive from school cultures in which conflict was managed for them by teachers or parents, and they find that at university nobody steps in. That is not a criticism. It is simply the situation in which the skill has to be acquired, often through painful trial and error. What this booklet does and does not cover The chapters move from principles to practice. The first two set out what an apology is for and what its working parts are, drawing on a body of research in social psychology, negotiation, and moral philosophy that has become quite specific about what makes apologies succeed. The third looks at why apologising is so hard, and at the familiar forms of the non-apology that people reach for instead. The fourth deals with timing, channel, and proportion: when to apologise, whether to do it in person or in writing, and how to avoid both the grudging under-apology and the anxious habit of apologising for everything. The next three chapters apply those principles to the three relationships the title names. Roommates and housemates present a distinctive problem: harms that are small, repeated, and embedded in routine, where an apology that does not change the routine is worthless. Friendships, and the group chats and group projects that surround them, raise questions about public and private repair, about digital permanence, and about harms done to someone who was not even present. Apologising to professors and teaching staff requires a different register altogether, one that respects the asymmetry of power without collapsing into grovelling or over-explanation, and that handles the most delicate case of all, the apology connected to academic misconduct. The final chapter turns the question around. It considers what to do when you are the one who is owed an apology, why forgiveness is not a duty, and what apology cannot fix. Some harms that occur on campus, including harassment, discrimination, and sexual misconduct, are not conflicts to be resolved between two people over coffee. They belong to formal processes, and an apology in those cases can be at best a small part of a much larger response and at worst a form of further pressure on the person harmed. Knowing where the tool stops is part of knowing how to use it. Throughout, the examples are composites: typical situations drawn from the ordinary run of university life, with invented names, rather than accounts of real individuals. Where research is cited, it is real, and the studies are described as they were actually conducted, including their limits. Much of the experimental work on apology uses scenarios, vignettes, and student samples, and it should be read as strong evidence about how people tend to respond rather than as a formula that guarantees any particular outcome. People are not vending machines. A perfectly constructed apology can still be refused, and a clumsy one offered with evident sincerity can still be accepted. What the research offers is a way of stacking the odds. A note on the word itself In ordinary speech "sorry" does many jobs. It expresses sympathy ("I'm sorry your exam went badly"), it softens a request ("Sorry, could you pass that?"), it signals embarrassment, it fills silence. None of those uses is wrong, and one later chapter looks at the surprisingly useful social work that such "superfluous" apologies do. But the apology this booklet is mostly concerned with is narrower. It is the act of acknowledging to another person that you did something that wronged or harmed them, that you were responsible for it, and that you regret it, in a way that aims to restore something the offence damaged. It is an act of communication with a purpose, and like any such act it can be done skilfully or badly. Done skilfully, it is also one of the more generous things one person can do for another. It tells the other person that their experience was real, that their standing matters, and that the relationship is worth the discomfort of admitting fault. Those are not small messages, and in a place as crowded and consequential as a university, they are worth learning to send well. Chapter 1: What an Apology Is For Consider two students who share a flat. Jonah borrows Priya's bicycle without asking, leaves it unlocked outside the library, and it is stolen. When she finds out, he says, immediately and with real distress, "I'm so sorry." She says it is fine. A week later she is still cold with him, and he cannot understand why. He apologised. She accepted. What more is there? The answer is that Jonah said the word without doing the work the word stands for. He expressed regret, which is real, but he said nothing about what he had actually done wrong, which was not losing the bicycle but taking it without permission in the first place. He did not acknowledge that she now had no way to get to her placement across town. He did not offer to replace it or pay toward a new one. He did not say anything about whether he would take her things again. Priya's "it's fine" was not acceptance; it was the reflexive reply that social life demands when someone says sorry, and her continued coldness was her honest verdict that nothing had been repaired. This small episode contains most of the problem this booklet addresses. People commonly treat an apology as a single speech act, a password that unlocks forgiveness. In fact it is a compound act that does several distinct jobs, and when some of those jobs are left undone, the word "sorry" does not cover the gap. To apologise well, it helps first to understand what an apology is actually for. Three things an offence damages When one person wrongs another, even in a small way, three things are typically damaged at once. An effective apology has to address all three, though not always with equal weight. The first is the injured person's standing. Being wronged carries an implicit message: you did not count for enough to be considered. If a housemate eats the food you bought for the week, the practical loss may be a few pounds or dollars, but the message is that your claims on shared space were not taken seriously. The psychiatrist Aaron Lazare, whose book On Apology remains one of the most careful treatments of the subject, argued that the restoration of the offended party's self-respect and dignity is among the most important needs an apology meets. An apology says, in effect: you did matter, and I was wrong to act as if you did not. This is why apologies that focus entirely on the offender's feelings ("I feel terrible, I'm such an idiot") often leave the recipient cold. They are about the wrong person. The second is the shared rules. Every relationship rests on a set of expectations, many of them never spoken: that friends do not share each other's secrets, that roommates ask before having overnight guests, that group members do their assigned part, that students submit their own work. An offence breaks one of those rules, and in doing so it raises a question about whether the rule still holds. If it is never acknowledged, the rule has in effect been rewritten. The philosopher Nick Smith, in I Was Wrong, treats this as one of the core functions of what he calls a "categorical" apology: the offender explicitly affirms the moral principle that was violated, and so reinstates it. When Jonah fails to say "I shouldn't have taken your bike without asking," he leaves open the possibility that he thinks it was acceptable, and Priya is left not knowing whether her belongings are safe. The third is trust, meaning the injured person's expectation about how the offender will behave in future. An offence is evidence. It tells the victim something about what the offender is willing to do, and a rational victim updates accordingly. An apology is, among other things, an attempt to supply counter-evidence: a signal that the offence was not representative, that the offender understands why it was wrong, and that it will not recur. This is why promises about future behaviour, and concrete offers of repair, carry so much weight. Words of regret cost little. Commitments that can be checked cost more, and people know the difference. Seeing apology as the repair of standing, rules, and trust explains a great deal about why some apologies work and others do not. An apology that restores standing but says nothing about the future leaves trust damaged. An apology that promises to do better but never names the rule that was broken leaves the recipient uncertain what exactly will be different. An apology that affirms the rule but is delivered with irritation or haste fails to restore standing, because its manner contradicts its content. What an apology is not Several things that look like apologies are something else, and it is worth separating them early because they account for most of the frustration people feel when "sorry" fails. An apology is not an expression of sympathy. "I'm sorry you feel that way" and "I'm sorry that happened to you" are statements of regret about a state of affairs, not acceptances of responsibility for causing it. There is nothing wrong with sympathy, but when it is offered in place of an apology it is usually heard, correctly, as a refusal to accept fault. An apology is not an explanation. Explanations have a place, and a later chapter looks at when an account of what happened helps and when it hurts. But an explanation answers the question "why did this happen?" whereas an apology answers "whose fault was it and what now?" People who have been wronged often experience a long explanation as a defence brief, and they are frequently right. An apology is not a transaction that obliges forgiveness. Offering an apology does not create a debt the other person must repay by forgiving, and apologies that come with an implied demand ("I said sorry, what more do you want?") tend to reopen the wound. The recipient is entitled to take time, to accept partially, or not to accept at all. The final chapter returns to this at length. An apology is not self-punishment. Some people, especially conscientious and anxious people, apologise in a way that turns the conversation into a display of their own suffering. They berate themselves, catalogue their failings, and may end up needing reassurance from the very person they wronged. This is a real and understandable response to guilt, but it reverses the direction of care. The injured party ends up comforting the offender. And an apology is not a formula. The research described in the next chapter identifies components that reliably make apologies more effective, and it is useful to know them. But an apology assembled from correct parts and delivered without understanding will usually be detected. What people respond to, underneath the structure, is evidence that the offender has genuinely grasped what they did and why it mattered. Guilt, shame, and the direction of attention Why do some people find it natural to focus on the other person when they apologise, while others slide into self-focus or defensiveness? One useful answer comes from the psychology of moral emotions, and in particular from the distinction between guilt and shame developed by June Price Tangney and her colleagues and set out in Shame and Guilt, which she wrote with Ronda Dearing. In Tangney's account, guilt and shame are both painful self-conscious emotions that follow wrongdoing, but they differ in what they are about. Guilt is focused on a behaviour: "I did a bad thing." Shame is focused on the self: "I am a bad person." The distinction sounds subtle, but it has consequences. Across a large body of research, guilt has been associated with a tendency to take reparative action, to apologise, confess, and try to put things right, because a bad action can be undone or compensated. Shame, by contrast, has been associated with hiding, withdrawal, and defensive anger, because if the self is the problem, there is nothing to repair; the only options are to escape or to deny. This helps explain both of the failure modes described above. The defensive non-apology, in which the offender minimises, justifies, or counter-attacks, is often shame in disguise: admitting the offence feels like admitting to being a bad person, so the offence must be denied. And the self-flagellating apology is often shame expressed directly: the offender cannot separate what they did from who they are, so the conversation becomes about their worth rather than about the harm. The practical upshot is that the most useful internal stance for apologising is a kind of focused guilt. You did a specific thing, it had a specific effect on a specific person, and you can do specific things about it. That frame is both more accurate and more bearable than the global judgement that shame delivers, and it points attention outward, toward the person who was harmed and what they need. One line of research lends this some support. Juliana Breines and Serena Chen, in a set of studies published in 2012, found that participants who were prompted to think about a past moral transgression with self-compassion, rather than with self-esteem boosting or no instruction, reported greater motivation to make amends and to avoid repeating the behaviour. Treating yourself decently after a mistake, it seems, does not let you off the hook. It makes it easier to stay on it. Small offences, large meanings One consequence of this three-part view is that the size of an offence and the size of the damage it does are only loosely related. A trivial act can carry a heavy message, and a costly accident can carry almost none. Take a seminar. In a discussion of a set reading, a student named Tomás makes an argument, and another student, Ellie, cuts across him with "that's not really what the text says, though," then goes on to make almost the same point in different words. The tutor praises Ellie's contribution. Nothing material has been lost. Tomás's grade is not affected. By any external measure the offence is tiny. But Tomás, who is the only first-generation student in the group and who spent an hour preparing that point, leaves the room convinced that he is seen as someone whose ideas do not count. The damage to his standing is real and out of all proportion to the act, and it will shape whether he speaks in the next seminar. Now take a costly accident. Two students are carrying a housemate's television down the stairs during a move, and one of them stumbles and it cracks. The loss is several hundred pounds or dollars. But if the stumbling student says at once, "I'm so sorry, that was me, I'll sort out the cost," the damage to standing and to the shared rules is close to zero. Nobody's claims were ignored and no rule was broken. What remains is a practical problem about money, which an offer of repair addresses directly. The lesson is that an apology should be calibrated to the damage as the injured person experiences it, not to the objective size of the act. This is harder than it sounds, because the offender's natural instinct is to measure the offence by their own intentions ("I didn't mean anything by it") and by its material consequences ("but it didn't actually affect you"). Neither measure captures what the other person lost. Ellie, if she noticed Tomás's silence afterwards, would do far better with a short, specific acknowledgement ("I cut you off in the seminar and then basically repeated your point; that wasn't fair to you") than with either a lengthy apology that makes the moment awkward or a decision that it was too minor to mention. The same principle works in reverse. Where the damage is almost entirely practical, a long emotional apology is often unwelcome. The housemate with the cracked television does not need a speech about how terrible the carrier feels; they need to know the cost will be covered. Matching the apology to the actual damage, standing, rules, trust, or simply material loss, is one of the recurring skills this booklet returns to. It also explains why the same act can require quite different apologies to different people. Reading someone's private messages over their shoulder might be a minor lapse to one friend and a serious breach to another, depending on what those messages were and what the friend's history with privacy has been. There is no universal price list for offences. The only reliable way to know what was damaged is to pay attention to the person, and, where it is unclear, to ask. Chapter 4 looks at how to do that without making the conversation about yourself. Why the campus raises the stakes The general functions of apology apply everywhere, but the university setting intensifies each of them. Standing matters acutely to people in their late teens and early twenties, who are in the process of establishing adult identities among peers they have only recently met. A slight that would be shrugged off by a settled forty-year-old can feel, to a first-year student eight weeks into a new city, like a verdict on whether they belong. This is not over-sensitivity. It is an accurate reading of a situation in which social position really is being worked out, and in which there is often no established network of old friends to fall back on. Shared rules are especially unclear on campus. A residence hall brings together people from different family cultures, different countries, and different assumptions about noise, cleanliness, guests, borrowing, privacy, and money. Many conflicts arise not because anyone intended harm but because two people were operating under different rules without knowing it. An apology in that situation does double duty: it repairs the breach and, if done well, makes the rule explicit for the first time. Trust has a long future. As the introduction noted, students are locked into relationships for terms and years. A friendship that survives a serious breach in the first year may become one of the defining relationships of a person's life. A working relationship with a supervisor that is damaged early and never repaired can shadow a whole degree. And the audience is often larger than two. On campus, conflicts are rarely private. Housemates take sides. Friend groups divide. Group chats relay events to people who were not present. Course cohorts notice who is not speaking to whom. That means apology on campus often has a social dimension that apologies in more dispersed settings lack: it repairs not only a relationship between two people but also a small public understanding of what happened. The case for learning to do it well It would be easy to treat everything in this chapter as common sense. Most people know, in principle, that they should own their mistakes. But the gap between knowing and doing is wide, for reasons Chapter 3 explores, and the cost of the gap on campus is high. A housemate conflict that nobody repairs can make a student dread going home for the rest of the year. Friendship ruptures in the first year can leave a student isolated at exactly the point when isolation is most harmful. Poorly handled lapses with teaching staff can turn a recoverable mistake into a lasting reputation. Against those costs, the skill of apologising well is cheap to acquire and pays off immediately. It does not require a change of personality. It requires understanding a handful of things: what the injured person needs, what the working parts of an apology are, what reliably goes wrong, and how the answers shift across the different relationships of campus life. The next chapter begins with the working parts. Before that, return briefly to Jonah and Priya. What would a better apology have sounded like? Something like this: "I took your bike without asking, and that was wrong; it's yours and I should have checked. Then I left it unlocked and it got stolen, which means you can't get to your placement. That's on me. I want to pay for a replacement, and until it arrives I'll cover your bus fares. And I won't borrow your things again without asking first." That apology is not eloquent. It takes about twenty seconds to say. It does not dwell on Jonah's feelings, though they are evident. But it restores Priya's standing by naming her ownership, it reaffirms the rule about asking, and it gives her concrete evidence about the future. It is the kind of apology after which "it's fine" might actually mean it. Chapter 2: The Working Parts If the previous chapter described what an apology is for, this one describes what it is made of. The question sounds almost too simple to research, but over the past three decades psychologists, negotiation scholars, and philosophers have taken it seriously, and they have arrived at a fairly consistent picture. Apologies are not all-or-nothing. They are built from recognisable elements, the elements do not carry equal weight, and which ones matter most depends in part on who is receiving the apology and what was damaged. Six elements, ranked The most frequently cited study of apology structure was published in 2016 by Roy Lewicki, Beth Polin, and Robert Lount in the journal Negotiation and Conflict Management Research. Drawing on earlier work, they identified six elements that commonly appear in apologies: an expression of regret, an explanation of what went wrong, an acknowledgement of responsibility, a declaration of repentance, an offer of repair, and a request for forgiveness. They then tested how much each element contributed to an apology's perceived effectiveness. Across two experiments involving 755 participants, 333 adults recruited online and 422 undergraduates, people read a scenario about a job applicant who had made an error on a previous employer's tax filing and then evaluated apologies containing different combinations of the six elements. Two findings stood out. First, apologies containing more of the elements were rated as more effective, credible, and adequate than apologies containing fewer. Second, the elements were not equally valuable. The most important by a clear margin was acknowledgement of responsibility. The second was an offer of repair. Expression of regret, explanation, and declaration of repentance sat in a middle tier. The least important was the request for forgiveness. Lewicki's own summary of the practical lesson was blunt: say it is your fault, that you made a mistake, and if you have to leave something out, leave out the request for forgiveness. The study has limits worth stating. It used a written workplace scenario rather than a real relationship, and participants were evaluating a stranger rather than someone they cared about. Real apologies are delivered with faces, voices, and histories, and those can outweigh any structural feature. But the ranking fits well with what other research and ordinary experience suggest, and it provides a sensible starting point. Table 1 sets out the six elements with what each one does and how it tends to sound when done well and badly in a campus setting. Table 1. The six elements of an apology, as identified by Lewicki, Polin and Lount (2016). Element Relative weight What it does Weak version Stronger version Acknowledging responsibility Highest Places fault clearly with the speaker "Mistakes were made with the rota" "I skipped my cleaning week twice" Offer of repair High Converts regret into action "Let me know if I can help" "I'll do the next two weeks" Expression of regret Middle Shows the speaker cares about the harm "My bad" "I'm genuinely sorry" Explanation Middle Makes the lapse intelligible A long list of pressures One sentence, no excuses Declaration of repentance Middle Commits to future change "It won't happen again" "I've set a reminder for my weeks" Request for forgiveness Lowest Invites the relationship to resume "So we're good, right?" "I hope you can forgive me" The rest of this chapter takes the elements in order of importance, because the order matters for how an apology is built. Acknowledgement of responsibility The core of an apology is a clear statement that you did something and that it was wrong. Everything else is supporting material. The reason this element carries so much weight is that it is the one most directly aimed at the injured person's standing and at the shared rules. By saying "I did this, and I shouldn't have," the offender confirms that the victim's grievance is legitimate and that the rule they relied on still holds. Three features distinguish a strong acknowledgement from a weak one. The first is agency. Grammar matters here. "The deadline got missed" and "I missed the deadline" describe the same event, but only the second has a person in it. The passive voice, which appears with suspicious frequency in institutional and political apologies, removes the actor from the sentence. In personal apologies, the equivalent move is to attribute the offence to circumstances ("things got hectic"), to a state of mind ("I wasn't myself"), or to a vague shared failure ("we both let things slide"). Sometimes those are partly true. But if they appear in place of a first-person admission rather than after it, they are heard as evasions. The second is specificity. "I'm sorry for everything" or "I'm sorry for how I've been" is almost always weaker than "I'm sorry I told Sam what you said about your parents." Specificity does two things. It shows the offender has actually understood what happened from the victim's point of view, rather than grasping only that the victim is upset. And it bounds the admission, which is reassuring to both parties: it makes clear what is being owned and what is not. Vague apologies can feel oddly threatening to recipients precisely because they are unbounded; the victim cannot tell whether the offender understands the real problem or is simply trying to make the discomfort stop. The third is naming the impact, not only the act. "I shared your message in the group chat" is an admission. "I shared your message in the group chat, and now people you barely know have read something you meant for me" shows that the offender has followed the act through to its consequence for the other person. That second step is often where apologies become genuinely restorative, because it is the step that tells the victim they have been seen. Offer of repair The second most important element is an offer to make things right. Its power lies in the fact that it costs something. Anyone can say sorry; offering to pay, to redo, to replace, to take on an extra share, or to put right a public account of events requires the offender to give up time, money, or face. That cost is what makes it credible. In the language of economics, it is a costly signal, and costly signals are believed in a way that cheap talk is not. A good offer of repair has three properties. It is concrete: not "let me know if there's anything I can do," which places the burden on the injured person to invent a remedy, but a specific proposal. It is proportionate: roughly matched to the harm, neither a token gesture for a serious loss nor an extravagant gesture for a small one, which can feel like an attempt to buy one's way out. And it is responsive: offered in a way that leaves room for the other person to say what would actually help. "I'd like to replace the book I damaged; would you rather I buy a new copy or give you the money?" is concrete and responsive at once. Not every offence admits of material repair. You cannot un-say a hurtful remark. But even then, repair is often possible in another currency. If you embarrassed someone in public, repair might mean correcting the record in front of the same audience. If you broke a confidence, it might mean going back to the person you told and explaining that you were wrong to pass it on. If you let a group down, it might mean taking on the next difficult task. The question to ask is: what did my action cost this person, and what can I give back that addresses that cost? The middle tier: regret, explanation, and repentance The three elements in the middle tier matter, but they are more easily overdone and their value is more dependent on delivery. Regret is the emotional register of the apology, the words "I'm sorry" or "I really regret it." It matters because it signals that the offender cares about the harm, not merely that they are conceding a point. Its absence is conspicuous: an apology that acknowledges responsibility in a flat, procedural tone can feel like a legal admission rather than a human one. But regret is also the element most prone to inflation. Excessive expressions of regret, particularly when they drift toward self-punishment, shift the focus to the offender. A sentence or two, sincerely meant, is usually enough. Explanation is the most double-edged element. On one hand, people who have been wronged often want to understand why. An unexplained offence is frightening because it is unpredictable; knowing that a friend forgot your birthday because their grandfather was in hospital that week makes the lapse intelligible and non-threatening. On the other hand, explanations slide easily into excuses. The difference lies partly in length and partly in placement. An explanation that comes after a clear acknowledgement of responsibility, and that is brief, is heard as context. An explanation that comes first, or that runs longer than the acknowledgement, is heard as defence. A useful rule is that an explanation should never contain the word "but" in a way that qualifies the admission: "I missed the meeting because my train was cancelled, but I should have messaged you" works; "I'm sorry I missed the meeting, but my train was cancelled" undoes the apology in its second half. Declaration of repentance is the promise not to do it again. It is aimed at trust, and it is strongest when it is specific and checkable. "It won't happen again" is a promise almost everyone has heard broken, and on its own it carries little weight. "I've put your birthday in my calendar with a week's warning" or "I'm going to stop borrowing things from the kitchen without asking, and if I do need something I'll text first" are declarations the other person can actually observe being kept. The more specific the commitment, the more it functions as evidence rather than aspiration. The request for forgiveness The least valuable element, according to Lewicki and his colleagues, is the request for forgiveness. This can seem surprising, since forgiveness is in a sense the point of the exercise. But the finding makes sense once you notice what the request does. It shifts the burden to the injured person, who now has to respond, and it can feel like pressure. A request for forgiveness offered too soon, or too insistently, turns a gift into a demand. This does not mean the request should never be made. In close relationships, saying "I hope you can forgive me" can be an honest expression of how much the relationship matters. The key is that it should be phrased as a hope rather than a question requiring an immediate answer, and that the offender should be prepared to accept that the answer may be "not yet." The request is also best left to the end, after everything else has been said, and it is the element most safely dropped when time or circumstance is short. Matching the apology to the person One further finding refines the picture. People differ in what they most need from an apology, and some of those differences are systematic. In a 2010 study published in Organizational Behavior and Human Decision Processes, Ryan Fehr and Michele Gelfand examined how the effectiveness of different apology components depended on the recipient's "self-construal," the extent to which people define themselves as independent individuals, in terms of close relationships, or in terms of larger groups. They found that offers of compensation were especially effective for people with a strongly independent self-construal, expressions of empathy for those with a strongly relational one, and acknowledgements that a rule or norm had been violated for those who defined themselves by group membership. For students this has a practical implication. A campus contains people from many cultural backgrounds and many family styles, and the apology that feels complete to you may feel incomplete to the person receiving it. Research on cultural differences in apology supports the same caution. William Maddux and colleagues, comparing American and Japanese participants in work published in 2011, found that Japanese participants were more likely to see apology as an expression of remorse and a way of repairing a relationship even when they were not personally at fault, while American participants were more likely to associate apology with an admission of personal blame. Neither view is wrong. But a student who apologises by offering to pay for damage, when the other person was mainly hurt by what the damage said about the friendship, will find the apology falls flat, and so will one who offers a heartfelt emotional apology to someone who simply wants the broken thing replaced. The practical answer is not to memorise cultural profiles, which is a good way to stereotype people. It is to include more than one component, since the research consistently finds that fuller apologies outperform thinner ones, and to pay attention to what the other person responds to. If a housemate brightens when you offer to replace something but remains distant when you talk about how bad you feel, that is information. If a friend seems more hurt by the fact that you did not ask how they were than by the practical consequences, that is information too. Putting the parts together In practice, a sound apology often follows a natural order: acknowledge what you did and its effect; express regret; offer repair; say what will be different; and, if appropriate, express the hope of forgiveness. Explanation, if needed, is best slotted in briefly after the acknowledgement. The whole thing can be short. The components are not a checklist to be recited but a way of checking, before you speak, whether anything essential is missing. Consider how this works in a real case. Aisha is one of four students on a marketing group project. She agreed to write the competitor analysis by Thursday so the group could assemble the slides over the weekend. She did not do it. She did not tell anyone until Friday night, when a groupmate messaged to ask where it was. The others then spent Saturday writing her section in a hurry, and the final presentation was weaker for it. Her first draft of an apology, typed into the group chat, reads: "Sorry guys, this week has been insane with everything going on, I had two other deadlines and I've been really stressed. I feel awful. Hope it's ok!" It contains regret and a long explanation. It contains no acknowledgement of what she actually did, which was not simply failing to write the section but failing to warn anyone. It offers nothing. It says nothing about the future. And its closing "Hope it's ok!" functions as a request for forgiveness that also asks the group to declare the matter closed. A rebuilt version might read: "I owe you all an apology. I said I'd have the competitor analysis done by Thursday, and I didn't, and worse, I didn't tell anyone until Sam asked on Friday night. That meant you three lost your Saturday redoing my part, and the presentation suffered for it. I'm really sorry. I had other deadlines, but I should have flagged it on Tuesday when I knew I was behind. For the next stage I'll take the write-up of the final report, which is the biggest remaining piece, and I'll post drafts in the shared folder by Wednesday each week so nobody has to chase me." The second version is not much longer than the first, but every element is different. The acknowledgement is specific, identifying both the missed work and the more damaging failure to warn. The impact on the group is named. The explanation is present but compressed to a single clause and placed after the admission, where it cannot be mistaken for a defence. The repair is concrete and proportionate: she takes on a larger share of the remaining work. The commitment about the future is checkable, since the others will see whether drafts appear on Wednesdays. And there is no request for forgiveness at all, which in a group-work context is probably right; the group's trust will return, or not, depending on what happens over the following weeks. Notice too what the rebuilt version does not do. It does not dwell on how awful Aisha feels. It does not ask anyone to reassure her. It does not over-promise, for example by offering to redo the whole presentation. It treats the other three as colleagues who were let down and who are owed a straight account and a practical remedy, which is exactly what they are. One final point concerns authenticity. Students who learn about the structure of apologies sometimes worry that using it will make them sound scripted. The worry is reasonable but mostly misplaced. The structure describes what sincere apologies already tend to contain. Knowing it does not make an apology less genuine; it makes it less likely that a genuine apology will fail because a key part was left out under stress. What makes an apology sound scripted is not structure but the absence of specific detail, the kind of detail only someone who has actually thought about what they did can supply. An apology that names the particular act, the particular effect, and the particular remedy cannot sound like a template, because no template contains those things. Chapter 3: Why Sorry Is Hard to Say If apologising well were only a matter of knowing the components, the previous chapter would be enough. It is not. Most people who fail to apologise, or who apologise badly, know perfectly well what they ought to say. The difficulty lies elsewhere: in the pull of self-justification, in the fear of what an admission will cost, and in a set of mental habits that make one's own offences look smaller than they are. Understanding these forces does not make them disappear, but it makes them recognisable, and a force you can recognise is one you can work around. Two stories about the same event One of the most illuminating studies in this area was published in 1990 by Roy Baumeister, Arlene Stillwell, and Sara Wotman in the Journal of Personality and Social Psychology. They asked people to write two autobiographical accounts: one of a time someone had angered them, and one of a time they had angered someone else. Because the same people wrote both kinds of story, differences between them could not be explained by personality. They reflected the position the writer occupied. The differences were systematic. When people wrote as perpetrators, they tended to describe the offence as a single, bounded incident that was over and done with, provoked by circumstances or by the other person, understandable given the context, and of limited lasting consequence. When they wrote as victims, they described offences as part of a longer pattern, as arbitrary or incomprehensible, and as having lasting effects. Each side's story contained some truth. But neither contained the whole of it, and each was shaped in a direction that suited the teller. This finding, sometimes called the magnitude gap, explains a great deal of campus conflict. The student who leaves dishes in the sink sees an isolated lapse during a busy week. The housemate who finds them sees the fifth time this month. The friend who made a joke at a party sees a moment of banter in a noisy room. The person the joke was about sees it as confirmation of something they have suspected for weeks. Both are reporting honestly. They are simply standing in different places. The practical consequence for apology is that the offender's sense of how big a deal something was is systematically unreliable. It is biased downward. This does not mean the victim's sense is always accurate, since victims too can inflate, but it means that when you are the one who caused harm, your first estimate of its seriousness should be treated with suspicion. If you find yourself thinking "this is a lot of fuss over nothing," that is precisely the moment to consider that the other person may be seeing a pattern you are not. The machinery of self-justification Why do perpetrators' accounts tilt so consistently in their own favour? One influential answer comes from the theory of cognitive dissonance, which the social psychologists Carol Tavris and Elliot Aronson apply to everyday wrongdoing in their book Mistakes Were Made (But Not by Me). Most people hold a view of themselves as reasonably decent, competent, and fair. When they do something that conflicts with that view, such as treating a friend unkindly, they experience discomfort. There are two ways to resolve it. One is to revise the self-image: "I am the kind of person who does unkind things." The other is to revise the account of the act: "It wasn't really unkind, given what she did first." The second is far less painful, and so the mind reaches for it automatically. Tavris and Aronson's point is that this process is not deliberate lying. It happens below awareness, and it is strengthened by each step. The first small justification makes the next one easier, until a person who began by feeling a flicker of guilt has constructed a complete and sincere narrative in which they were the wronged party. This is why arguments between housemates or friends can escalate so quickly and why, a week later, both parties are genuinely convinced they are owed the apology. The antidote is not to distrust yourself completely, which is paralysing, but to notice the characteristic signs of justification at work. A few are especially common. One is the sudden relevance of the other person's past faults: "Well, she never cleans the bathroom either." Another is the emergence of a technicality: "I never actually promised to be there." A third is an intense focus on intention: "I didn't mean it that way." Each may be true. But when they arrive quickly and with a feeling of relief, they are often doing the work of dissonance reduction rather than honest analysis. Three barriers The psychologist Karina Schumann, who has spent much of her career studying why people do and do not apologise, reviewed the evidence in a 2018 article in Current Directions in Psychological Science. She grouped the obstacles into three broad barriers, and they provide a useful map. The first is low concern for the victim or the relationship. People who care less about the person they have harmed, or who are less inclined to take others' perspectives, are less likely to apologise. This is the least surprising barrier, but it has a practical implication: when you find yourself reluctant to apologise, it is worth asking honestly whether the relationship matters to you. If it does, remembering that fact can be enough to get past the reluctance. If it does not, that is also worth knowing, though on a campus where you will keep seeing people, even relationships you do not value may be worth maintaining at a civil level. The second is perceived threat to the self. Apologising means admitting fault, and admitting fault can feel like conceding that you are incompetent, unkind, or untrustworthy. Schumann cites evidence that transgressors tend to overestimate how unpleasant apologising will be. They also tend to feel it as a loss of power. In a 2013 study with a memorable subtitle, "and we issue no mea culpa for this research finding," Tyler Okimoto, Michael Wenzel, and Kyli Hedrick found that people who were asked to refuse to apologise for a transgression reported greater feelings of power, control, and self-worth than those who apologised. Refusing, in other words, can feel good in the short term. That is part of why it is tempting, and it is worth naming, because the boost is bought at the expense of the relationship. The third is perceived ineffectiveness: the belief that apologising will not help, that the other person will not forgive anyway, so there is no point. Research by Marret Leunissen and colleagues on what they called the "apology mismatch" found that offenders tend to underestimate how much victims value apologies and how likely victims are to respond positively. People who have been wronged often want an apology more than the offender realises, and they are frequently more willing to move on after one than the offender expects. Fixed selves and the fear of being a bad person One of the more hopeful findings in this area concerns what people believe about personality itself. Carol Dweck's research on "mindsets," which is widely known in education, distinguishes between people who see traits as fixed and those who see them as capable of growth. In a 2014 paper in Personality and Social Psychology Bulletin, Schumann and Dweck applied this distinction to wrongdoing. They found that people who believed personality could change were more likely to accept responsibility for their transgressions and to offer fuller apologies, and that this was partly because they found the admission less threatening. If a mistake reveals a fixed flaw, admitting it is a verdict on who you are forever. If it reveals something you can change, admitting it is simply the first step in changing it. The researchers also found that encouraging people to adopt the growth view increased their willingness to take responsibility. Schumann has found a related effect with self-affirmation. In other work, people who first had a chance to reflect on values important to them, affirming their sense of themselves as fundamentally decent, went on to offer more comprehensive apologies. The logic is the same as with the Breines and Chen findings on self-compassion discussed in Chapter 1. When the self feels secure, it can afford to admit error. When it feels under siege, it defends. The personal implication is straightforward, if not easy. Before apologising for something that stings, it can help to remind yourself of two things: that you are more than this one action, and that the point of admitting it is to change it. Neither thought lets you off the hook. Both make it easier to get on it. The anatomy of the non-apology When the barriers win, people rarely refuse outright. They reach instead for one of several familiar forms that look like apologies but lack the essential element of responsibility. These forms are so common that most people can recognise them instantly when they are on the receiving end, and fail to recognise them when they are the ones speaking. The most common forms, what each signals to the listener, and how each can be repaired are set out in Table 2. Table 2. Common forms of non-apology and how to repair them. Form Typical wording What the listener hears Repair The conditional "I'm sorry if I upset you" Doubt that harm occurred Name the act: "I'm sorry I said that" The feelings apology "I'm sorry you feel that way" The problem is your reaction Own the cause, not their response The "but" "I'm sorry, but you started it" The apology is withdrawn Apologise fully; raise grievances separately The passive "Things got out of hand" Nobody is responsible Put yourself in the sentence The minimiser "It was just a joke" Your hurt is excessive Acknowledge the effect, not the intent The deflection "I'm a terrible friend" Now comfort me Keep attention on their experience The demand "I said sorry, what else?" Your response is on a clock Let them take their time Each of these forms fails for a slightly different reason, but they share a common structure: they allow the speaker to say the word "sorry" while withholding the substance. The conditional apology ("if I upset you") treats the harm as hypothetical. The feelings apology locates the problem in the other person's reaction. The "but" apology offers an admission and then retracts it. The passive and minimising forms remove the actor or shrink the act. The deflection turns the conversation toward the speaker's own distress. The demand converts the apology into a transaction. It is worth saying that not every sentence containing "if" or "but" is a non-apology. "I'm sorry I snapped; if there's anything else I did that bothered you, I'd like to know" uses "if" in a way that invites more honesty, not less. And there are situations in which an offender genuinely did not cause the harm attributed to them, and where a feelings-based response ("I'm sorry you've had such a hard time with this") is an honest expression of sympathy rather than an evasion. The test is not the presence of particular words but whether the speaker has clearly owned what they actually did. Attachment and defensiveness A final factor worth mentioning concerns the patterns people bring from earlier relationships. Research on adult attachment, which examines how people approach closeness and dependence, has linked attachment styles to apology quality. Schumann and Edward Orehek reported in 2019 that people higher in avoidant attachment, who tend to be uncomfortable with emotional closeness, offered lower-quality apologies, in part because apologising requires a vulnerability they find aversive. This is not a reason to diagnose yourself or anyone else. But it is a reminder that the difficulty some people have with apology is not simply stubbornness. For someone who learned early that admitting weakness invited criticism, or that emotional exposure was unsafe, saying "I was wrong" can feel genuinely dangerous. Recognising that in yourself can be the beginning of changing it. Recognising it in others can make their clumsy or partial apologies easier to receive generously. Barriers peculiar to campus The general barriers apply everywhere, but university life adds three of its own. The first is the audience. In a close community, an apology is rarely a purely private act. If you apologise to a housemate, the other housemates will probably hear about it. If you apologise in a friend group, the apology itself becomes part of the story that circulates. For students who are still working out their social position, this can make apology feel like a public loss of face, a concession that everyone will know about. The fear is understandable, but it usually gets the social arithmetic wrong. In most groups, people who own their mistakes cleanly are respected for it, and the person whose standing suffers over time is the one who never does. The student who says "I got that wrong, sorry" in front of the group is rarely the one people talk about afterwards. The one who digs in usually is. The second is mutual fault. Many campus conflicts are not cases of a clear offender and a clear victim. Two housemates have both been passive-aggressive about the cleaning. Two friends both said things they regret during an argument. In these situations each party is waiting for the other to go first, on the reasoning that apologising first would amount to accepting all the blame. The result is often a standoff that lasts far longer than the original conflict. The way out is to recognise that an apology for your own part does not concede anything about the other person's part. "I'm sorry for what I said about your family; that was out of line" is a complete apology for one act. It does not say "and everything else was my fault too." Going first, in these situations, is not surrender. It opens the door for the other person to do the same, and in practice they often walk through it. The third is the fear of formal consequences. Some students hesitate to apologise because they worry that an admission might be used against them, for example in a housing dispute, an academic integrity case, or a disciplinary process. In a minority of situations this concern is legitimate, and Chapter 7 discusses how to handle apologies when a formal process is under way. But in the great majority of everyday conflicts, no formal process is involved or likely, and the fear is simply the threat-to-self barrier wearing an official-looking disguise. A student who will not apologise to a roommate for breaking a lamp because "it might count as admitting liability" is usually protecting pride, not legal position. It is also worth noticing one barrier that is often more acute for students than for older adults: inexperience with repair itself. Many people reach university having rarely seen an adult apologise well. They may have grown up in families where conflicts were resolved by silence and a gradual return to normal, or where apologies were demanded as rituals of submission rather than offered as acts of repair. If the only apologies you have witnessed were forced out of children by parents ("say sorry to your sister"), it is natural to associate apology with humiliation. Part of what university offers, if you let it, is the chance to learn a different model, one in which apology is something competent adults do routinely and without drama. What to do with all this The lesson of this chapter is not that people are bad at apologising because they are bad people. It is that apologising cuts against several deep and ordinary features of human psychology: the tendency to see one's own offences as smaller than others see them, the urge to protect self-image, the short-term reward of refusing, and the pessimistic belief that apology will not help anyway. Each of these forces is strongest in exactly the moments when apology is most needed. Knowing this suggests a few practical habits. When you feel the rush of self-justification, slow down and ask what the other person's account of events would sound like. When you feel the threat to your self-image, remind yourself that the admission concerns an act, not your worth. When you feel that apologising is pointless, remember that the evidence suggests others value apologies more, and respond to them more generously, than offenders tend to expect. And when you hear yourself reaching for "if," "but," or "I'm a terrible person," take that as a signal to start the sentence again. Hashtags: #TheArtOfTheApologyOnCampus #CampusCommunication #RelationshipMaintenance #EffectiveApologies #ConflictRepair #AcknowledgementOfResponsibility #OfferOfRepair #ExpressionOfRegret #DeclarationOfRepentance #RequestForForgiveness #TrustRepair #RestoringStanding #SharedRules #InterpersonalConflict #RoommateConflict #FriendshipRepair #GroupProjectCommunication #ProfessorStudentCommunication #NonApologies #SelfJustification #CognitiveDissonance #GuiltAndShame #Forgiveness #CommunicationSkills #FutureOfCampusRelationships

Download the Book (PDF): Introduction At three in the morning a fourteen-year-old with type 1 diabetes is asleep in a bedroom down the hall from her parents. Her glucose, which ran high after a late slice of pizza, has been drifting down for an hour. A sensor under the skin of her arm reports a new value every five minutes. A small pump on her abdomen receives each value by radio, and software inside it, or on a phone beside the bed, runs a calculation: where will her glucose be in half an hour, or an hour, if nothing changes? The answer comes back lower than the system likes, so it trims the insulin it was about to deliver. Five minutes later it asks again. Nobody wakes. By breakfast her overnight trace is a nearly flat line in the target range, and her mother, who for three years set an alarm to check her daughter's glucose at two and four every night, has slept through. That scene, now ordinary in many countries, is what people mean when they speak of an artificial pancreas. The clinical term is automated insulin delivery, or a closed-loop system: a glucose sensor, an insulin pump and a control algorithm that connects them, so that insulin dosing responds to glucose without a person making every decision. Since the first commercial hybrid system reached the United States market in 2016, these devices have moved from research wards into routine care. In the 2026 Standards of Care, the American Diabetes Association describes automated insulin delivery as the preferred insulin delivery method for all people with type 1 diabetes who can use it [33]. In England, the National Institute for Health and Care Excellence recommended hybrid closed-loop systems in December 2023 for all children and young people with type 1 diabetes, for those who are pregnant or planning pregnancy, and for adults whose glucose remains above target or who have disabling hypoglycemia [34]. The trial evidence behind this shift is strong and consistent. Across children, adults, older adults and pregnant women, closed-loop systems add roughly two to three and a half hours per day in the target glucose range compared with the best non-automated care, while holding or reducing time spent low. The gains are largest overnight, and they are largest of all for people whose control was worst to begin with. And yet the phrase "artificial pancreas" oversells what these machines are. People using them still count or estimate carbohydrates, still announce meals, still change infusion sets every two or three days and sensors every week or two, still respond to alarms, still manage exercise, illness and alcohol, and still end up in hospital with ketoacidosis when a cannula kinks unnoticed. In large real-world datasets, users of current systems typically spend about three quarters of the day in range, not all of it [18]. A healthy pancreas, by comparison, keeps glucose between roughly 70 and 140 mg/dL (3.9 to 7.8 mmol/L) almost all of the time, through meals, sprints, fevers and sleep, without being told anything. The argument of this book Why the gap? The answer usually offered is that the algorithms are still immature, or that sensors are not yet accurate enough. Both are partly true, but neither is the main reason. The central argument of this book is that the artificial pancreas is, at bottom, a machine for managing delay, and that almost everything about it, from the design of its algorithms to what it still asks of the person wearing it, follows from that one fact. A pancreatic beta cell senses glucose in the blood that bathes it and releases insulin within a minute or two directly into the portal vein, which carries it straight to the liver, the organ that matters most for glucose. A closed-loop system senses glucose in the fluid between cells under the skin, several minutes behind the blood. It then delivers insulin into fatty tissue under the skin, from which it is absorbed slowly into the general circulation, reaching peak effect an hour or more later and continuing to act for several hours after that. The controller is always acting on old information, and whatever it decides will not take full effect until the situation has changed again. A thermostat in a house with a furnace that took an hour to warm up, and kept blowing heat for four hours after it was switched off, would face the same problem. Once that is understood, the rest of the field comes into focus. Every commercial algorithm is a different strategy for acting under delay: by predicting ahead, by damping its own enthusiasm with estimates of insulin already on board, by being cautious on the way down and aggressive on the way up. The mealtime announcement that users find so burdensome is a way of buying back the time the insulin needs. The residual hypoglycemia after exercise, the hyperglycemia after a large unannounced meal, and the danger of an infusion site failure all trace back to the same slow pharmacology. The ways forward, too, sort neatly by which delay they attack: faster insulins, additional hormones such as glucagon and amylin analogs, smarter meal detection, or delivery routes that reach the liver more directly. How the book is organized The first chapter describes what a healthy pancreas actually does and why subcutaneous insulin, however it is dosed, cannot fully copy it. The second traces the long history of closing the loop, from bedside machines of the 1960s and 1970s to the pocket systems of today, and shows that the obstacles were sensing and safety rather than computing power. The third examines the glucose sensor, the system's only window on the body, and the metrics by which its data are judged. The fourth explains how the main families of control algorithms work and how the commercial systems differ. The fifth takes up the problem of delayed insulin absorption directly and shows why meals are the hardest part of the day for any controller. The second half of the book turns from mechanism to consequence. The sixth chapter weighs the trial evidence and real-world data, including the populations in which it is strongest and the one important question on which it proved disappointing. The seventh is about the lived experience of handing part of one's physiology to a machine: trust, alarms, workload and who gets access. The eighth deals with what happens when the loop fails, and with situations such as illness, pregnancy, exercise and hospital admission that stretch it. The ninth looks at the attempts to build a fully closed loop that needs no meal announcements, and at the competing prospect of biological replacement. The conclusion draws out what the argument implies for how clinicians should offer these systems, how users should be taught, and what remains genuinely unsettled. The book is written for clinicians and students who want to understand how these systems work rather than merely which buttons to press, and for people living with type 1 diabetes and their families who want the same. It is educational and does not replace local protocols, device instructions or the judgment of a diabetes team who knows the individual. Glucose values are given in mg/dL, with mmol/L alongside where precision matters; to convert, divide mg/dL by 18. A final note on words. "Artificial pancreas" is the popular name and a useful one, provided it is taken as an aspiration rather than a description. "Hybrid closed loop" is the more honest term for today's commercial systems: closed for background insulin and corrections, open at mealtimes, where the user still supplies the information the machine cannot get quickly enough on its own. "Automated insulin delivery" is the umbrella term now used in guidelines. These pages use all three, and the difference between them is, in a sense, the subject of the whole book. Chapter 1: What the Pancreas Does, and Why Injected Insulin Cannot Copy It Any machine built to replace an organ is judged against the organ, so the place to begin is with what the endocrine pancreas does when it works. The comparison is humbling, and it is meant to be. The islets of Langerhans make up only a small fraction of the pancreas by weight, a scattering of perhaps a million cell clusters through an organ otherwise devoted to digestive enzymes. Yet they hold blood glucose in a narrow band across a range of challenges that would defeat any engineered system now on the market. Understanding precisely how they do it explains why an artificial pancreas, built from a skin sensor and a skin pump, faces limits that no amount of clever software can fully remove. The islet at work The beta cell as sensor, computer and pump A beta cell is a sensor, a controller and an actuator folded into one cell. Glucose enters it through transporter proteins in proportion to its concentration in the blood. Inside, the enzyme glucokinase acts as the glucose sensor proper: it phosphorylates glucose at a rate that rises steeply across the physiological range, so that the cell's metabolic activity tracks the blood glucose concentration closely. As glucose is metabolized, the ratio of ATP to ADP rises, potassium channels in the cell membrane close, the membrane depolarizes, calcium floods in, and granules of stored insulin fuse with the cell surface and empty into the capillary blood. From a rise in glucose to the release of insulin takes about a minute. That is the first advantage the natural system has over any artificial one. It measures glucose in arterial blood, the fluid whose concentration actually determines what the brain and muscles receive, and it measures it continuously, not every five minutes. There is no interstitial fluid to diffuse through, no enzyme electrode to drift, no radio link to drop. The second advantage is the shape of the response. When glucose rises suddenly, insulin secretion comes in two phases. The first phase is a sharp spike within the first few minutes, drawn from granules already docked at the membrane and ready to go. The second phase is a slower, sustained rise that continues as long as glucose stays elevated. The first phase matters more than its brief duration suggests: it is largely responsible for switching off the liver's production of glucose early in a meal, which prevents the postprandial rise from ever getting large. Loss of first-phase secretion is one of the earliest detectable defects in people progressing toward both type 1 and type 2 diabetes, and when it is lost, glucose after a meal climbs higher and stays up longer even if total insulin output is normal. The beta cell also anticipates. Before a single carbohydrate molecule reaches the bloodstream, the sight, smell and taste of food trigger a small cephalic-phase insulin release through the vagus nerve. As food reaches the small intestine, cells in the gut wall release the incretin hormones GLP-1 and GIP, which amplify the beta cell's response to glucose. By the time glucose from the meal begins to appear in the blood, insulin is already rising. The natural pancreas, in other words, doses for the meal slightly before the meal arrives, using information that no glucose sensor can see. Finally, insulin is not secreted as a smooth stream but in pulses every few minutes. The pulsatility appears to keep the liver's insulin receptors responsive, and it disappears early in diabetes. Nothing in current closed-loop practice tries to reproduce it; the point is simply that the natural signal is richer than a single number rising and falling. The liver comes first The most underappreciated feature of normal insulin secretion is where it goes. Insulin from the pancreas enters the portal vein, which drains the gut and pancreas and runs directly to the liver. The liver is therefore exposed to insulin concentrations considerably higher than the rest of the body, and on this first pass it extracts and degrades roughly half of the insulin it receives. The rest of the body, muscle and fat included, sees a lower, filtered concentration. This arrangement matters because the liver is the central organ of glucose control. Between meals it releases glucose, from stored glycogen and from new glucose made out of lactate, amino acids and glycerol, at a rate that closely matches the brain's steady consumption. After a meal, rising portal insulin swiftly suppresses that output and tells the liver to store incoming glucose as glycogen. A large part of the normal postprandial response is not muscle soaking up glucose but the liver simply stopping its own production. And because it happens under high portal insulin, the rest of the body does not need to be flooded with insulin to achieve it. Subcutaneous insulin reverses this geometry. Insulin injected or infused under the skin is absorbed into peripheral capillaries and lymphatics and enters the general circulation. The liver receives only what reaches it through the hepatic artery and portal vein after mixing with the whole blood volume. To suppress hepatic glucose output adequately, peripheral insulin concentrations must be pushed well above normal. People with type 1 diabetes on any subcutaneous regimen therefore live with a relative insulin deficiency at the liver and a relative excess in the periphery. That peripheral excess contributes to weight gain and to the tendency of muscle, especially during exercise, to draw glucose out of the blood faster than the liver can replace it. No algorithm, however well tuned, can change where the insulin goes. It can only change how much and when. The counter-regulatory half of the system The beta cell is only half of the islet's work. Alpha cells in the same islets secrete glucagon, which acts on the liver in the opposite direction: it stimulates glycogen breakdown and new glucose production. In a healthy person, glucagon and insulin are regulated reciprocally. As glucose falls, insulin secretion drops first, then glucagon rises, then, if the fall continues, adrenaline, cortisol and growth hormone join in. The first defense against hypoglycemia is simply that the pancreas stops making insulin, and the insulin already in the portal circulation is cleared within minutes. In type 1 diabetes, all three layers of this defense are compromised. The first is lost outright: insulin given under the skin cannot be withdrawn once delivered, and it continues to act for hours regardless of what glucose does. The second is lost within a few years of diagnosis in most people: the glucagon response to falling glucose becomes blunted or absent, even though alpha cells remain present and can still secrete glucagon in response to other stimuli, sometimes inappropriately after meals, which worsens post-meal highs. The third is often degraded by repeated hypoglycemia itself. Each low episode resets the glucose threshold at which adrenaline and warning symptoms appear, so that people with frequent lows lose their early warning, a condition called impaired awareness of hypoglycemia. These people are at the greatest risk of severe hypoglycemia, in which they need another person's help to recover. This is why the artificial pancreas problem is asymmetric. Too little insulin produces high glucose, which is harmful over months and years and dangerous over hours only when insulin is almost completely absent. Too much insulin produces low glucose, which can cause confusion, seizures or death within an hour, and which the body can no longer reliably correct. A controller for type 1 diabetes has an accelerator and no brake. When it overshoots on the way down, the only remedies are to stop delivery and wait for the insulin already in the tissue to wear off, or for the person to eat. A third hormone: amylin Beta cells co-secrete a second peptide with insulin, called amylin, in a fixed ratio. Amylin slows the emptying of the stomach, suppresses the inappropriate rise in glucagon after meals, and contributes to fullness. In type 1 diabetes amylin is lost along with insulin. The consequence is that food leaves the stomach faster than it would in someone without diabetes, glucose from a meal appears in the blood sooner, and glucagon rises when it should fall. The postprandial problem in type 1 diabetes is thus not only that insulin arrives late but also that glucose arrives early. An injectable amylin analog, pramlintide, has existed for two decades, and the idea of combining it with insulin in a closed loop, discussed in the final chapter, rests directly on this piece of physiology. What is lost, and what replaces it What goes wrong in type 1 diabetes Type 1 diabetes is an autoimmune disease in which T cells destroy the beta cells, usually over months to years. By the time of diagnosis most people have lost the majority of their functional beta-cell mass, and in most people the remaining secretion dwindles over the following years, though small amounts of residual insulin production, measurable as C-peptide, persist in a surprising number of people with long-standing disease. Residual C-peptide matters clinically: people who retain some have fewer severe hypoglycemic episodes and more stable glucose. That observation motivated one of the most interesting closed-loop trials, discussed in the sixth chapter, which asked whether tight glucose control from diagnosis could protect the surviving beta cells. It could not. Without insulin, cells cannot take up glucose efficiently and the liver produces it unchecked, so glucose rises. At the same time, fat is broken down, and the liver converts the resulting fatty acids into ketone bodies. Ketones accumulate, the blood becomes acidic, and the person develops diabetic ketoacidosis, a medical emergency. This happens quickly when insulin is absent entirely. In a person using an insulin pump, who has no long-acting insulin depot and relies wholly on a continuous trickle of rapid-acting insulin, a blocked or dislodged cannula can lead to rising ketones within a few hours. That vulnerability, inherent in pump therapy and therefore in every current closed-loop system, is the subject of the eighth chapter. The long-term cost of hyperglycemia was established decisively by the Diabetes Control and Complications Trial, published in 1993 [1]. Among 1,441 people with type 1 diabetes followed for an average of six and a half years, intensive therapy that lowered average glucose substantially reduced the development and progression of retinopathy, kidney disease and neuropathy. Follow-up of the same cohort later showed lasting reductions in cardiovascular events. But intensive therapy in that trial also roughly tripled the rate of severe hypoglycemia. The DCCT defined the dilemma that the artificial pancreas exists to resolve: lower glucose protects the eyes, kidneys, nerves and heart, but with the tools of the time, lowering glucose meant more frequent and more dangerous lows. For thirty years, clinicians and patients have negotiated a compromise between these two harms. The promise of closed-loop control is to shift that compromise, to lower average glucose without paying for it in hypoglycemia. Why subcutaneous insulin is a different drug It helps to be precise about why insulin under the skin behaves so differently from insulin secreted by the pancreas. Human insulin in solution naturally associates into hexamers, clusters of six molecules held together with zinc. A hexamer is too large to pass quickly through capillary walls, so after an injection it must first dissociate into dimers and then monomers before it can be absorbed. This is why regular human insulin, injected under the skin, takes thirty to sixty minutes to begin working and reaches its peak two to three hours later. The rapid-acting analogs introduced from the mid-1990s, lispro, aspart and glulisine, were engineered with small changes in amino acid sequence that weaken self-association, so they dissociate faster and are absorbed sooner. According to the American Diabetes Association's summary, they begin to act in about fifteen minutes, peak at about one to two hours, and continue to act for about two to four hours. In practice, pharmacodynamic studies show a long tail: a meaningful fraction of the glucose-lowering effect of a bolus is still present four or five hours after it was given, which is why pump settings for "active insulin time" are usually set longer than the label's nominal duration. Newer formulations, faster aspart (with added niacinamide) and ultra-rapid lispro (with treprostinil and citrate to increase local blood flow and permeability), shave several minutes off the early absorption, but they do not change the basic shape. Insulin under the skin still takes the better part of an hour to do most of its work and hours more to finish. Absorption is also variable. The same dose given into the same person on different days can be absorbed at noticeably different rates. Site matters: the abdomen tends to absorb faster than the thigh. Local blood flow matters: heat, massage and exercise of the underlying muscle speed absorption, and cold slows it. Tissue quality matters: repeated use of the same area produces lipohypertrophy, rubbery thickened fat in which insulin absorption is erratic and slower. Even the age of the infusion site matters; absorption from a pump cannula changes over the two to three days of its wear. A controller can learn a person's average insulin action, but it cannot see this day-to-day scatter until it shows up, too late, in the glucose trace. A moving target One further complication deserves mention because every controller must cope with it: the amount of insulin a person needs is not constant. Insulin sensitivity changes across the day. Many people with type 1 diabetes need more insulin in the early morning, when growth hormone and cortisol released during the night raise glucose, the so-called dawn phenomenon, and less in the afternoon. Sensitivity rises for many hours after exercise, as muscles replenish their glycogen, and falls during infection, with corticosteroid treatment and under psychological stress. In women it can shift across the menstrual cycle, with many needing more insulin in the days before menstruation. During puberty, growth hormone and sex steroids can raise insulin needs by a third or more, and needs change again through pregnancy, rising steeply in the second and third trimesters and falling abruptly after delivery. A healthy beta cell does not need to know any of this. It responds to glucose, and whatever the body's current sensitivity, it secretes as much insulin as glucose demands. A dosing system that works from fixed settings, a basal rate, a carbohydrate ratio, a correction factor, will be wrong whenever sensitivity changes, and it will stay wrong until someone notices and adjusts the settings. Much of the tedium of manual pump therapy was exactly this: noticing patterns over days, guessing at their causes and changing settings in small steps. A closed loop that responds to glucose rather than to fixed settings handles slow changes in sensitivity naturally, which is one of its main advantages. But because it acts through a delay, it handles abrupt changes, such as the sudden sensitivity of the hours after a long hike, less well. What the machine is up against Set side by side, the differences look like this. The natural system senses glucose in blood, instantly, and continuously; the artificial one senses it in interstitial fluid, several minutes behind, and every five minutes. The natural system anticipates meals through nervous and gut signals; the artificial one learns about meals only when glucose starts to rise, unless the person tells it. The natural system delivers insulin to the liver first, in pulses, within a minute or two of deciding to; the artificial one delivers it to fat under the skin, from which it reaches the general circulation over an hour and lingers for several more. The natural system can switch insulin off and see it cleared within minutes; the artificial one can only stop adding more. The natural system has glucagon and amylin to shape the response in both directions; nearly all artificial systems have only insulin. Engineers describe a system like this as having large "transport delays" and "actuator saturation": the controller's actions take effect late, and it cannot push in both directions. They know that such systems cannot be controlled tightly by feedback alone. Any controller that reacts strongly to current error will overshoot, because by the time its action takes effect, the situation it was reacting to has changed. Any controller that reacts gently will be too slow to stop large disturbances, such as a meal. The only ways to do better are to predict disturbances before they happen, to act before they are seen, or to shorten the delays themselves. That is the frame for the rest of this book. The sensor chapter examines the delay and error on the input side. The algorithm chapter examines how controllers predict and hedge. The chapter on insulin absorption examines the delay on the output side, and why meals remain the weak point. What the trials show, how people experience these systems and where the field is headed can then be read against the same frame. The artificial pancreas is not failing to be a pancreas because its engineers lack ambition. It is doing something genuinely hard: controlling a fast, dangerous, asymmetric variable through a slow, one-directional actuator with a delayed sensor. That it succeeds as well as it does is the real achievement, and the remaining gap is a map of exactly where the next advances must come from. Chapter 2: Closing the Loop: From the Bedside Machine to the Pocket The idea of an artificial pancreas is almost as old as the ability to measure glucose automatically. The engineering principle, measure a variable, compare it with a target, adjust an input, is the same one that runs a thermostat or a ship's autopilot, and it was well understood by the 1950s. Yet more than half a century separated the first working closed-loop glucose controller from the first one a person could wear home. The history is worth telling because it shows that the bottlenecks were never really about computing. They were about getting a trustworthy glucose signal from a living body, delivering insulin by a route that did not require a needle in a vein, and proving that an automated system could be allowed to dose a drug capable of killing the person wearing it. Early machines Intravenous beginnings In 1964 Arnold Kadish, a Los Angeles physician, described a "servomechanism for glucose monitoring and control" [2]. Blood was drawn continuously from a vein and analyzed by an automated chemistry instrument; when glucose went above a set point, the device infused insulin intravenously, and when it went below, it infused glucagon. It worked in principle, but the apparatus was the size of a large backpack and its control logic was simply on or off. It was a proof that the loop could be closed, not a treatment. A decade later, a team in Toronto led by Michael Albisser and Bernard Leibel reported what they called an artificial endocrine pancreas [3]. It too sampled venous blood continuously and infused insulin or dextrose intravenously, but its dosing was governed by mathematical algorithms, run on a small computer, designed to imitate the beta cell's response to both the level of glucose and its rate of change. It was tested first in pancreatectomized dogs and then in people. The Toronto group's conceptual contribution endured: they framed the task not as switching insulin on and off but as reproducing a physiological dose-response relationship that included anticipation of where glucose was heading. In the late 1970s the Biostator, a commercial bedside instrument made by Miles Laboratories, brought this design into hospitals and research units. The Biostator withdrew blood continuously through an intravenous line, measured glucose every minute with a glucose oxidase sensor, and infused insulin or dextrose intravenously according to algorithms derived from the Toronto work. It could hold glucose in the normal range for hours or days, and it became a workhorse of metabolic research, used for decades in glucose clamp studies of insulin action. But it required a double intravenous line, a trained operator, frequent recalibration, and a machine the size of a small refrigerator. It showed that a computer could control glucose if it had fast, accurate blood measurements and a fast intravenous route for insulin. The problem was that neither of those could be carried into daily life. The long detour through open-loop therapy For the next three decades, practical diabetes care moved in a different direction. Portable insulin pumps appeared in the late 1970s and became progressively smaller and more reliable through the 1980s and 1990s. Blood glucose meters for home use made fingerstick testing routine. The DCCT, completed in 1993, proved that intensive therapy, whether by multiple daily injections or pump, prevented complications, and it made tight control the standard of care [1]. But these were open-loop tools. The patient was the controller. Every dose was a human decision informed by a few fingerstick readings a day, carbohydrate estimates, and rules of thumb for correction. The problem with the human as controller is not intelligence; it is bandwidth. A person who tests four times a day samples glucose at intervals of several hours, in a system whose important dynamics unfold over tens of minutes. Overnight, most people are not sampling at all. Nocturnal hypoglycemia was, and remains, one of the most feared aspects of type 1 diabetes, partly because it can occur undetected during sleep. The "dead in bed" syndrome, sudden unexplained death of young people with type 1 diabetes found in an undisturbed bed, is rare, but it haunts parents and has been linked in some cases to nocturnal hypoglycemia. Any real advance had to address the night. Building the modern loop The sensor arrives The decisive technical change was the continuous glucose monitor. The first device approved in the United States, in 1999, was a "professional" monitor worn for a few days whose data were downloaded and reviewed afterward; the wearer could not see readings in real time. Real-time monitors, displaying a new reading every few minutes and able to alarm at high and low thresholds, reached the market in the mid-2000s. Early sensors were bulky, lasted three days, needed several fingerstick calibrations a day, and were inaccurate enough that many people abandoned them. But for the first time a stream of glucose data was available from a person going about ordinary life, and the stream could be fed into an algorithm. Around the same time, the research foundation now called Breakthrough T1D (then the Juvenile Diabetes Research Foundation) launched an artificial pancreas project that funded academic groups in the United States, the United Kingdom and elsewhere and pushed regulators to develop a pathway for approval. Three developments followed that made the modern field possible. The first was a simulator. Testing a new control algorithm had traditionally required animal studies before any human work. In 2007 Claudio Cobelli's group in Padova and Boris Kovatchev's group at the University of Virginia built a detailed mathematical model of glucose-insulin dynamics and populated it with 300 virtual subjects representing the variability of real people with type 1 diabetes. In January 2008 the United States Food and Drug Administration accepted this simulator as a substitute for animal trials in the preclinical testing of insulin-dosing algorithms [4]. That decision was quietly revolutionary. Algorithms could now be designed, tuned and stress-tested against thousands of simulated days, including extreme scenarios no ethics committee would allow, before any person wore them. Much of the subsequent progress in algorithm design was made possible by this ability to fail cheaply in silico. The second was a series of carefully controlled inpatient studies. In Cambridge, Roman Hovorka's group had developed a model predictive controller in the early 2000s [10]. In a randomized crossover trial in children and adolescents published in 2010, they showed that overnight closed-loop control, with glucose readings from a subcutaneous sensor and insulin by a subcutaneous pump, increased time in target and reduced time spent low compared with the children's usual pump therapy [5]. The loop in that study was closed manually, by a nurse transcribing each sensor reading into a laptop and each dosing recommendation into the pump. But the principle, fully subcutaneous closed loop, had been proven in the setting where it mattered most. The third was the move from the clinical research unit to the real world. Groups at Virginia, Cambridge, Boston, Montpellier, Padova and elsewhere built portable platforms, often smartphones running control algorithms and communicating wirelessly with sensor and pump, and tested them first in hotel rooms and diabetes camps, then at home. The Boston group led by Edward Damiano and Steven Russell, working with Firas El-Khatib, reported in 2014 an outpatient bihormonal "bionic pancreas" delivering both insulin and glucagon, tested in adults and adolescents over five days of normal life [6]. Across these studies, a consistent pattern emerged: closed-loop control was safe, it improved time in range, and its benefit was greatest overnight. From simulation to home: a staged path The research of this period established a staged path that every new algorithm still follows, and it is worth describing because it explains both the caution of the field and its speed. An algorithm is first developed and tested in simulation, where it can be run through thousands of virtual days, including missed meals, sensor failures, sudden exercise and extreme insulin sensitivity. It then moves to closely supervised inpatient studies in a clinical research facility, with frequent blood sampling and staff ready to intervene. Next come supervised outpatient studies, in hotels, camps or research houses, where participants eat real meals and walk around town but remain close to the study team. Only then does the system go home, first for days or weeks with remote monitoring and then for months in a randomized trial. Each stage answers a different question. Simulation shows whether the logic is sound and where it breaks. Inpatient studies show whether the system behaves in real bodies as it did in virtual ones. Supervised outpatient studies show whether it copes with ordinary life. Home studies show whether people will use it and whether it stays safe without a researcher at hand. Simulation has limits that the later stages expose. Virtual patients are only as varied as the data used to build them, and early simulators represented exercise, illness, stress and the changing absorption of an aging infusion site poorly or not at all. An algorithm that looked excellent in silico could perform disappointingly when a teenager played football after lunch. Part of the history of the field since 2008 has been the gradual enrichment of the simulators with these real-world effects, informed by the very trials they were used to design. This staged approach also shaped the regulatory conversation. Regulators could see a chain of evidence from mathematical model to home use, with each link tested separately, and it became possible to argue that an automated dosing system could be at least as safe as the human decisions it replaced. That argument, more than any single trial, is what eventually allowed closed-loop systems to be approved for home use. Safety first: suspending insulin Before regulators would allow a device to increase insulin automatically, they approved devices that could only decrease it. In 2013 the FDA approved a Medtronic pump with a "threshold suspend" feature, which stopped insulin delivery when the sensor glucose reached a preset low and the user did not respond to the alarm. The next generation, "predictive low glucose suspend," stopped insulin when glucose was forecast to go low within the next half hour and resumed it automatically when glucose recovered. These systems had modest effects on average glucose but meaningfully reduced the time spent low, especially overnight. These early systems are easy to overlook now, but they were an essential step. They demonstrated that a device could act on sensor data without a human in the loop, and they generated a regulatory precedent. They also illustrated the asymmetry described in the previous chapter. Suspending insulin is the one action a controller can take to prevent a low, and its effect is delayed by the insulin already on board. Predictive suspension works better than threshold suspension precisely because it acts earlier. Commercial systems and the community The first hybrid closed loop In September 2016 the FDA approved the Medtronic MiniMed 670G, the first commercial system that both increased and decreased insulin automatically. It was a hybrid closed loop: in its "Auto Mode" it adjusted basal insulin every five minutes according to sensor glucose, aiming at a fixed target of 120 mg/dL, while the user still had to bolus for meals by entering carbohydrate amounts. The pivotal trial enrolled 124 adolescents and adults, who used the system at home for three months [7, 8]. Glycated hemoglobin (HbA1c) fell from 7.7% to 7.1% in adolescents and from 7.3% to 6.8% in adults, and time in range rose, with no severe hypoglycemia or ketoacidosis. The trial had no control group, and its population was already well controlled, but it was enough for approval. The 670G proved that closed-loop control could be commercialized. It also exposed how much the human experience of a device matters. Its sensor required at least two fingerstick calibrations a day. Its algorithm was conservative and, when it lost confidence in the sensor or had delivered at its maximum or minimum rate for too long, it exited Auto Mode and required the user to intervene, often at night. In a prospective study of patients starting the system in routine care at Stanford, a third had stopped using Auto Mode within a year [28]. The most common reason was problems with the sensor. The lesson, which later systems took to heart, was that a closed loop is only as usable as its least usable component. We are not waiting While commercial systems moved through trials and regulatory review, a community of people with type 1 diabetes and their families decided not to wait. Beginning around 2013, under the banner #WeAreNotWaiting, programmers with diabetes reverse-engineered the communication protocols of their own pumps and sensors so that they could view glucose data remotely and, eventually, send commands to their pumps. Dana Lewis, a Seattle woman with type 1 diabetes, and Scott Leibrand built a system they later released as OpenAPS, the Open Artificial Pancreas System, running on a small computer that talked to an older model of pump [9]. Other projects followed: Loop, running on an iPhone; AndroidAPS, running on Android phones. The code was published openly. Anyone who could obtain compatible hardware and follow detailed documentation could build and run a closed loop, and thousands did. These open-source systems were not approved by any regulator, and many clinicians were initially uneasy about supporting them. But they were built by people with a direct stake in their safety, they were transparent in a way commercial algorithms were not, and they were highly configurable. They introduced features, such as small automatic correction boluses and algorithms that detected unannounced meals from the shape of the glucose rise, that commercial systems later adopted in their own forms. In 2022 a randomized trial in New Zealand, discussed in the sixth chapter, showed that an open-source system was as effective as the commercial ones [21]. In January 2023 the FDA cleared a version of Loop developed by the nonprofit Tidepool, bringing the community's work into the regulated world. The 2026 ADA Standards now explicitly support people choosing open-source systems and drop the older "do-it-yourself" label [33]. Interoperability and the modern generation A final regulatory development shaped the present landscape. Rather than approving each closed-loop system as a single fixed package of sensor, pump and algorithm, the FDA created categories for components that could be combined: in 2018 the integrated continuous glucose monitor (iCGM), a sensor meeting accuracy standards strict enough to feed an automated system; in 2019 the alternate controller enabled (ACE) infusion pump, a pump designed to take commands from an external algorithm; and, also in 2019, the interoperable automated glycemic controller, the algorithm itself. The first algorithm authorized in that category was Tandem's Control-IQ, derived from the University of Virginia's research controller, approved in December 2019 on the strength of a randomized trial published the same year [17]. The years since have brought a steady stream of systems. Insulet's Omnipod 5, the first tubeless patch-pump closed loop, was authorized in the United States in early 2022. Medtronic's MiniMed 780G, an advanced hybrid system with automatic correction boluses and a lower target than the 670G, reached the United States in 2023 after several years in Europe. Beta Bionics' iLet, the commercial descendant of the Boston bionic pancreas, cleared in May 2023, needs only the user's body weight to start and asks only that meals be announced as smaller, typical or larger than usual. CamAPS FX, the Cambridge algorithm running as a smartphone app, was already in use across Europe and the United Kingdom, including in pregnancy, and received FDA clearance in 2024. Indications have widened to include children as young as two, and since 2024 the leading systems have been authorized for adults with type 2 diabetes as well. The pattern across this history is consistent. The mathematics of control was ready decades before the hardware. What held the field back was a glucose signal good enough to act on, a delivery route convenient enough to live with, and a regulatory framework able to trust software with a potentially lethal drug. Each of those was solved, imperfectly, in the fifteen years after 2005. What was not solved, because it could not be solved by engineering alone, is the slowness of subcutaneous insulin. That is the physiological ceiling under which every system described in the following chapters still operates. Chapter 3: What the Machine Sees: Continuous Glucose Sensing A closed-loop controller knows nothing about the body except what its glucose sensor tells it. It cannot see the meal on the plate, the stairs being climbed, the fever developing, or the dose absorbed faster than usual from a warm abdomen. It sees a number every five minutes, together with whatever the user chooses to tell it. The quality of that number, how accurate it is, how noisy, how late, and how it fails, therefore sets the upper limit on what any algorithm can achieve. Seen from the controller's side, the sensor is an imperfect, delayed and occasionally misleading witness, and it is worth understanding exactly how. The sensor and its errors How a sensor measures glucose Nearly all current continuous glucose monitors work on the same electrochemical principle. A thin, flexible filament a few millimeters long is inserted under the skin, usually of the upper arm or abdomen, where it sits in the interstitial fluid that bathes the cells. The filament is coated with the enzyme glucose oxidase, held in place by membranes that control how much glucose and oxygen reach it. Glucose oxidase converts glucose and oxygen into gluconolactone and hydrogen peroxide; at the electrode, the hydrogen peroxide, or in some designs an intermediary molecule, is oxidized, generating a tiny electrical current proportional to the local glucose concentration. A transmitter on the skin measures the current, converts it to a glucose estimate using an algorithm, and sends the result by Bluetooth to a phone, a receiver or a pump. One important exception exists. A fully implantable sensor, placed under the skin of the upper arm in a minor office procedure, uses a fluorescent chemical that binds glucose and changes its light emission; an adhesive transmitter worn over the implant reads it. Its newest version lasts a year. It is compatible with some closed-loop systems, but the great majority of users wear the enzymatic type, typically lasting ten to fifteen days. Each design must deal with the same problems. The enzyme and membranes change over time. The body reacts to the filament as a foreign object, surrounding it over days with inflammatory cells that consume glucose and oxygen locally and alter the signal. The electrical signal is noisy. And the thing measured is not blood glucose but interstitial glucose, a related but distinct quantity. The lag between blood and tissue Glucose reaches the interstitial fluid by diffusing out of capillaries. When blood glucose is stable, interstitial glucose is close to it. When blood glucose is changing, interstitial glucose follows behind. Direct measurements using glucose tracers, injected into a vein and then sampled in the subcutaneous tissue of the abdomen, have quantified the physiological delay. In healthy fasting adults it was about five to six minutes [13]; in fasting adults with type 1 diabetes it was a median of about seven minutes, with all participants' tracers detectable in the tissue within ten minutes [14]. To the physiological lag, the device adds its own. Sensors filter their raw signal to remove noise, and filtering introduces a small additional delay. Most sensors used in closed loops report a value every five minutes, so on average the latest reading is a couple of minutes old even before the controller acts on it. Manufacturers have reduced the effective delay over time, in part by using algorithms that project slightly forward. But the controller should be thought of as seeing, at any moment, where glucose was about five to fifteen minutes ago. Is that a large delay? In steady conditions, no. Overnight, when glucose changes slowly, a ten-minute delay introduces an error of a few mg/dL. But after a meal, glucose can rise at 2 to 3 mg/dL per minute, and during vigorous exercise it can fall at similar speeds. At those rates, a ten-minute lag means the sensor reads 20 to 30 mg/dL behind the blood. That matters in two ways. A rising glucose is recognized late, so the controller starts adding insulin late. A falling glucose is recognized late, so the controller suspends insulin late, and the insulin it has already delivered will continue to act for hours. The researchers who measured the lag concluded, reasonably, that it should not be an obstacle to closed-loop control in fasting or overnight conditions. It is the fast transitions, meals and exercise, where it bites. Accuracy and its measurement Sensor accuracy is usually summarized by the mean absolute relative difference, or MARD: the average percentage difference between the sensor reading and a simultaneous laboratory reference measurement. Early sensors in the mid-2000s had MARDs in the mid to high teens; current sensors used with closed-loop systems report figures in the high single digits to low double digits under clinical study conditions. A lower MARD is better, but it is an average that conceals what the controller most cares about: how the sensor performs in the low range, how it behaves on the first day of wear, and how often it produces large errors. For this reason, the FDA's iCGM standard, introduced in 2018, set requirements not only for average accuracy but for accuracy within specific glucose ranges, for the proportion of readings far from the reference, and for the rarity of readings that would suggest the opposite of the truth, such as a normal reading when glucose is actually low. A sensor qualified as iCGM is certified to be accurate enough to drive automated dosing, and modern closed-loop systems do not require fingerstick calibrations, although most allow them. Several predictable sources of error persist, and users and clinicians need to know them because the controller does not. The first day. Sensor accuracy is usually worst in the first twelve to twenty-four hours after insertion, while the tissue settles around the filament. Some users insert a new sensor a day before the old one expires, so the new one has bedded in before the loop depends on it. Compression. When a person lies on the sensor, pressure reduces blood flow to the tissue around the filament; local glucose falls as cells continue to consume it, and the sensor reports a false low, often a steep drop followed by a sudden recovery when the person rolls over. For a closed loop this is not harmless. The controller suspends or reduces insulin in response to the false low, and the person may wake a few hours later with glucose that is genuinely high. Interfering substances. Certain drugs and supplements interfere with the electrochemistry of particular sensors. Reported examples include hydroxyurea and, for some sensor designs, paracetamol (acetaminophen) at higher doses, which can falsely raise readings, and large doses of vitamin C, which falsely raise readings on others. The specific list differs by device and is in each sensor's labeling. A falsely high reading is more dangerous for a closed loop than for a person reading a monitor, because the loop may respond by delivering more insulin. Physiology at the extremes. At very low glucose, and during rapid change, the relationship between interstitial and blood glucose is least stable. Sensors tend to be less accurate exactly where accuracy matters most. Dehydration, poor perfusion and critical illness. When peripheral circulation is compromised, as in shock, severe dehydration or during vasopressor treatment, interstitial glucose may not reflect blood glucose at all. This is one reason why closed-loop systems are not currently recommended in critical illness. A worked example: the lag during a run To see what the lag means in practice, consider a man who sets out for a forty-minute run in the late afternoon with glucose at 160 mg/dL and a moderate amount of insulin on board from a snack bolus an hour earlier. Twenty minutes in, his blood glucose is falling at about 2 mg/dL per minute and has reached 120. His sensor, reading ten minutes behind, still shows about 140, and it shows the fall as gentler than it is, because the sensor's own filtering smooths the change. His closed loop, seeing 140 and a modest downward trend, reduces insulin somewhat but does not suspend. At thirty minutes his blood glucose is about 100 and the sensor shows 120. By now the controller's forecast predicts a low and it suspends insulin, but the insulin delivered over the previous two hours is still being absorbed, and exercising muscle is taking up glucose at a high rate. At the end of the run his blood glucose is near 80, still falling, and the sensor reads around 100. The low that the controller is trying to prevent arrives ten or twenty minutes after the run ends, and it arrives because of decisions made before he laced up his shoes. None of the numbers in this illustration is extreme, and no component has failed. The sensor is accurate by its specifications and the controller has responded exactly as designed. The low is produced by the combination of sensor lag, the controller's reasonable reluctance to act on a single downward reading, and the long tail of insulin already delivered. The remedy, discussed in the fifth chapter, lies not in the sensor but in acting earlier, by setting an exercise target an hour or two before the run so that the insulin on board has already fallen by the time the fall begins. What fingersticks are still for Most current systems neither require nor encourage routine fingerstick checks, and for most of the day the sensor is the better guide, because it shows trend as well as level. But fingersticks retain three roles that every closed-loop user should know. The first is to check a reading that does not fit symptoms or circumstances, for example a normal sensor reading when the person feels clearly hypoglycemic, or a sudden unexplained high. The second is to confirm glucose before treating a low or giving a large manual correction when the sensor is in its first day or has been behaving erratically. The third is as a backup when the sensor fails or is out of supply. A meter with unexpired strips remains part of the essential kit. Noise, signal loss and what the controller does about them Besides systematic errors, sensors produce random noise: small fluctuations from one reading to the next that do not reflect real changes. Controllers must distinguish noise from signal. A controller that reacted fully to every five-minute fluctuation would oscillate, adding and withdrawing insulin in response to nothing. So every algorithm filters its input, whether explicitly, by smoothing recent readings, or implicitly, by using a model that makes implausible jumps unlikely. Filtering adds delay, and the designer must balance responsiveness against stability. This is an early example of a theme that recurs throughout the book: in a delayed system, every protection against error costs some speed. Sensors also sometimes stop reporting: the Bluetooth connection drops, the phone is left in another room, or the sensor signals that its readings are unreliable. Controllers handle missing data conservatively. A short gap may be bridged by predicting from recent trend; a longer one typically returns the pump to a preprogrammed basal rate or a "safe" basal amount, and eventually to manual mode. How often this happens, and how gracefully a system handles it, is a large part of what users experience as reliability. Time in range: how the loop's output is measured Continuous sensing changed not only how insulin is dosed but how glucose control is judged. For decades, HbA1c, which reflects average glucose over roughly the preceding two to three months, was the single measure of control. It predicts complications well, but it says nothing about variability or hypoglycemia. Two people can have identical HbA1c values, one with steady readings near 150 mg/dL and the other swinging between 50 and 300. Continuous monitoring allows a richer description. In 2019 an international consensus panel convened at the Advanced Technologies and Treatments for Diabetes congress agreed on standard metrics and targets [15]. The headline measure is time in range: the percentage of sensor readings between 70 and 180 mg/dL (3.9 to 10.0 mmol/L). Alongside it are time below range, divided into levels below 70 mg/dL and below 54 mg/dL (3.0 mmol/L), and time above range, divided into levels above 180 and above 250 mg/dL (13.9 mmol/L). Different targets apply to older or high-risk people, for whom avoiding hypoglycemia takes priority, and to pregnancy, where a tighter range of 63 to 140 mg/dL (3.5 to 7.8 mmol/L) is used, as Table 1 sets out. Table 1. Consensus targets for continuous glucose monitoring metrics. Population Time in range Time below range Time above range Most adults with type 1 or type 2 diabetes 70–180 mg/dL: more than 70% Below 70: less than 4%; below 54: less than 1% Above 180: less than 25%; above 250: less than 5% Older or high-risk adults 70–180 mg/dL: more than 50% Below 70: less than 1% Above 250: less than 10% Pregnancy with type 1 diabetes 63–140 mg/dL: more than 70% Below 63: less than 4%; below 54: less than 1% Above 140: less than 25% Source: Battelino et al., International Consensus on Time in Range, Diabetes Care 2019 [15]. Time in range relates to HbA1c, but loosely. In an analysis of 545 adults with type 1 diabetes from four trials, a time in range of 70% corresponded on average to an HbA1c of about 7%, and 50% to about 8%; each 10-percentage-point increase in time in range corresponded to an average decrease in HbA1c of about 0.6 percentage points, though with wide scatter between individuals [16]. A 10-point increase in time in range is 2.4 hours a day. This conversion is useful for reading closed-loop trials, most of which report time in range as their primary outcome. When a trial reports a gain of 11 percentage points, it means about two and a half more hours each day spent in the target range. The consensus also recommended that at least 70% of possible readings be captured over fourteen days for the metrics to be reliable, and that glucose variability, expressed as the coefficient of variation, be kept at or below 36%. Standardized reports, the ambulatory glucose profile, now summarize these metrics and overlay days to show the typical daily pattern. For closed-loop users these reports are the primary way clinicians review therapy, and they show the time spent in automated mode as well, a figure that says a good deal about how well a person is getting on with a system. What the sensor cannot tell the controller The most important limitation of the sensor is not an error but a blind spot. The sensor reports glucose, and only glucose. It tells the controller nothing about what is about to happen to glucose. A meal eaten now will show up in the sensor fifteen or twenty minutes later, as glucose from the gut appears in the blood and then diffuses into the tissue. A run started now will begin to lower glucose within minutes, but the sensor will show it only when the fall has begun, and the insulin delivered in the hour before the run is already committed. A healthy beta cell uses additional channels to see these events coming: nerve signals from the head and gut, incretin hormones, adrenaline. The closed loop has only what the user tells it, whether a meal announcement, a carbohydrate estimate, or an exercise mode. Experiments have tested additional inputs, such as heart rate and accelerometry from wearable devices, to recognize exercise automatically. They help somewhat, but none is yet part of a mainstream commercial system's core algorithm. The sensor therefore imposes a double constraint: its own lag, measured in minutes, and its blindness to the future, which, combined with the slowness of subcutaneous insulin, is measured in hours. How the algorithms try to see around that blindness is the subject of the next chapter. Hashtags: #TheArtificialPancreas #Type1Diabetes #AutomatedInsulinDelivery #ClosedLoopSystems #HybridClosedLoop #ContinuousGlucoseMonitoring #InsulinPumpTherapy #ControlAlgorithms #GlucosePrediction #InsulinOnBoard #ModelPredictiveControl #InterstitialGlucose #SensorLag #SubcutaneousInsulin #InsulinAbsorption #TimeInRange #HypoglycemiaPrevention #PredictiveLowGlucoseSuspend #MealAnnouncement #ExerciseManagement #BihormonalSystems #Glucagon #Amylin #OpenSourceArtificialPancreas #FutureOfAutomatedDiabetesCare

Download the Book (PDF): Introduction Exponential technologies are fundamentally altering global labour markets and macroeconomic structures. Brynjolfsson and McAfee's definitive text charts this disruption, but its massive scope — covering everything from the digitization of goods to the decoupling of productivity and employment — can easily overwhelm students attempting to synthesise a cohesive capstone thesis on digital transformation. This companion acts as your macroeconomic filter. It is explicitly written to explain The Second Machine Age, isolating the authors' complex economic models and translating them into clear, citable academic prose. We rigorously define the mechanics of "bounty" (the wealth generated by digital innovation) and "spread" (the resulting inequality). This guide gives you the exact terminology needed to critically analyse the future of corporate strategy and global labour. The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies was published by W. W. Norton in January 2014. Its authors are colleagues at the Massachusetts Institute of Technology: Erik Brynjolfsson, then director of the MIT Initiative on the Digital Economy and now a professor at Stanford and director of the Stanford Digital Economy Lab, and Andrew McAfee, a principal research scientist at MIT Sloan. Two years earlier they had self-published a short, urgent book called Race Against the Machine, which argued that the weak employment recovery after the 2008 financial crisis was not only cyclical but structural — that technology was destroying certain kinds of work faster than it was creating replacements. The Second Machine Age is the fuller, more ambitious treatment of that claim. Three years after it, in Machine, Platform, Crowd (2017), the same authors turned to the managerial implications of the same forces. The middle book is the one that entered the economics curriculum, the policy debate, and the vocabulary of practitioners. Its argument can be stated compactly. The Industrial Revolution was a rupture in human history because the steam engine overcame the limits of muscle power — human and animal — and in doing so bent the curve of population, income and welfare sharply upward after millennia of near-flatness. Digital technology, Brynjolfsson and McAfee argue, is a rupture of comparable magnitude, because it does for mental power what steam did for physical power. That is the first machine age and the second. The second is driven by three interacting forces. Computing capacity improves exponentially rather than incrementally, and because exponential growth compounds, its most dramatic effects arrive late — the authors borrow the ancient parable of the chessboard and the doubling grains of rice to make the point. Digital goods are non-rival and cost almost nothing to reproduce, so more and more of what the economy produces behaves unlike wheat or steel. And innovation is recombinant: new ideas are built by assembling existing ideas, so the stock of available building blocks grows faster than it is consumed. The consequences split into two streams, and the book's central analytical contribution is insisting that they are genuinely separate. The bounty is the growth in the volume, variety and quality of what the economy produces — a good thing, and larger than conventional statistics register, because free digital goods generate consumer surplus that never enters gross domestic product. The spread is the widening distance between economic winners and losers: between skilled and unskilled labour, between capital and labour, between superstars and everyone else in the same occupation. Technology drives both. The bounty is not compensation for the spread, and the spread is not evidence that the bounty is illusory. The book's diagnosis is that societies are getting the bounty and failing to manage the spread, and its prescriptions follow: not to slow the machines, but to reorganise education, taxation, immigration, infrastructure and research so that more people share in what the machines produce. Two features of the book explain why it has proved durable and also why it needs a companion. The first is its use of vivid technological demonstrations — a self-driving car on a public highway, a question-answering computer beating human champions on a quiz show, a robot that could be taught by being shown — to make an economic argument. Those demonstrations are memorable, but they are illustrations of a mechanism, not the mechanism itself, and students routinely mistake the one for the other. The second is that the book is a forecast. It was written in 2013 and published in 2014, before the deep learning boom, and eight years before large language models became a consumer product. A companion written now can do something the original could not: check the forecast against the record. That check is the controlling idea of this guide. Brynjolfsson and McAfee's framework — three forces producing two divergent consequences, requiring a policy response aimed at the distribution rather than the direction of technical change — has held up far better as an analytical apparatus than as a set of specific predictions. Measured productivity did not take off in the way the book's optimism implied; the great decoupling of productivity from median income, which the book treated as a durable structural fact, looks less clean in the data now available; and the automation of driving arrived a decade later and far more narrowly than the Google car demonstration suggested. Meanwhile the mechanism the book identified — that machines take over tasks, not occupations, and that the value of the tasks left to humans determines what happens to wages — has become the standard language of the entire field, including among the authors' sharpest critics. Understanding The Second Machine Age well means understanding both halves of that verdict. The guide is organised to build the argument the way the book builds it, then to test it. It begins with the historical claim about steam and the shape of human progress, and with the technological demonstrations the authors use to argue that the second machine age has arrived. It then takes each of the three forces in turn, giving each a full treatment, because students who can define exponential improvement, non-rivalry and recombinant innovation precisely can reconstruct almost everything else in the book from first principles. From there it follows the two consequences: the bounty, together with the deep measurement problem that makes the bounty hard to see in official statistics, and the spread, together with the task-based model of technical change that explains why the middle of the labour market hollowed out. A separate treatment of superstar economics explains why digital markets concentrate rewards so severely. The guide then sets out the book's recommendations — for individuals, for near-term policy, and for the long term, including the argument over a negative income tax versus a universal basic income — and the objections raised by Robert Gordon, Daron Acemoglu and others. A clearly marked final chapter brings the argument up to 2026: what generative artificial intelligence has done to the labour market and to measured output since 2022, what the best empirical studies show, and where the framework has earned its keep. Throughout, the distinction between what Brynjolfsson and McAfee argue and what this guide adds is kept explicit. Their claims are attributed to them. Evidence published after 2014, critiques of their position, and assessments of how the forecast has aged belong to the guide and are marked as such. The aim is not to summarise the book but to make you able to use it: to deploy its terminology accurately, to see where its mechanisms apply, and to know where an examiner or a reader will expect you to push back. Chapter 1: The First Machine Age and Why the Second One Matters Brynjolfsson and McAfee open with a question that sounds like a parlour game and is not: what, in the whole record of human existence, actually bent the curve? Their answer, borrowed from the historian Ian Morris, is that if you plot a composite index of human social development — population, energy capture, urbanisation, information technology, war-making capacity — across thousands of years, the line is essentially flat until the late eighteenth century, and then it turns almost vertically upward. Agriculture did not do that. Nor did writing, the wheel, bronze, the Roman road network, gunpowder, the printing press, the discovery of the Americas, or the founding of the great religions. All of these matter enormously to human affairs. None of them shows up as a break in the trend. What shows up is the steam engine, and specifically James Watt's separate-condenser improvement of the 1760s and 1770s, which turned Newcomen's inefficient pumping device into a general-purpose source of mechanical power. That is the first machine age: the moment when human and animal muscle stopped being the binding constraint on what could be produced, moved and built. The authors are careful about the causal claim. They do not say the steam engine alone caused modernity; they say it is the technology that coincides with the inflection and that is the plausible candidate, because power is upstream of nearly everything else. Factories, railways, steamships, mechanised agriculture and eventually electrification are all consequences of having abundant, portable, controllable power. The historical framing does a specific job in the argument, and it is worth being precise about what that job is. It sets a bar. If you want to claim that some new technology constitutes a genuine discontinuity rather than an ordinary improvement, you have to claim it does something to a fundamental constraint — not that it is impressive, or fast-growing, or widely adopted. The first machine age lifted the constraint on physical power. The second machine age, Brynjolfsson and McAfee argue, lifts the constraint on cognitive power: on our ability to process information, recognise patterns, generate and test hypotheses, communicate and coordinate. That is the whole basis for the analogy, and every subsequent chapter of the book depends on it holding. What the analogy commits the authors to Taking the steam analogy seriously has consequences the authors accept openly, and students should be able to name them. The first is that the effects will be broad rather than sectoral. A general-purpose technology, in the terminology economists use for this category, is one that spreads across almost all industries, improves continuously over decades, and spawns complementary innovations. Steam, electricity and the internal combustion engine qualify. If digital computing is a general-purpose technology of that class — and Brynjolfsson and McAfee are emphatic that it is — then its economic footprint should eventually show up everywhere, not merely in the technology sector. The second is that the effects will be slow to appear in the aggregate statistics, and then large. Electrification is the canonical case, studied in detail by the economic historian Paul David: electric motors were available in factories from the 1880s, but the productivity payoff did not arrive until the 1920s, because capturing it required rebuilding factories around the new technology rather than bolting it onto the old layout. The old design put a single massive steam engine in the basement driving overhead shafts, which forced every machine to be positioned by its access to the drive shaft. Small electric motors allowed a machine to be powered wherever it made sense to put it, which meant single-storey plants organised around workflow — but only for firms willing to scrap the buildings they had. That lag is not a failure of the technology. It is a feature of general-purpose technologies, and it becomes central to the book's treatment of productivity measurement. The third consequence is the uncomfortable one. The first machine age was not a smooth or painless transition for the people living through it. British real wages stagnated for decades during early industrialisation even as output per worker climbed — the "Engels pause" that economic historians still argue about. Handloom weavers were destroyed as a trade. The Luddites, whom the authors treat with more respect than the popular caricature allows, were not confused about technology; they were skilled workers correctly perceiving that a particular machine would eliminate the specific competence that gave them bargaining power. If the second machine age is really analogous to the first, then the analogy carries the disruption along with the abundance. Brynjolfsson and McAfee do not dodge this. Their entire treatment of the spread follows from taking it seriously. The claim that the discontinuity has already begun The book's more contentious move is not the historical analogy but the timing. The authors argue that we are not approaching the second machine age but are already some way into it, and that the mid-2000s to early 2010s mark the point where digital technologies crossed from disappointing to startling. Their evidence for this is deliberately anecdotal in form and systematic in intent. In 2004, the economists Frank Levy and Richard Murnane published The New Division of Labor, a careful and influential analysis of which human tasks computers could and could not perform. They used driving in traffic as their example of a task that resisted codification: executing a left turn against oncoming traffic requires integrating an enormous volume of visual information and applying judgement that the driver cannot articulate as rules. Their conclusion was reasonable, well argued, and — Brynjolfsson and McAfee point out — overtaken within a few years. The first Defense Advanced Research Projects Agency Grand Challenge, held in the Mojave Desert in March 2004, was a humiliation: no vehicle completed the course, and the best performer covered roughly 7.3 miles of a route of about 150 miles before failing. In October 2005, five vehicles finished a 132-mile desert course, with Stanford's entry, Stanley, winning in just under seven hours. By 2010 Google had autonomous vehicles logging public road miles in California. The authors' point is not that the Levy and Murnane analysis was sloppy. It is that a task can look permanently beyond machines right up until it is not, and that the interval between "clearly impossible" and "commercially demonstrated" had compressed to a handful of years. If that compression is general, then judgements about which human capabilities are safe from automation have a much shorter shelf life than anyone's career. The book assembles several such demonstrations, and the next chapter examines each in detail. What matters at this stage is the structure of the inference. Brynjolfsson and McAfee are not predicting from a trend line in the labour data. They are arguing from technological capability: here is a set of things machines could not do, here is evidence they now can, and the economic consequences follow from the capability. That is a legitimate method, and it is also the place where the book is most vulnerable, because a demonstration is not a deployment and a deployment is not an economy-wide effect. Google's self-driving car was operating on public roads in 2010; paid driverless taxi services reached meaningful scale only in the 2020s, and even in 2026 they operate in a limited set of cities. The capability was real. The economic transmission was slower than the demonstration implied. Where the argument came from The book did not appear from nowhere, and knowing its immediate prehistory helps explain both its urgency and its blind spots. In 2011 Brynjolfsson and McAfee published Race Against the Machine, a short self-issued volume written in the aftermath of the financial crisis. The economic puzzle of that moment was the jobless recovery: American output regained its pre-crisis level well before American employment did, and the employment-to-population ratio, which had already been sliding since 2000, fell sharply and then refused to climb back. The standard explanations were cyclical — weak demand, damaged balance sheets, a housing market that had trapped workers in place. A second camp argued the problem was structural in the sense of mismatch: workers with the wrong skills in the wrong places. Brynjolfsson and McAfee proposed a third reading. Firms had used the downturn to substitute capital, specifically information technology, for labour on a large scale, and much of that substitution would not be reversed when demand returned. The jobs were not waiting to come back because the tasks that constituted them had been reassigned to software. That argument was contested at the time and remains contested. What matters for reading The Second Machine Age is that the later book extends it in two directions. It pushes backwards to establish the historical scale of the claim, and forwards to separate the distributional problem from the growth problem. Race Against the Machine is mostly a book about employment. The Second Machine Age is a book about welfare, in which employment is one channel among several. The second piece of background is a chart the authors popularised, which they called the great decoupling. Plot United States labour productivity and real median household income on the same axes from 1947. Through the post-war decades the two lines rise together, which is what the textbook predicts: workers get paid roughly what they produce, so when output per hour rises, the typical household's income rises with it. From some point in the 1970s or 1980s — the precise date depends on which income series and which price deflator you use, and this matters more than it sounds — the lines separate. Productivity keeps climbing. Median income flattens. Whatever the extra output is buying, it is not going to the household in the middle of the distribution. That chart is the empirical heart of the book's case that something has changed, and it deserves a careful health warning that the book itself does not fully supply. The gap between the two lines is partly a measurement artefact. Productivity is computed from output deflated by a producer price index; household income is deflated by a consumer price index; the two indices diverge, and a share of the apparent decoupling is that divergence rather than any economic event. Productivity is measured per hour while income is measured per household, and household size has fallen. Compensation includes employer-paid health insurance, which grew rapidly, while money income does not. Careful reconstructions that correct for these factors shrink the gap without eliminating it. And the subsequent data complicates the story further: real median household income in the United States reached a record of $87,460 in 2025, 2.6 percent above the previous year, on the Census Bureau's measure — the strongest reading in the series since it began in 1967. A guide written in 2014 would have described a decade and a half of stagnation. A guide written now has to acknowledge that median households have made real gains since, which does not refute the decoupling thesis but does qualify it. The third strand of background is intellectual rather than empirical. The authors are writing within a tradition of economics that takes technology seriously as an endogenous force rather than treating it as an unexplained residual. Robert Solow's growth accounting in the 1950s established that most of the growth in output per worker could not be explained by adding capital or labour, and the unexplained portion — total factor productivity — was attributed to technical change. Paul Romer's work in the late 1980s and 1990s made the production of ideas itself the object of analysis, emphasising that ideas are non-rival and that this non-rivalry is what permits sustained growth rather than diminishing returns. Brynjolfsson and McAfee's second and third forces are applied Romer: non-rivalry as a property of digital goods, and the combinatorial production of new ideas. Students who can place the book in this lineage write far better about it than those who treat it as journalism about robots. Three forces, two consequences, one policy problem The architecture of the book is simple enough to hold in your head, and holding it there is most of what a student needs. Three forces produce the second machine age. Two consequences follow. One policy problem results from the gap between them. The three forces are exponential improvement in digital hardware, the peculiar economics of digital goods, and the recombinant character of innovation. They are analytically distinct — one is about hardware trends, one about the properties of information as a commodity, one about how ideas are produced — but they reinforce one another. Cheap computation makes it feasible to digitize more; digitized things become raw material for recombination; recombination produces new applications that generate demand for more computation. Table 1 sets out the three forces, the mechanism behind each, and the economic property each one generates, and it is worth returning to as the argument develops. Table 1. The three forces of the second machine age. Force Core mechanism Book's leading example Economic property generated Exponential Sustained doubling of computing capacity per unit cost Moore's Law; the chessboard parable Constraints fall away abruptly, late in the sequence Digital Information goods are non-rival and near-costless to copy Digitized maps, text, music, sensor data Near-zero marginal cost; vast consumer surplus Recombinant New ideas are combinations of existing ideas The self-driving car as assembled components Innovation prospects grow rather than exhaust The two consequences are the bounty and the spread. The bounty is the increase in the quantity, quality and variety of goods and services available, much of it delivered at zero price and therefore substantially invisible to gross domestic product. The spread is the increase in dispersion: between high-skill and low-skill workers, between owners of capital and suppliers of labour, and between the top handful of performers in a field and everyone else in it. Brynjolfsson and McAfee insist that both are consequences of the same technological forces, and that treating them as a trade-off — accepting more inequality as the price of more growth — misunderstands the mechanism. The spread is not a payment for the bounty. It is a separate output of the same process, and it is the one that responds to policy. The policy problem falls out of this. If technology is generating enormous value and distributing it narrowly, the sensible response is neither to obstruct the technology nor to wait for the market to redistribute on its own. It is to change the institutions that determine who captures the value: education systems that determine who can work alongside machines, tax systems that determine how returns to capital and labour are treated, immigration rules that determine where innovators locate, public investment in the research and infrastructure that private firms underinvest in. The book's recommendations are unusually concrete for a work of this kind, and they are examined in full later in this guide. One clarification is worth making now, because it prevents a common misreading. The Second Machine Age is not a book about robots taking all the jobs. The authors explicitly reject the claim that technological unemployment is inevitable or imminent, and they spend considerable effort on why humans and machines are complements in a great many settings. Their worry is compositional and distributional: which tasks remain valuable, who can perform them, and what happens to the earnings of people whose particular bundle of skills has just been rendered cheap. That is a different and more defensible claim than the one the book is often credited with, and getting it right is the first test of whether you have read it carefully. Key Takeaways · Brynjolfsson and McAfee argue that the Industrial Revolution is the only event in the historical record that visibly bends the curve of human development, and that the steam engine is the technology responsible. · The second machine age is defined by analogy: digital technology relieves the constraint on cognitive power as steam relieved the constraint on physical power. · Treating computing as a general-purpose technology commits the authors to broad effects, long lags before those effects appear in the statistics, and substantial disruption to existing workers. · The book argues the discontinuity has already begun, using technological demonstrations rather than labour-market trends as evidence — a method that is powerful but conflates capability with economic transmission. · The architecture is three forces (exponential, digital, recombinant) producing two consequences (bounty and spread) and one policy problem (who captures the value). Review Questions 1. What specific claim does the historical comparison with the steam engine commit Brynjolfsson and McAfee to, and what would count as evidence against it? 1. Why does the electrification of factories serve as the authors' template for how a general-purpose technology affects measured productivity? 2. Explain why the authors treat Levy and Murnane's 2004 analysis of driving as instructive rather than as an error. 3. Distinguish between a technological demonstration, a commercial deployment, and an economy-wide effect. Where does the book's argument rely on eliding these? 4. Why do the authors insist that the bounty and the spread are separate consequences rather than a trade-off? Chapter 2: The Demonstrations — What the Machines Could Suddenly Do Brynjolfsson and McAfee build their case for the second machine age out of a small number of technological events, each chosen because it broke a barrier that competent observers had recently judged durable. The demonstrations are the most quoted part of the book and the most frequently misremembered. Students who get them slightly wrong — who say Watson understood natural language, or that Baxter was a commercial success — undermine everything they write afterwards. This chapter states what actually happened in each case, and then examines what the examples are and are not evidence for. Driving, and the collapse of a reasonable prediction The self-driving car is the book's anchor example because its timeline is so compressed. Frank Levy and Richard Murnane's The New Division of Labor, published in 2004, is a serious piece of labour economics, not a casual prediction. Their framework distinguished tasks that can be expressed as explicit rules from tasks that rely on tacit knowledge — pattern recognition and judgement that the practitioner cannot fully articulate. Computers, they argued, are excellent at the former and hopeless at the latter, and driving in traffic sat firmly in the latter category. Executing an unprotected left turn requires reading the speed and intention of oncoming vehicles, anticipating pedestrians, and making a continuous risk judgement, none of which the driver could write down as a procedure. In March 2004, the Defense Advanced Research Projects Agency ran the first Grand Challenge, offering a prize for an autonomous vehicle that could complete a roughly 150-mile desert route. Every entrant failed; the furthest travelled about 7.3 miles. This looked like confirmation. Nineteen months later, in October 2005, five vehicles finished a 132-mile course. Stanford's Stanley won in six hours and fifty-four minutes at an average of about 19 miles per hour, ahead of two Carnegie Mellon entries. By 2010 Google had a programme running autonomous vehicles on Californian public roads with safety drivers, and the authors report riding in one. The lesson Brynjolfsson and McAfee draw is about the shape of technological progress rather than about cars. Capabilities that improve exponentially do not approach a threshold gradually; they are far from it and then past it, and the interval in which they are visibly "almost there" is short. Any judgement of the form "machines will never be able to do X" is therefore a judgement about the present state of the art dressed up as a claim about the nature of X. This guide would add a qualification the book could not have made. The gap between demonstration and economic effect turned out to be long. Public-road autonomy was demonstrated in 2010; driverless commercial services began carrying paying passengers without a safety driver only in the late 2010s, and scale arrived in the 2020s. By March 2026 Waymo was providing roughly 500,000 paid rides a week across about ten United States cities, having grown tenfold from around 50,000 weekly rides in May 2024. That is a genuine industry and a genuine displacement risk for drivers in those cities, and it is also, after sixteen years, a small fraction of American road transport. The capability claim was vindicated. The implied speed of economic transmission was not. Watson, and the difference between answering and understanding In February 2011, IBM's Watson system played three televised games of Jeopardy! against Ken Jennings and Brad Rutter, the show's two most successful human contestants, and won a first prize of one million dollars. The result is impressive for reasons that are easy to state imprecisely, so it is worth being careful. Jeopardy! is hard for a machine for three separate reasons. The clues are in natural language and are frequently oblique, relying on puns, cultural allusion and misdirection. The domain is unbounded: a clue can concern anything. And the game is adversarial in real time, requiring the system to estimate its own confidence and decide whether to buzz in, since a wrong answer costs money. Watson handled all three by running many candidate-answer generation strategies in parallel across a large corpus of unstructured text, scoring the candidates by multiple independent pieces of evidence, and combining those scores into a confidence estimate learned from past games. It was not connected to the internet during the match; it had already ingested its sources. What Watson did not do is understand anything, in the sense a person means by that word. It was a statistical evidence-aggregation machine of considerable sophistication. Brynjolfsson and McAfee are reasonably careful about this, and their point is economic rather than philosophical: for the purposes of doing valuable work, it may not matter whether the machine understands. If a system can produce reliably correct answers to open-ended natural-language questions, the tasks that consist of producing such answers become automatable regardless of what is happening inside. The subsequent history is instructive and should temper any triumphalism. IBM spent the following decade attempting to commercialise Watson, most prominently in oncology decision support, and the results fell well short of the promises. Divisions of the Watson Health business were sold off in 2022. Winning a quiz show is not the same problem as advising on cancer treatment, where the cost of error is asymmetric, the evidence base is contested, and the data is fragmented across incompatible institutional systems. This is a useful corrective to the book's method: a demonstration proves that a capability exists under demonstration conditions. Machine translation and the arrival of statistical methods The book's third capability example is language translation, and it is the one that has aged best. For four decades machine translation was attempted primarily by encoding grammatical rules and bilingual dictionaries, and it produced output that was usable only as a rough gist. The shift that changed the field was statistical: rather than teaching the machine grammar, feed it enormous quantities of text that has already been translated by humans — parliamentary proceedings, United Nations documents, multilingual corporate websites — and let it learn which sequences of words in one language correspond to which in another. The approach required no linguistic theory and improved automatically as more data became available. Brynjolfsson and McAfee use the business application of this to make their point, citing services that translate customer-support conversations in real time so that agents and customers who share no language can transact. The relevant economic property is that the quality improves as a function of data volume and computing power, both of which were growing exponentially, rather than as a function of linguists' effort, which was not. Since 2014 the field has changed again, twice: first to neural machine translation around 2016, which improved fluency dramatically, and then to large language models, which handle translation as one capability among many. The direction the authors identified was right, and the improvement has been larger than they suggested. Translation is also a cautionary case for the labour argument, because the profession did not disappear. Demand for translated content expanded, and the human role shifted towards post-editing, certification, and the categories where error is expensive — legal, medical, literary. This is exactly the pattern of task reallocation the book's own framework predicts, and it is the pattern its rhetoric sometimes obscures. Baxter and the physical world Robotics supplies the book's counterpoint to its software examples, because robots must contend with physical reality, which is unforgiving. Rethink Robotics, founded by the roboticist Rodney Brooks, introduced Baxter in 2012: a two-armed industrial robot with a screen displaying a cartoon face, priced at around twenty-two thousand dollars — roughly an order of magnitude below the installed cost of a conventional industrial robot arm at the time. Baxter's design was significant on three counts. It was safe to work next to, using compliant joints and force sensing rather than being caged, which meant it could be placed on a production line among people. It was programmed by demonstration: an operator physically moved its arms through a task and Baxter learned the motion, so deploying it did not require a robotics integrator. And it was cheap enough that a small manufacturer could plausibly buy one. Together these attributes pointed at a category that barely existed — the collaborative robot, or cobot — aimed at small-batch, high-variety work that had never justified conventional automation. The honest postscript is that Rethink Robotics failed. The company shut down in October 2018, having raised around $150 million, after sales fell far short of expectations; its intellectual property was acquired by a German firm. The collaborative robot category it helped create did succeed, and is dominated commercially by other manufacturers. Baxter itself was not accurate or fast enough for most production work, and the tasks it was ideally suited to were often tasks where a human was simply cheaper. The book is not wrong that Baxter mattered — it demonstrated a design philosophy that became standard — but a student citing it as evidence of automation's advance should know that this particular machine did not survive. Kiva, Amazon, and what large-scale deployment actually looks like The most economically consequential example in the book is also the least glamorous. Kiva Systems built warehouse robots that invert the logic of order fulfilment: instead of sending a human to walk to the shelf, the robot drives the entire mobile shelving unit to a stationary human picker. The saving is in walking time, which in a conventional fulfilment centre can consume the majority of a picker's shift. Amazon acquired Kiva in March 2012 for $775 million in cash and stopped selling the systems to anyone else, which tells you how valuable it considered the technology. Amazon's subsequent trajectory is the best available natural experiment on the book's central question, and it is worth following past the book's publication. By July 2025 Amazon reported having deployed more than one million robots across its operations, beginning from a single robot type in 2012. Over the same period the company's global workforce grew enormously; it employs well over a million people. Amazon states that its next-generation robotic fulfilment centre in Shreveport, Louisiana requires about 30 percent more employees in reliability, maintenance and engineering roles than a conventional site, and that it has put more than 700,000 employees through upskilling programmes since 2019. Three readings of this are possible, and a good answer holds all three. The optimistic reading is that automation and employment grew together, exactly as the complementarity argument predicts. The sceptical reading is that Amazon's employment grew because e-commerce volume exploded, and the relevant counterfactual is how many people the same volume would have required without robots — a number nobody knows, but certainly larger. The compositional reading, which is the one the book's own framework recommends, is that the content of the work changed: fewer hours walking, more hours at fixed stations working at machine-set pace, plus a smaller number of much better-paid technical roles maintaining the fleet. The aggregate headcount is the least interesting statistic in the story. Instagram, Kodak, and the disappearing payroll One further comparison appears in the book and belongs with the demonstrations, because it makes a point about capital and labour that none of the others do. In April 2012 Facebook agreed to acquire Instagram for about a billion dollars. Instagram was a photo-sharing service roughly eighteen months old with a staff of around a dozen people. Three months earlier, on 19 January 2012, Eastman Kodak had filed for Chapter 11 bankruptcy protection. Kodak had defined the consumer photography industry for a century, had employed well over a hundred thousand people at its height, and had — this is the part that makes the story sting — invented the digital camera internally in the 1970s and declined to let it cannibalise the film business. The authors use the juxtaposition to illustrate two things at once. The first is the digitization force: photography went from a physical good with per-unit costs to an information good with essentially none, and the value migrated with it. The second is distributional. A firm can now capture an enormous market with almost no payroll, which means the returns flow overwhelmingly to a small number of founders, engineers and equity holders rather than to a large workforce. The industry did not shrink — far more photographs are taken now than ever were on film — but the labour required to serve it collapsed. The comparison is rhetorically effective and somewhat unfair as accounting, and a careful student should say so. Instagram ran on infrastructure it did not own and did not have to build: smartphone cameras manufactured by hundreds of thousands of workers, cellular networks, data centres, payment systems. Its thirteen employees sat on top of a very large amount of embodied labour. What is true, and is the point worth keeping, is that the marginal labour required to add another user to a digital service is close to zero, and that this property determines who gets paid. Why expert predictions about machine capability fail Running underneath all these examples is a claim about forecasting that deserves separate statement, because it is the part of the chapter most useful outside the book. Predictions about what machines cannot do are usually generated by introspection. The forecaster imagines performing the task, notices that their own performance depends on something they cannot articulate, and concludes that the task requires a faculty machines lack. This is the structure of Levy and Murnane's argument about driving, and it is also the structure of most confident claims about the limits of artificial intelligence made since. The trouble is that the inference is invalid. That a human accomplishes a task by means of inarticulable judgement does not imply that the task can only be accomplished that way. Machines frequently solve problems by a completely different route — brute-force search, statistical pattern-matching over volumes of data no person could read — and arrive at comparable output. This is sometimes called Moravec's paradox, after the roboticist Hans Moravec, who observed in the 1980s that the tasks humans find hardest, such as playing chess at a high level or performing complex arithmetic, are comparatively easy to mechanise, while the tasks a small child performs effortlessly — recognising a face, walking over uneven ground, picking up an unfamiliar object — are extraordinarily hard. The explanation is evolutionary: perception and locomotion have been under selection for hundreds of millions of years and are implemented in enormously refined but entirely unconscious machinery, whereas abstract reasoning is recent, thin, and deliberate. Our intuitions about difficulty track how much conscious effort a task costs us, which is almost exactly the wrong signal. Brynjolfsson and McAfee's demonstrations are all cases where the paradox bit: driving and object manipulation are Moravec-hard, and progress on them was slow and then sudden because it depended on data and compute rather than on insight. The practical implication for anyone reasoning about automation is to stop asking whether a job seems to require human judgement and start asking two narrower questions. Can the task be decomposed into steps with observable inputs and outputs? Does a large volume of examples of the task being performed correctly already exist, or could it be generated? Where the answer to both is yes, the task is a candidate regardless of how sophisticated it feels to perform. What the demonstrations do and do not prove The examples establish, collectively and persuasively, that the boundary of machine capability moved substantially and unexpectedly in the decade before the book was written, and moved in directions that expert judgement had recently ruled out. That is a real finding and the book is right to insist on it. They do not establish three things that readers frequently take from them. They do not establish a rate of diffusion; every one of these capabilities took far longer to reach economic scale than the demonstration suggested, and two of the five specific commercial vehicles — Watson Health and Rethink Robotics — failed outright. They do not establish aggregate effects, because a capability that is deployed in a few thousand firms does not move national statistics. And they do not, by themselves, establish anything about employment, since whether a capability displaces or complements workers depends on the elasticity of demand for the output, on what the freed-up labour does next, and on the cost of the machine relative to the wage. The book supplies those arguments separately and properly. The demonstrations are the premise, not the conclusion. Key Takeaways · The Grand Challenge sequence — total failure in 2004, five finishers in 2005 — is the book's evidence that machine capability crosses thresholds abruptly rather than gradually. · Watson's 2011 Jeopardy! victory demonstrated open-domain question answering by evidence aggregation and confidence estimation, not understanding; IBM's later failure to commercialise it in medicine shows the gap between demonstration and application. · Machine translation improved through statistical rather than rule-based methods, making quality a function of data and computation; the profession reallocated towards post-editing rather than disappearing. · Baxter demonstrated safe, cheap, demonstration-programmed robotics and defined the collaborative robot category, but Rethink Robotics shut down in 2018. · Amazon's acquisition of Kiva in 2012 and its subsequent growth to over a million robots alongside a growing workforce is the book's most consequential case, and supports a compositional rather than a headcount reading. Review Questions 1. Reconstruct Levy and Murnane's reasoning about driving. Which part of their framework was wrong, and which part survives? 1. What three distinct technical problems did Watson have to solve, and why does the distinction between answering and understanding matter economically rather than philosophically? 2. Why did the shift to statistical methods change the growth path of machine translation quality? 3. Rethink Robotics failed commercially. Does that weaken the book's use of Baxter as evidence? Give the strongest argument on each side. 4. Amazon's robot fleet and its workforce both grew. Explain why headcount is a poor test of the automation hypothesis, and state what a better test would measure. Chapter 3: The Exponential Force — Moore's Law and the Second Half of the Chessboard The first of the three forces is the simplest to state and the hardest to feel. Digital technology improves at a rate that is roughly constant in percentage terms, which means the absolute increments grow without limit. Brynjolfsson and McAfee argue that most of the confusion about the second machine age — including the confusion of very intelligent people — comes from minds calibrated to linear change trying to reason about a process that is not linear. What Moore's Law actually says In April 1965, Gordon Moore, then director of research at Fairchild Semiconductor and later a co-founder of Intel, published a short article in Electronics magazine under the title "Cramming more components onto integrated circuits." He had four data points. He observed that the number of components that could be placed on an integrated circuit at minimum cost per component had been roughly doubling each year, and he projected the trend forward for a decade. In 1975 he revised the doubling interval to about two years. The regularity has been called Moore's Law ever since. Several clarifications are essential, because the term is used loosely and the looseness causes real errors in student writing. Moore's Law is not a law of physics. It is an observed empirical regularity that became, in the semiconductor industry, a self-fulfilling coordination device: firms, equipment suppliers and materials producers planned their capital investment against a published roadmap of expected improvement, and the roadmap made the improvement happen. When the industry's collective expectation shifts, so does the trend. Moore's original statement is about transistor density and cost, not about speed, and certainly not about the vaguely defined "power" of computers. Related but distinct regularities cover clock speed, cost per operation, storage density, memory bandwidth and network capacity, and they have different rates and different histories. Clock-speed scaling essentially stopped around 2004 when power dissipation made further frequency increases impractical; performance gains after that came from putting multiple cores on a chip, from specialised architectures, and from software. Conflating these is the commonest error in discussing the topic. Brynjolfsson and McAfee's actual claim is broader and more defensible than the narrow semiconductor one. They argue that sustained exponential improvement has been observed across a family of digital technologies over decades, that it has not stopped, and that its economic consequences follow from the compounding rather than from any particular mechanism. Whether the next factor of two comes from smaller transistors, from three-dimensional stacking, from purpose-built accelerators or from better algorithms is, for their purposes, immaterial. The parable of the chessboard The device the book uses to convey compounding is an old story, retold by the inventor Ray Kurzweil, about the invention of chess. The inventor presents the game to his emperor, who offers a reward. The inventor asks for rice: one grain on the first square of the board, two on the second, four on the third, doubling on each of the sixty-four squares. The emperor, thinking this modest, agrees. The arithmetic is where the lesson lives. Doubling through the first half of the board — thirty-two squares — yields a total of about four billion grains, which is a large heap of rice, roughly a field's worth. It is a quantity an emperor could deliver. But the total on the sixty-fourth square alone is two to the sixty-third power, which is more than nine quintillion grains: a pile that would dwarf a mountain. Crossing from the first half to the second half of the board is the moment at which the quantities stop being comprehensible in ordinary terms. The point is not that the numbers get big. It is that within an exponential process, every doubling adds more than the entire history that preceded it, and therefore the process looks manageable and then abruptly does not. Brynjolfsson and McAfee argue that digital technology entered the second half of its chessboard sometime in the 2000s. Their arithmetic for this is a rough count: if you date the start of sustained doubling to the era of commercial business computing around 1958 and assume doublings roughly every eighteen months, thirty-two doublings brings you to the mid-2000s. The dating is a heuristic rather than a measurement, and the authors present it as such, but the underlying claim is serious: the absolute increments in computing capacity delivered each year now dwarf anything delivered in earlier decades, even though the percentage rate is unchanged. The practical consequence is a rule for interpreting stalled technologies. A capability that has been "ten years away" for thirty years may be genuinely impossible, or it may be on an exponential curve that has not yet crossed the threshold. From inside the curve, the two look identical until suddenly they do not. Speech recognition spent decades as a perpetual near-miss and then, within a few years in the 2010s, became good enough to be embedded in every phone. The same profile applies to image recognition, to machine translation, and later to text generation. Why linear intuition fails The book spends real effort on the psychology of the thing, and the effort is justified, because the analytical error it describes is committed constantly by people who could do the arithmetic if asked. Human beings reason well about processes that add a constant amount per period and badly about processes that multiply by a constant factor. The reason is not innumeracy. It is that almost every quantity we have evolved to track — distance walked, food gathered, a child's height — behaves additively over the horizons that matter. Exponential processes in nature are typically short-lived, because they run into a resource ceiling and become logistic curves. Sustained exponentiality over decades is a peculiarity of certain human-engineered systems, and we have no trained intuition for it. The practical symptom is systematic mis-forecasting in two opposite directions at once. Over short horizons, people overestimate what an exponential process will deliver, because they extrapolate from the excitement of the moment rather than from the base. Over long horizons, they wildly underestimate, because they cannot hold in mind what thirty more doublings does. Both errors are visible in the discourse around every digital technology, often in the same commentator within a single paragraph. A useful discipline for writing about this is to work in doubling times rather than in percentages. A quantity growing at 7 percent a year doubles in roughly ten years; at 14 percent, in five; at 40 percent, in about two. The rule of thumb — divide seventy by the annual percentage growth rate to get the doubling time in years — converts a rate into an interval the mind can handle. If a capability is currently at one percent of what would be required to displace some category of work, and it is doubling every two years, the honest statement is not that it is far away. It is that it is about fourteen years away, and that it will look far away for twelve of them. The corollary the authors emphasise is about the timing of surprise. In an exponential process the last doubling before a threshold takes the system from half of what is needed to all of it. The period during which the technology is visibly nearly adequate is therefore always brief, regardless of how long the process has been running. This is why capabilities appear to arrive suddenly even when the underlying trend has been public and stable for decades, and why "nobody saw this coming" is so often false as a matter of record and true as a matter of experience. Cost, not just capability The economically important form of the exponential claim concerns price rather than performance, and the book makes this vivid with a comparison that repays careful handling. ASCI Red, built for the United States Department of Energy at Sandia National Laboratories, was the first computer to sustain a trillion floating-point operations per second, crossing the teraflop threshold in December 1996. It occupied roughly the floor area of a tennis court, consumed power on the scale of a large building, and cost tens of millions of dollars. It was decommissioned in June 2006, having held the record as the world's fastest computer for four consecutive years, longer than any other machine. In the year ASCI Red was retired, Sony released the PlayStation 3, a games console priced in the hundreds of dollars whose graphics and processing hardware delivered performance in the same teraflop territory on the specific kinds of parallel arithmetic it was built for. The comparison is not exactly like for like — a supercomputer's sustained general-purpose throughput on scientific workloads is a different quantity from a console's peak parallel arithmetic, and any serious writer should flag that — but the order-of-magnitude point survives the caveat. A capability that cost tens of millions of dollars and a dedicated building in 1996 cost a few hundred dollars and sat under a television a decade later. That collapse in price is what makes exponential improvement economically transformative rather than merely technically interesting. When something gets cheap enough, it stops being a capital project to be justified and becomes an input to be used casually, and the applications multiply in ways nobody planned. Cheap computation is why every vehicle, appliance, machine tool and doorbell now contains a processor; why a start-up can rent supercomputer-class resources by the hour; and why training a large machine learning model, an activity that would once have required a national laboratory, is within reach of a few thousand organisations. Where exponential improvement does not reach A discipline the book applies less consistently than it should is asking which parts of the economy actually sit on an exponential curve. Getting this right is essential, because the answer determines how large the aggregate effect can be. Computation, storage, bandwidth and — through them — anything that can be reduced to information have improved exponentially for decades. Many other inputs have not. The cost of constructing a mile of urban rail, a hospital bed, a housing unit or a kilometre of transmission line has risen rather than fallen in real terms in most rich countries. The cost of a university education has risen. The cost of a medical procedure has risen. Energy storage and photovoltaic generation are genuine exceptions, having followed steep and sustained learning curves, and they are worth noting precisely because they are rare. The economist William Baumol's account of cost disease explains the pattern. In sectors where output per worker cannot easily be raised — a string quartet requires four players for the same duration it did in 1800 — wages nonetheless rise, because employers must compete for labour with the sectors where productivity is climbing. The relative price of the stagnant sectors therefore rises without end, and they absorb a growing share of spending and employment. Over time, an economy where one sector improves exponentially and the rest do not comes to be dominated, in expenditure terms, by the sectors that did not improve. The exponential sector shrinks as a share of the economy precisely because it is getting cheaper. This is not an objection to the book's technological claim; it is a constraint on how much that claim can move aggregate statistics, and it is the mechanism underlying much of the productivity puzzle examined later in this guide. If computation gets a thousand times cheaper and computation is three percent of costs, the effect on total costs is bounded at roughly three percent no matter how spectacular the improvement. Growth comes from the exponential technology making the non-exponential sectors more productive — from software changing how hospitals, construction firms and schools work — and that transmission depends on organisational change, regulation and complementary investment, none of which is exponential. Brynjolfsson and McAfee are aware of this and address it through their treatment of complementary innovation, but the emphasis of the writing points the other way, and students who quote only the chessboard passage end up asserting more than the book can support. The stronger formulation, and the one worth adopting, is this: exponential improvement in a general-purpose input creates the potential for broad transformation, and the realisation of that potential is governed by slow, non-exponential processes of adoption and reorganisation. What has happened since, and how to write about it honestly The Second Machine Age asserts that Moore's Law shows no sign of stopping. That assertion requires qualification a decade on, and handling the qualification well is a good test of whether a student understands the argument or is repeating it. Transistor density scaling has slowed and become far more expensive. The economic form of Moore's Law — cost per transistor falling with each node — weakened substantially in the 2010s, as the capital cost of leading-edge fabrication plants rose into the tens of billions of dollars and the number of firms capable of operating at the frontier fell to a handful. Dennard scaling, the related principle that shrinking transistors also reduced their power consumption proportionally, broke down around 2005, which is why clock speeds stopped rising. And yet the delivered capability relevant to the book's argument has continued to improve at an extraordinary rate, by routes Moore did not describe. Specialised processors designed for the matrix arithmetic that machine learning requires deliver performance per dollar far beyond what general-purpose chips achieve. Parallelism at data-centre scale substitutes coordination for single-chip speed. Algorithmic improvement has contributed at least as much as hardware in several important domains: the compute required to train a neural network to a given standard of image classification fell by orders of magnitude over the 2010s through better methods alone. And the total quantity of computation applied to frontier machine learning models grew at a rate far exceeding Moore's Law during the 2010s and 2020s, driven not by cheaper chips but by vastly larger capital expenditure on buying them. That last point matters for the book's logic and is worth stating plainly, because it is the kind of distinction that separates a strong answer from an adequate one. Brynjolfsson and McAfee's exponential force is an argument about falling cost: capability arrives because the price of computation collapses, so anyone can have it. A great deal of the recent frontier progress has instead come from rising expenditure: capability arrives because a small number of very large firms are spending unprecedented sums. Investment in data centres, computing hardware and networking reached roughly 0.8 percent of United States gross domestic product by the first quarter of 2026, with total computing infrastructure investment at around 1.5 percent of output against a 2015–2022 average near 0.7 percent, according to analysis by Epoch AI. Those are different mechanisms with different distributional implications. Cheap computation democratises; expensive computation concentrates. The second machine age the book describes is a story about the first. Much of what has happened since 2020 is a story about the second. Key Takeaways · Moore's 1965 article observed doubling of components per integrated circuit at minimum cost, revised in 1975 to roughly every two years; it is an empirical regularity and industry coordination device, not a physical law. · The claim concerns transistor density and cost, not clock speed; Dennard scaling and clock-speed growth ended around 2004–2005. · The chessboard parable illustrates that in an exponential process each doubling exceeds the entire accumulated total before it, so change appears manageable and then abruptly is not. · The economically decisive version of the exponential claim is about collapsing price: ASCI Red's teraflop capability of 1996 was matched a decade later by a consumer games console. · Density scaling has slowed and grown costly since 2014, but delivered capability has continued to rise through specialised hardware, parallelism, algorithmic improvement, and sharply increased capital expenditure — the last of which concentrates rather than democratises access. Review Questions 1. State Moore's original claim precisely. Name two distinct technological trends that are often mistakenly labelled Moore's Law. 1. Explain why the second half of the chessboard is a claim about the size of increments rather than about rates of growth. 2. Why do the authors treat falling cost, rather than rising performance, as the economically decisive form of exponential improvement? 3. The ASCI Red and PlayStation 3 comparison is not strictly like for like. State the objection and assess how much it damages the argument. Distinguish capability gains that come from cheaper computation from those that come from greater spending on computation. What different distributional consequences follow? Hashtags: #TheAutomationEconomy #TheSecondMachineAge #DigitalTransformation #Automation #TechnologicalChange #ExponentialTechnologies #MooresLaw #SecondHalfOfTheChessboard #DigitalGoods #NonRivalGoods #NearZeroMarginalCost #RecombinantInnovation #GeneralPurposeTechnology #BountyAndSpread #ConsumerSurplus #GreatDecoupling #TaskBasedAutomation #HumanMachineComplementarity #LabourMarketPolarization #SkillBiasedTechnologicalChange #SuperstarEconomics #ProductivityGrowth #DigitalCapital #EconomicInequality #FutureOfTheAutomationEconomy

Download the Book (PDF): Introduction Writing code that works is easy; writing code that survives is incredibly difficult. "Uncle Bob" Martin's classic text outlines the universal rules of software architecture. Yet the book's deep dives into object-oriented programming paradigms, dependency inversion and component cohesion can quickly become a confusing blur of engineering jargon for students studying the macro-level management of software teams. This guide structures the engineering logic into managerial theory. It is designed specifically to explain Clean Architecture by Robert C. Martin, separating the high-level architectural philosophy from the granular code examples. We meticulously break down the SOLID principles and the concept of separating business rules from delivery mechanisms. Equipped with simplified structural models and clear takeaways, this guide gives you a massive advantage in understanding how software structure dictates business flexibility. The Book and Its Author Robert C. Martin, known across the software industry as "Uncle Bob", has been programming professionally since the early 1970s. He was one of the seventeen signatories of the Manifesto for Agile Software Development in 2001, ran a long-standing consultancy and training firm, and edited the C++ Report in the 1990s. His reputation outside specialist circles rests mainly on three books. Clean Code (2008) is about the craft of writing readable functions and classes. The Clean Coder (2011) is about professional conduct: saying no, estimating honestly, and practising the discipline of testing. Clean Architecture: A Craftsman's Guide to Software Structure and Design, published by Prentice Hall in 2017, completes the arc by moving from the line of code and the individual programmer up to the shape of whole systems. The book is short by the standards of software engineering texts, and it is organised as a climb in scale. It opens with an argument about why structure matters at all, then works upward from programming paradigms, through principles for designing modules and classes, through principles for grouping those modules into deployable components, to architecture proper, and finally to the parts of a system Martin insists are not architecture at all: databases, the web and frameworks. Several chapters draw on long professional experience, and two contributors add material of their own. James Grenning writes on embedded systems, and Simon Brown supplies a closing chapter that questions how code should be packaged. Much of what the book says was not new in 2017. The SOLID principles had circulated since the early 2000s, and Martin's component metrics had appeared in his earlier writing on agile development. What the book did was assemble these ideas into a single argument and give that argument a memorable image: a set of concentric circles with business rules at the centre and every technical detail pushed to the outside. That image, and the rule that governs it, became one of the most widely cited reference models in modern software design. Teams building mobile apps, banking platforms and cloud services describe their systems in its vocabulary, whether or not they have read the book. Why a Manager Should Care A student of management might reasonably ask why any of this belongs outside the engineering department. The answer is that software structure is an economic variable. It determines how much a change costs, how long a new feature takes, how many people must coordinate to ship anything, and whether an organisation can switch suppliers, databases or delivery channels without rewriting its core systems. Martin states the goal of architecture in explicitly economic terms: the aim is to minimise the human effort required to build and maintain a system over its life. A well-structured system keeps the cost of each change roughly flat as the system grows. A badly structured one makes every change more expensive than the last, until the organisation is paying more and more engineers to deliver less and less. Those outcomes are rarely visible on a project plan. Structure is invisible to customers and usually to executives. Its effects arrive late and indirectly, as slipping schedules, rising headcount, and a creeping sense that the software has become a constraint on the business rather than an instrument of it. A manager who understands architecture can recognise those symptoms early, ask the right questions of technical staff, and make better decisions about when to invest in structure and when to accept a shortcut. The Controlling Idea The argument of this guide can be put in one sentence. Good architecture is the deliberate arrangement of dependencies so that the parts of a system most valuable to the business, its rules and policies, never depend on the parts most likely to change, its technologies and delivery mechanisms, and this arrangement is what keeps the cost of change low and the organisation's options open. Almost every principle in Martin's book is a restatement of that idea at a different scale. The Single Responsibility Principle separates code by the people who ask for changes. The Dependency Inversion Principle lets high-level policy stay ignorant of low-level detail. The component principles decide which way whole deployable units should depend on each other. The Dependency Rule of the Clean Architecture applies the same logic to the system as a whole. Once a reader sees the common thread, the jargon becomes much less intimidating: each term names a particular way of controlling who depends on whom. How This Guide Is Organised The guide follows the rising scale of Martin's book, with a closing turn toward evaluation. It begins with the economic case: the difference between design and architecture, the two kinds of value software provides, and why structure is always important but rarely urgent. It then treats the three programming paradigms as a history of disciplines, each of which removed a dangerous freedom from programmers and, in doing so, created the tools architecture relies on. The middle of the guide covers the principles. The five SOLID principles are explained without code, through small everyday illustrations that preserve the reasoning. The component principles follow in two stages, first the rules for deciding what belongs together, then the rules for how components should depend on one another, including the numerical measures of stability and abstractness that Martin proposes, worked through with a concrete example. The guide then turns to architecture itself: what it means to keep options open, how boundaries are drawn, why business rules deserve their own protected core, and how the Clean Architecture arranges a system in layers. A further stretch deals with the practical questions that arise when the model meets real systems, such as partial boundaries, the role of the startup code, the place of microservices, testing and embedded software. Martin's argument that databases, the web and frameworks are details gets its own treatment, because it is the claim most likely to surprise newcomers. Finally, the guide steps outside the book. It sets Martin's model beside close relatives, notably Alistair Cockburn's hexagonal architecture and Jeffrey Palermo's onion architecture, considers the most serious criticisms of the approach, and examines documented cases from Netflix, Shopify and Amazon Prime Video that show both the value of clean boundaries and the limits of fashionable alternatives. Throughout, the text distinguishes Martin's own arguments from the guide's commentary, so that readers can tell what the book says from what others have said about it. Each chapter closes with key takeaways and review questions. Several tables compress comparisons that would otherwise sprawl across pages, and a glossary at the end collects the terms that recur most often. None of this requires the ability to program. It requires only the willingness to follow an argument about how the shape of a thing determines what can be done with it, which is, in the end, a management question as much as an engineering one. Chapter 1: Why Structure Is a Business Problem Most people who commission software judge it by what it does. Does the checkout accept payment? Does the report show the right totals? Does the app load quickly on a phone? These are reasonable questions, and they are the ones customers ask. Martin's book begins by insisting on a second question that customers never ask and that managers often forget: how hard will it be to change this system tomorrow? His answer to why that question matters is not aesthetic. It is economic, and it frames everything that follows. Design and Architecture Are the Same Thing Software people often talk as if "architecture" were the grand, high-level shape of a system and "design" were the fine-grained arrangement of its parts. Architecture, on this view, is what senior people draw on whiteboards; design is what junior people do in the code. Martin rejects the distinction. He compares a software system to a house. The architect of a house produces drawings that show the overall shape, the elevations and the placement of rooms, but the same drawings also show where every power outlet, light switch and water pipe goes. The low-level details and the high-level structure are part of one continuous description. You cannot understand the house by looking at only one layer of it. The same holds for software. Decisions about how a single class is organised feed into decisions about how modules relate, which feed into decisions about how services and deployable units are arranged. There is no clean line where design ends and architecture begins. For a manager, the practical consequence is that architecture cannot be delegated to a single meeting at the start of a project and then forgotten. It is the cumulative result of thousands of small decisions made by everyone who writes code, and it keeps evolving for as long as the system lives. The Goal of Architecture If design and architecture are one thing, what is that thing for? Martin gives a definition that sounds modest but has sharp edges: the goal of software architecture is to minimise the human resources required to build and maintain the required system. Architecture is measured by effort. When the effort needed to meet each new requirement stays low across the life of the system, the design is good. When the effort grows with every release, the design is bad, regardless of how elegant it looked at the start. This definition turns an abstract engineering topic into something a finance department can recognise. Human effort is the dominant cost in software. Servers and licences matter, but salaries usually dwarf them. An architecture that doubles the effort needed per feature doubles the largest line in the budget. The signature of a mess Martin illustrates the point with data from a company he does not name. Over a series of releases, the company's engineering staff grew steadily, from a small team to a much larger one. Over the same releases, the number of lines of code the organisation produced barely moved, and the cost of each line rose steeply. Engineers were working as hard as ever; indeed, they were working harder. But more and more of their effort went into fighting the existing code rather than adding new capability. By the later releases, the company was paying many times as much for each increment of functionality as it had at the start. The shape of those numbers is what matters, not the precise figures. Headcount climbs. Output flattens. Cost per unit of output rises. Martin calls this the signature of a mess. Managers see it as a sequence of apparently unrelated problems: schedules slip, estimates grow, new hires take longer to become productive, and fixes in one place cause failures in another. Each symptom has a local explanation. The common cause is structural, and it is invisible unless someone is looking for it. The hare and the tortoise Why do capable teams produce messes? Martin's explanation is overconfidence. Developers under deadline pressure tell themselves that they will ship now and clean up later. Later never arrives, because the next deadline comes, and the mess from the last round makes the next round slower, which increases the pressure to cut corners again. The cycle is self-reinforcing. He retells Aesop's fable of the hare and the tortoise to make the point. The hare is so confident of its speed that it rests; the steady tortoise wins. In software, the team that sprints without structure feels fast in the first weeks and then slows dramatically. The team that keeps its code clean feels slower at first and wins over any horizon longer than a few months. Martin's slogan for this is that the only way to go fast is to go well. He supports it with a small informal experiment in which the same programming exercise was completed repeatedly with and without disciplined test-first practice. The disciplined runs finished sooner. The experiment is anecdotal rather than scientific, and the guide treats it as an illustration, not proof, but the direction it points in matches what many engineering leaders report from experience. The managerial lesson is that a mess is not a technical problem that engineers have inflicted on the business. It is a business decision, usually made implicitly, to borrow time from the future at a very high rate of interest. Technical debt and the cost of change The borrowing metaphor has a history that predates Martin's book. Ward Cunningham introduced the phrase "technical debt" in a 1992 conference report, describing how shipping code that reflects an incomplete understanding is like taking out a loan: useful in the short run, but interest accrues until the code is reworked. Martin does not rely heavily on the term, but his argument is the same. Structure is the principal; the extra effort each change requires is the interest. It helps to picture the relationship between time and the cost of change as a curve, described here in words. In the classic account associated with Barry Boehm's work on software economics in the early 1980s, the cost of fixing a problem rises sharply the later in a project it is discovered: a mistake caught during requirements is cheap, one caught in production is very expensive. The curve climbs steeply to the right. In the late 1990s Kent Beck, writing about Extreme Programming, argued that disciplined practices such as continuous testing, simple design and relentless refactoring could flatten that curve, so that a change late in a system's life costs roughly what it would have cost early on. Martin's architecture is, in effect, a programme for flattening the curve. A system with good boundaries keeps each change local. A system without them makes every change ripple outward, so the cost of change grows with the size of the codebase. Students should be able to describe the two shapes: a curve that rises ever more steeply, where each year's changes cost more than the last year's, and a curve that stays close to horizontal, where the tenth year's change costs about what the first year's did. The gap between the two lines, summed over the life of a system, is the economic value of architecture. The Two Values of Software Martin next argues that every software system offers its owners two distinct kinds of value, and that organisations routinely mistake which is more important. The first value is behaviour. Software exists to make machines do something that saves or earns money for its users. Programmers are hired to write code that behaves as the requirements say, and to fix it when it does not. Many programmers, Martin observes, believe that this is the whole of their job. The second value is structure, or what Martin also calls architecture. The word "software" is built from "soft" and "ware", and the point of software, as opposed to hardware, is that it should be easy to change. When stakeholders change their minds about a feature, the change ought to be simple to make, and its difficulty ought to be proportional to the scope of the change, not to the shape the system happens to be in. When a small change requires a huge effort because it collides with an awkward structure, the software has lost the property that justifies its name. Which value is greater Business stakeholders, if asked, will almost always say that working behaviour matters more. Martin disagrees, and his argument is worth stating carefully, because it is the kind of reasoning examiners like to test. Consider two extreme programs. The first works perfectly today but is impossible to change. The second does not work at all but is easy to change. Which is worth more? The first program will become useless as soon as requirements shift, and requirements always shift. Nobody can adapt it, so its value decays to zero. The second program can be made to work, because making it work is simply another change, and it can keep working as requirements evolve. The easy-to-change program is therefore the more valuable one. Martin concedes that a truly unchangeable program is rare, but he notes that many systems become so expensive to change that the cost of a change exceeds the benefit it would bring, which amounts to the same thing in practice. The argument does not say that behaviour is unimportant. It says that behaviour is fragile without structure, and that an organisation which trades structure away for short-term behaviour is consuming the thing that lets it keep delivering behaviour. The Eisenhower matrix To explain why organisations still get the priority wrong, Martin borrows a device associated with President Dwight D. Eisenhower, who is reputed to have said that he had two kinds of problems, the urgent and the important, and that the urgent ones were rarely important and the important ones rarely urgent. Management trainers turned this into a two-by-two grid. Picture two questions asked of every task: is it important, and is it urgent? The answers produce four quadrants, which are conventionally ranked like this: 1. Urgent and important. 1. Important but not urgent. 2. Urgent but not important. 3. Neither urgent nor important. Martin places the two values of software onto this grid. Behaviour is usually urgent, because a customer or a deadline is waiting, but it is not always important in the long run. Architecture is important but almost never urgent, because no customer is shouting for a cleaner dependency structure this week. So behaviour sits in the first and third quadrants, and architecture sits in the first and second. The classic failure is to treat everything in the third quadrant, urgent but unimportant, as if it belonged in the first. A feature request arrives with a deadline attached, and because it is loud, it displaces the quieter work of the second quadrant. Repeat this for a year and the second quadrant is never visited. The architecture decays, and the signature of a mess appears. Martin's conclusion is that business managers are often not equipped to judge the importance of architecture, because it is invisible to them. That is precisely why software developers are hired: part of their professional responsibility is to assert the importance of structure over the urgency of features. He describes this as a struggle, and he means it literally. The development team is one of the stakeholders of the system, and it must fight for the long-term health of what it builds, just as sales fights for features and finance fights for cost control. If the architecture comes last, the system will become ever more costly to develop, and eventually change will become practically impossible. When that happens, Martin says, the development team did not fight hard enough. What This Means for Managers The first two ideas of the book translate into a small set of managerial habits. The guide draws them out here because they recur in every later chapter. The first habit is to treat changeability as a requirement. When a project is scoped, the question "how will we change this later?" deserves the same seriousness as "what will this do?". A system that meets every stated feature but hard-wires a particular database, vendor or user interface into its core has quietly failed a requirement nobody wrote down. The second habit is to watch the right indicators. Rising headcount paired with flat throughput, lengthening estimates for changes of similar size, and frequent regressions in unrelated areas all suggest structural decay. Several engineering metrics used in industry, such as the lead time from an idea to production and the rate at which changes fail, capture some of this, though none measures structure directly. The third habit is to budget for the second quadrant. Important but non-urgent work does not happen by accident. Teams that keep their systems healthy usually protect time for it explicitly, whether as a share of each iteration, as refactoring folded into every feature, or as dedicated improvement work tied to a business goal. The fourth habit is to take engineers seriously when they argue for structure. Martin's point about the development team as a stakeholder cuts both ways. Engineers must make the case in terms the business understands, such as cost, speed and risk, rather than in terms of purity or taste. Managers, in turn, must accept that an argument about structure is an argument about money. The guide adds one qualification. Not every system deserves heavy investment in structure. A prototype meant to test a market hypothesis and then be thrown away, or a script that will run once, has a short life, and the cost-of-change curve never has time to bend. Martin's argument has most force for systems expected to live and grow for years, which, in practice, is most systems that reach production, including many that were described as prototypes when they were built. Who owns the architecture A final question follows from Martin's argument: if architecture emerges from everyone's decisions, who is responsible for it? Martin's answer has two parts. Architects must remain programmers, close enough to the code to feel the effects of their decisions. And every developer carries part of the responsibility, because every change either respects the structure or erodes it. The guide adds the managerial corollary. Accountability for structure cannot be assigned to a single role and then forgotten. It has to be built into how work is planned, reviewed and measured, so that the long-term health of the system is someone's explicit concern at every level, from the individual pull request to the portfolio budget. Key Takeaways · Martin treats design and architecture as one continuous activity; high-level shape and low-level detail cannot be separated. · The goal of architecture is economic: to minimise the human effort needed to build and maintain a system over its life. · The signature of a mess is growing headcount, flat output and rising cost per change; it arises from repeatedly trading structure for speed. · Software has two values, behaviour and structure; Martin argues structure is the greater, because an easy-to-change system can always be made to work. · On the Eisenhower matrix, architecture is important but rarely urgent, so it is chronically neglected unless developers and managers deliberately defend it. Review Questions 1. Explain Martin's reason for refusing to distinguish between design and architecture, and state one consequence for how architecture is managed. 1. Describe the pattern Martin calls the signature of a mess and list three symptoms a manager might observe. 2. Reconstruct Martin's argument that a program that does not work but is easy to change is more valuable than one that works but cannot be changed. What objection could be raised? 3. Using the Eisenhower matrix, explain why architectural work is routinely displaced by feature work. 4. In words, contrast a steep cost-of-change curve with a flat one, and explain how architecture relates to the difference. Chapter 2: Paradigms as Disciplines Before Martin discusses principles or architecture, he takes a detour through history. He asks what the major programming paradigms actually contributed to the craft. His answer is counterintuitive and, once grasped, illuminating: each paradigm took something away from programmers. None of them added a new capability that machines lacked. Each imposed a discipline, a restriction on what programmers are allowed to do, and those restrictions turn out to be the raw materials of architecture. A paradigm, in Martin's usage, is a way of programming that tells you which structures to use and when. He identifies three: structured programming, object-oriented programming and functional programming. He adds the provocative claim that there are probably no more to come, because each of the three removes a fundamental freedom and there are few freedoms left to remove. Whether that claim holds is debatable, but the framework is useful for anyone trying to understand why software engineers argue so fiercely about seemingly small matters of style. Structured Programming: Discipline on Direct Transfer of Control In the early decades of computing, programs routinely used an instruction usually called "goto", which told the machine to jump unconditionally to some other point in the program. Jumps made it possible to write anything, but they also made it possible to write programs whose flow was impossible to follow, with control leaping backward and forward across hundreds of lines. Edsger Dijkstra, the Dutch computer scientist, saw this as a barrier to reasoning. He wanted programmers to be able to prove, in something like the way mathematicians prove theorems, that their programs were correct. He found that unrestrained jumps made such proofs impractical, whereas programs built from a small set of control structures could be decomposed into pieces small enough to reason about. His short 1968 letter to the Communications of the ACM, published under the famous title "Go To Statement Considered Harmful", set off a long controversy. Earlier theoretical work by Corrado Böhm and Giuseppe Jacopini had already shown that any program can be written using just three structures: sequence (do this, then that), selection (if this, do that, otherwise the other) and iteration (repeat this while a condition holds). Over the following years the programming world adopted this discipline, and modern languages either lack an unrestricted jump or discourage it. Proof gave way to testing Martin makes an interesting observation about how the story ended. Dijkstra's dream of mathematical proof never became everyday practice; formal proofs of ordinary programs proved too laborious. What survived was a different idea borrowed from science. A scientific theory cannot be proved true, but it can be shown to be false by a single counterexample. Theories that resist repeated attempts at falsification are accepted as good enough to rely on. Dijkstra himself had remarked that testing shows the presence of bugs, not their absence. Software, Martin argues, works the same way. We cannot prove a program correct, but we can write tests that try to show it is incorrect, and a program that survives thorough testing is accepted as correct enough. This only works if the program can be broken into small units that can each be tested. Structured programming makes that decomposition possible. It is therefore, in Martin's view, the foundation of functional decomposition, and the reason architects still care about breaking systems into small, provable, testable parts. The managerial reading For a non-programmer, the lesson is that constraints enable verification. A team that restricts itself to a predictable set of forms can check its work. A team with unlimited freedom produces work that nobody, including its authors, can confidently assess. The same principle appears in finance, where standard reporting formats allow audit, and in manufacturing, where standard parts allow inspection. Object-Oriented Programming: Discipline on Indirect Transfer of Control Object-oriented programming is usually explained with three words: encapsulation, inheritance and polymorphism. Martin examines each and concludes that only the third is a genuine contribution to architecture. Not a model of the real world Martin first sets aside a popular definition. Object orientation is often described as a way of modelling the real world, with software objects standing for things such as customers, invoices and vehicles. Martin regards this as too vague to be useful. Plenty of good object-oriented code has no obvious real-world counterpart, and plenty of real-world modelling can be done without objects. He looks instead for something more precise: a mechanism that object-oriented languages provide and that earlier approaches did not provide safely. Encapsulation and inheritance were not new Encapsulation means drawing a line around data and the functions that operate on it, so that outsiders see only what they need. Martin points out that older languages such as C already supported a strong form of this: a programmer could publish a list of available functions while keeping the underlying data entirely hidden. Some later object-oriented languages actually weakened this separation, because their conventions required details of a class's internal data to be visible to other parts of the program even when they were not meant to be used. Inheritance means defining a new kind of thing as an extension of an existing kind. Martin shows that careful programmers achieved much the same effect before object-oriented languages existed, by arranging data so that one structure could be treated as a variant of another. Object-oriented languages made this more convenient and safer, but they did not invent it. Polymorphism is the real contribution Polymorphism, from the Greek for "many forms", is the ability of a single instruction to produce different behaviour depending on what kind of thing receives it. An everyday example helps. A library issues the instruction "return this item". A printed book goes to a shelf, a laptop goes to a charging cabinet, and an e-book simply expires. The person issuing the instruction does not need to know which kind of item is involved. Each item knows how to handle "return" in its own way. Programmers had achieved polymorphism before, by hand, using tables of pointers to functions. Operating systems, for instance, allowed a program to write output without knowing whether the output was going to a screen, a printer or a file. But hand-built polymorphism was fiddly and dangerous, because a single mistake in the table could cause baffling failures. Object-oriented languages, beginning with Simula, designed by Ole-Johan Dahl and Kristen Nygaard in the 1960s, made polymorphism safe and routine. That, Martin says, is the discipline: object orientation imposes order on indirect transfers of control, just as structured programming imposed order on direct ones. Dependency inversion: the architectural payoff Why does this matter so much? Because polymorphism lets programmers reverse the direction of dependencies in source code. In a traditional program, high-level functions call lower-level functions, and to do so they must name them. The high-level code therefore depends on the low-level code: if the low-level code changes, the high-level code may have to be recompiled, retested and redeployed. The direction in which control flows at runtime and the direction of source-code dependency are the same, and both run downward, from policy toward detail. With polymorphism, the high-level code can instead depend on an abstract description of a service, often called an interface, which says what operations exist without saying how they are performed. The low-level code then promises to fulfil that description. At runtime, control still flows from high level to low level. But in the source code, the arrow now points the other way: the low-level module depends on the interface, which belongs with the high-level module. This reversal is called dependency inversion. Martin regards this as the single most important capability object-oriented programming offers. It gives architects absolute control over the direction of every source-code dependency in a system. That control makes two things possible. First, a system can be arranged so that business rules do not depend on the user interface or the database; instead, the user interface and the database depend on the business rules. Second, the parts can be compiled and deployed separately. A change to the database code need not force the business rules to be rebuilt. Martin calls the resulting style a plugin architecture: details plug into the core, much as a peripheral plugs into a computer through a standard port. For a manager, dependency inversion is the technical basis of vendor independence and parallel development. If the core business logic defines the port and the suppliers build adaptors to fit it, then suppliers can be replaced, and separate teams can work on the core and on the adaptors without constantly blocking one another. Functional Programming: Discipline on Assignment The third paradigm is, in terms of its ideas, the oldest. It grows out of the lambda calculus developed by Alonzo Church in the 1930s, and one of its best-known early languages, LISP, was created by John McCarthy in the late 1950s. Its defining discipline, in Martin's account, is a restriction on assignment. In a purely functional style, once a value has been given to a variable, it is never changed. Programs compute new values from old ones rather than altering existing values in place. Why immutability matters to architects The practical consequence concerns concurrency, meaning situations where several parts of a program, or several programs, run at the same time. Martin observes that nearly all the hardest bugs in concurrent systems, such as race conditions, deadlocks and inconsistent updates, arise because two activities try to change the same data at once. If data is never changed, those problems cannot arise. Immutability removes a whole family of failure modes at a stroke. Complete immutability is rarely practical, because real systems must record that things happen. Martin's architectural advice is therefore to segregate. Divide a system into components that never change data and a small number of components that do. Push as much logic as possible into the immutable components, and protect the mutable ones carefully with mechanisms designed to manage concurrent updates. Event sourcing He also describes a technique that shows how far the idea can go. Imagine a bank that, instead of storing each customer's current balance and overwriting it after every transaction, stores only the transactions themselves. The balance is never saved; it is calculated whenever needed by adding up the history. Nothing is ever updated or deleted, only appended. This approach is known as event sourcing. It trades storage and computation for a complete, trustworthy record and freedom from update conflicts. Martin notes that source-code version control systems work in essentially this way, and that as storage and processing become cheaper, the approach becomes more practical for business systems. A manager will recognise the accounting parallel. A general ledger records entries; balances are derived. The history is the source of truth, and corrections are made by adding new entries rather than erasing old ones. That design, centuries old in accounting, is exactly what functional thinking recommends for software. Paradigms in Today's Languages Martin's framing is historical, but it applies directly to the languages organisations use now. Most mainstream languages are no longer pure examples of any one paradigm. Java, first released in the mid-1990s as a thoroughly object-oriented language, added lambda expressions, a functional feature, in 2014. C#, JavaScript, Python, Kotlin and Swift all mix structured control flow, objects and functional techniques. Several newer languages make immutability the easy default: Kotlin and Swift, for example, distinguish between values that can be reassigned and values that cannot, and nudge programmers toward the latter. Popular user-interface libraries on the web encourage developers to treat application state as immutable and to produce new state rather than altering the old. The practical consequence is that the disciplines Martin describes are rarely enforced by the language alone. A modern language usually allows a programmer to write tangled control flow, to bypass polymorphism with long chains of type checks, and to change shared data freely. Whether a team works within the disciplines depends on its conventions, code review, automated checks and the example set by senior engineers. For managers, this shifts attention from language selection to engineering practice. Choosing a fashionable language does not, by itself, confer the benefits of any paradigm. What confers them is a team that understands which freedoms to give up and why. Coding standards, review checklists and static analysis tools are, in Martin's terms, instruments for imposing discipline, and their value lies in the classes of error they prevent. When a team proposes adopting a new language or library, a useful question is which mistakes it makes harder to commit, and whether those are the mistakes that have actually been costing the organisation money. There is also a staffing implication. Because the disciplines are portable across languages, engineers who understand them can move between technologies with relatively little loss of effectiveness. Engineers who know only the surface features of one framework cannot. Martin's argument that the fundamentals of software have changed little over half a century supports investing in those fundamentals when hiring and training. What the Three Paradigms Teach Together Martin draws the threads together in a way that serves the rest of the book. The three paradigms map onto the three great concerns of architecture. Structured programming underpins the algorithmic decomposition of a system into small, testable units. Object-oriented programming, through polymorphism, provides the means of controlling dependencies across boundaries. Functional programming provides a discipline for managing where and how data changes. Table 1 sets the three side by side. Table 1. The three paradigms as disciplines. Paradigm Freedom removed What it enables Architectural concern Structured Unrestricted jumps Decomposition into testable units Function and testing Object-oriented Unsafe indirect calls Safe polymorphism, dependency inversion Separation across boundaries Functional Changing values in place Freedom from concurrency conflicts Control of data and state Martin also makes a broader, almost philosophical point. In roughly half a century since these paradigms appeared, the basic materials of software have changed remarkably little. Programs are still made of sequence, selection, iteration and indirection, applied with discipline. Hardware has become enormously faster and cheaper, and languages and tools have improved, but the rules that decide whether a program is well structured are much the same. This continuity is why Martin believes that architectural principles can be stated in general terms and expected to last. The guide adds a note of caution. Martin's claim that no further paradigms remain is a prediction, not a finding, and many practitioners would point to logic programming, dataflow and reactive styles as candidates he sets aside. His narrower point, that each paradigm's value lies in a discipline rather than a new power, is the part worth keeping. It shifts the conversation about technology choices away from "what can this language do?" toward "what mistakes does this approach prevent?", which is usually the more useful question for an organisation. Key Takeaways · Martin presents structured, object-oriented and functional programming as disciplines that each remove a freedom rather than add a capability. · Structured programming restricts jumps, enabling decomposition into small units that can be tested; testing, like science, shows the presence of defects, not their absence. · Object orientation's decisive contribution is safe polymorphism, which lets architects invert source-code dependencies so that details depend on policy. · Dependency inversion underpins plugin architectures, independent deployment and freedom from particular vendors. · Functional programming restricts assignment; segregating immutable from mutable components, and techniques such as event sourcing, remove whole classes of concurrency failures. Review Questions 1. Explain what Martin means by calling each paradigm a discipline. Why does he think this is a more accurate description than calling them new capabilities? 1. How did the idea of scientific falsification come to replace mathematical proof in Martin's account of structured programming? 2. Using a non-technical example, explain polymorphism and describe how it allows the direction of a source-code dependency to be reversed. 3. Why does Martin regard dependency inversion as the most important contribution of object orientation to architecture? Give one business benefit. 4. Describe event sourcing and explain its connection to both immutability and traditional bookkeeping. Chapter 3: The SOLID Principles If programming paradigms are the bricks of software, Martin says, then the SOLID principles tell us how to arrange the bricks into walls and rooms. They operate at what he calls the mid-level of software structure: the level of modules and classes, where functions and the data they use are grouped together. The principles emerged in Martin's writing and in discussions among software practitioners from the late 1980s onward, and they acquired their acronym in the early 2000s, when Michael Feathers pointed out that the initial letters of the five principles could be rearranged to spell SOLID. Martin states the purpose of the principles plainly. They aim to create structures that tolerate change, are easy to understand, and can serve as the basis of components used in many systems. Each principle is a rule for managing change: who causes it, where it lands, and how far it spreads. This chapter explains each principle without any program code, using small illustrations from ordinary organisational life. The reasoning is preserved; the syntax is not needed. S: The Single Responsibility Principle The Single Responsibility Principle is the most misunderstood of the five, and Martin devotes care to correcting the misunderstanding. Many programmers take it to mean that a module should do only one thing. That is a sound rule for individual functions, but it is not what the principle says. Martin's final formulation is that a module should be responsible to one, and only one, actor. An actor here is a group of people, one or more users or stakeholders, who want the system changed for the same reasons. The principle is therefore about people and organisations, not about technical tasks. It asks: whose requirements drive changes to this piece of code? If the answer is more than one group, the code is at risk. The payroll illustration Martin's example concerns an employee record in a business system that offers three operations: calculating pay, reporting hours worked, and saving the record to the database. Each operation serves a different part of the organisation. Pay calculation is specified by the finance function, reporting hours by the operations function, and saving by the technical function that manages the database. Suppose the calculations for pay and for hours reporting both rely on a shared routine that computes regular, non-overtime hours. Now the finance team asks for a change in how regular hours are counted for pay purposes. A developer makes the change in the shared routine, the tests for payroll pass, and the change is deployed. But the operations team's hours report also used that routine, and nobody told them. Their figures are now wrong, and the error may go unnoticed for months while decisions are made on bad numbers. Martin calls this accidental duplication. Two groups appeared to need the same thing, so the code served both from one place. In truth their needs were only coincidentally similar and were free to diverge. By tying them together, the design allowed one actor's change to damage another actor's results. A second symptom is merge conflict. If the finance team and the technical team both request changes to the same module at the same time, two developers edit the same file independently, and their changes must be reconciled. Reconciliation is error-prone and puts both groups at risk. The remedy The cure is to separate code that different actors depend on. Martin describes several ways to do this, including keeping the shared data in a simple structure and giving each actor's logic its own module, with a thin front door, sometimes called a facade, that directs requests to the right place. The details matter less than the principle: each actor's rules live in their own place, so that changes requested by one group cannot quietly break another group's work. The organisational reading Students of management will recognise the logic. A job description that reports to three bosses creates conflicts; a process owned by three departments gets changed inconsistently. The Single Responsibility Principle is the software version of clear accountability. It also has a striking connection to Conway's law, the observation by Melvin Conway in 1968 that systems tend to mirror the communication structures of the organisations that build them. Martin's principle turns that observation into a design rule: align module boundaries with the boundaries between the groups who request changes. O: The Open-Closed Principle The Open-Closed Principle was formulated by Bertrand Meyer in his 1988 book on object-oriented software construction. It says that a software artefact should be open for extension but closed for modification. In plain terms, you should be able to change what a system does by adding new code rather than by altering existing code. Martin considers this the most fundamental reason we study software architecture at all. If simple extensions to requirements force massive changes to existing code, the architects have failed. The financial report illustration His illustration is a system that displays a financial summary on a web page, with negative numbers shown in red. The stakeholders then ask for the same information as a printed report, paginated, with proper headers and with negative numbers shown in brackets. How much existing code should have to change to add the printed version? In a well-designed system, very little. Martin describes separating the work into distinct responsibilities: one part calculates the figures, another prepares them for presentation, and separate parts render them for the web and for print. Crucially, he arranges the dependencies so that they all point toward the part that calculates the figures. The calculation, which embodies the business rules, depends on nothing that could change because of a presentation decision. The web and print renderers depend on the calculation, not the other way round. The result is a hierarchy of protection. The business rules sit at the top, protected from changes in everything else. Presentation logic depends on them. The specific renderers depend on the presentation logic. Adding a new output format, such as a spreadsheet export or a mobile view, means adding a new renderer. Nothing above it changes. The managerial reading The Open-Closed Principle is the software equivalent of a product platform. A car manufacturer that designs a common chassis can add new body styles without redesigning the drivetrain. A bank that defines a stable interface for payment providers can add a new provider without rewriting its ledger. The strategic value is that growth in variety does not produce growth in risk to the core. Martin adds a subtle point about direction. The principle is achieved by arranging components so that higher-level ones, which represent more important policy, are protected from changes in lower-level ones. This idea returns later as the organising logic of the whole Clean Architecture. L: The Liskov Substitution Principle The Liskov Substitution Principle takes its name from Barbara Liskov, the computer scientist who stated a version of it in a 1988 paper on data abstraction and hierarchy. In informal terms, it says that if a system is written to work with a certain kind of thing, it should keep working correctly when given any subtype of that thing. Substitutes must honour the contract of what they replace. The square and the rectangle Martin uses a classic puzzle. In geometry, a square is a special kind of rectangle. It seems natural, therefore, to define a square in software as a kind of rectangle. But a rectangle in software typically allows its width and height to be set independently. A square cannot allow that; changing one side must change the other. Now imagine a piece of code that takes a rectangle, sets its width to five and its height to two, and expects the area to be ten. Give it a square instead and the area comes out wrong, because setting the height also changed the width. The square is not a valid substitute for the rectangle, whatever geometry says. The code using it would have to check which kind of shape it had received, and every such check is a place where the design has broken down. The taxi dispatch illustration Martin's more architectural example concerns a taxi aggregation service that dispatches rides to many independent taxi companies. Each company exposes a web address through which the aggregator can send a booking, and all companies are supposed to follow the same format, so the aggregator can treat them interchangeably. Then one large company, in Martin's hypothetical, implements the format slightly differently, abbreviating one of the fields. The aggregator must now add a special case: if the company is this one, use the abbreviation. Every such exception clutters the core dispatch logic, invites errors, and creates a security and maintenance burden. Martin suggests that, if exceptions cannot be avoided, they should be handled through a configuration mechanism kept away from the core, but the real lesson is that a violation of substitutability pollutes the architecture. The managerial reading The Liskov principle is about honest contracts. A supplier who claims to conform to an industry standard but deviates in small ways imposes a hidden cost on every customer, who must write special handling. Procurement teams deal with this constantly: two parts with the same specification should be interchangeable, and if they are not, the specification is lying. In software, the principle has expanded from its origins in class hierarchies to cover any interface, including web services and data feeds. Wherever a system promises that implementations are interchangeable, they must truly be so. I: The Interface Segregation Principle The Interface Segregation Principle says that no module should be forced to depend on things it does not use. The operations illustration Martin describes a single module, which he calls an operations class, that offers three operations used by three different users. Each user needs only one of the operations. But because all three are bundled together, each user depends, in the source code, on all three. When the operation used only by the first user changes, the other two users may have to be rebuilt and redeployed, even though nothing they use has changed. In languages that require recompilation, this is directly costly; in others, it is a risk and a source of confusion. The remedy is to give each user a narrow interface containing only what it needs. The underlying module can still provide all three operations, but each user sees, and depends on, just its own slice. Beyond classes Martin generalises the principle to architecture. Depending on something that carries baggage you do not need can cause trouble you did not expect. He imagines a system that uses a framework, which in turn is tied to a particular database. The system needs none of the database's features, yet a change in the database can force the framework to be redeployed, which forces the system to be redeployed. Worse, a failure in the unused database feature can break the system. The managerial reading The organisational analogy is the all-in-one contract or the monolithic software suite. An organisation that buys a vast platform to use a single feature inherits every upgrade, outage and licence change affecting features it never touches. Narrow, purpose-specific dependencies reduce exposure. The guide notes that this is also an argument for small, well-defined service agreements between internal teams: a team should commit to what its consumers actually need, not expose its entire internal surface. D: The Dependency Inversion Principle The final principle generalises the mechanism introduced in the previous chapter. The most flexible systems, Martin says, are those in which source-code dependencies refer only to abstractions, not to concrete implementations. Stable abstractions, volatile details Taken literally, this would be impossible; every system must eventually use concrete things. Martin therefore refines the rule. It is not a problem to depend on concrete things that are stable, meaning they rarely change. The standard text-handling facilities of a programming language, for example, are concrete but change so rarely that depending on them is harmless. The danger lies in depending on volatile concrete things: the modules the team is actively developing and that will change frequently. Abstractions, by contrast, tend to be stable. An interface that describes what a payment gateway must do changes far less often than any particular gateway's implementation. So the principle becomes practical guidance with four parts, which Martin states roughly as follows. Refer to volatile things through abstract interfaces rather than directly. Do not build new modules by extending volatile concrete ones. Do not override concrete behaviour that other code relies on. And never mention the name of anything concrete and volatile. The last rule raises a practical question: if high-level code may not name a concrete class, how does it ever create one? Martin's answer is a pattern called an abstract factory. The high-level code asks an abstract factory to produce the object it needs. The concrete factory, which does know the specific class, lives on the low-level side of the boundary. Martin describes the boundary between the two sides as a curved line separating the abstract from the concrete. All source-code dependencies cross that line pointing toward the abstract side. Control, at runtime, flows the other way. That is the inversion. Some violation is unavoidable. Somewhere, a concrete object must be created and wired to the abstractions. Martin says such violations should be gathered into a small number of concrete components, typically the main component that starts the system, which is discussed in a later chapter. An office illustration Consider a head office that needs documents printed. If its procedures say "send the file to the Model X printer on the third floor", every printer upgrade requires rewriting procedures. If its procedures say "send the file to the print service", and the facilities department is responsible for making some printer satisfy that service, then printers can change freely. The head office depends on an abstraction it defined; facilities depends on the same abstraction. The dependency of policy on equipment has been inverted. The managerial reading Dependency inversion is the principle that most directly protects strategic choices. It is how an organisation keeps the option to switch cloud providers, payment processors, messaging tools or databases without rewriting its core logic. It also clarifies who owns an interface. In an inverted design, the interface belongs to the high-level policy that uses it, not to the supplier that implements it. That ownership reflects bargaining power: the business defines what it needs, and suppliers adapt. The Principles Side by Side The five principles work together. The Single Responsibility Principle tells us where to cut. The Open-Closed Principle tells us to arrange the pieces so that extension does not disturb the core. The Liskov principle ensures that the pieces we plug in keep their promises. Interface segregation keeps each dependency narrow. Dependency inversion ensures that dependencies point toward stable policy rather than volatile detail. Table 2 condenses them for revision. Table 2. The SOLID principles for managers. Principle Core rule Everyday analogue Warning sign Single Responsibility Answer to one actor One boss per role One team's change breaks another's report Open-Closed Extend without modifying Common product platform Every new variant edits the core Liskov Substitution Substitutes honour the contract Truly interchangeable parts Special cases for particular suppliers Interface Segregation Depend only on what you use Narrow service agreements Redeploying for changes you do not use Dependency Inversion Depend on stable abstractions "Print service", not "Model X" Business logic names specific technologies A final point is worth making about the status of these principles. Martin presents them as principles, not laws. Applied mechanically, each can produce clutter: interfaces with a single implementation that will never have a second, or modules split so finely that nobody can find anything. Martin's own framing ties each principle to a specific kind of change. Where that kind of change is plausible, the principle earns its cost. Where it is not, the principle can be relaxed. This theme of judgement, rather than rule-following, returns in the critical discussion toward the end of this guide. Key Takeaways · The SOLID principles govern the mid-level arrangement of modules and classes, and each is a rule for controlling the impact of change. · The Single Responsibility Principle means a module should answer to one actor, a group of people who request changes for the same reasons; it is not "do one thing". · The Open-Closed Principle protects high-level policy by arranging dependencies so that new behaviour is added rather than existing code edited. · The Liskov and Interface Segregation principles demand honest, narrow contracts, so that substitutes behave as promised and nothing depends on what it does not use. · The Dependency Inversion Principle directs dependencies toward stable abstractions owned by business policy, which is the technical basis of vendor and technology independence. Review Questions 1. Why does Martin reject the reading of the Single Responsibility Principle as "a module should do only one thing"? Illustrate accidental duplication with an example of your own. 1. Explain how the Open-Closed Principle creates a hierarchy of protection, using the financial report illustration. 2. Why is a square not a valid substitute for a rectangle in software, and what broader lesson does the taxi dispatch example teach? 3. Give an organisational example of a violation of the Interface Segregation Principle and describe its costs. Distinguish stable from volatile concrete dependencies. Why does Martin allow the former but not the latter, and where should unavoidable violations be collected? Hashtags: #TheBlueprintOfCode #CleanArchitecture #RobertCMartin #SoftwareArchitecture #SoftwareDesign #ArchitectureEconomics #CostOfChange #TechnicalDebt #DependencyManagement #DependencyRule #BusinessRules #SeparationOfConcerns #SOLIDPrinciples #SingleResponsibilityPrinciple #OpenClosedPrinciple #LiskovSubstitutionPrinciple #InterfaceSegregationPrinciple #DependencyInversionPrinciple #Polymorphism #PluginArchitecture #StructuredProgramming #ObjectOrientedProgramming #FunctionalProgramming #EventSourcing #FutureOfSoftwareArchitecture

Download the Book (PDF): Introduction On 23 August 2026, in an exhibition hall at Porte de Versailles on the southern edge of Paris, a French Trackmania driver named Gwendal Duparc won a race that was worth far more than the race itself. His victory settled the Club Championship of the Esports World Cup, a seven-week festival of twenty-five competitions across two dozen video games, and it handed his organisation, the Chinese club AG.AL, a first prize of $7 million. The club that finished second, Team Falcons, had won the same title in each of the two previous years. The whole event carried a prize pool of $75 million, the largest in the history of competitive gaming. It was funded by Saudi Arabia's Public Investment Fund, and it was meant to be held in Riyadh. It moved to Paris only in May, after war with Iran made the Gulf an unsafe place to fly two thousand competitors and their audiences. Almost everything about that afternoon illustrates how the business of e-sports actually works in the middle of the 2020s. The money came from a sovereign wealth fund rather than from ticket buyers or broadcasters. The prize was paid not to a player but to a club, and the club was rewarded for breadth, for fielding teams across many unrelated games, rather than for excellence in one. The games themselves belonged to other companies: Trackmania to Ubisoft, League of Legends to Riot Games, Counter-Strike to Valve, and so on through the list. None of those publishers could be told what to do by the organiser, and each had licensed its game to the event on its own terms. And the venue could be moved across a continent in a matter of weeks, because nothing about the competition was tied to a place in the way that a football club is tied to a city. This booklet is about the commercial machinery behind scenes like that one: how sponsorships are sold, how players are hired and paid, how leagues are organised and why so many of them have been reorganised, who has invested in teams and what they thought they were buying, how the broadcasts are distributed, and how the sale of virtual items inside games has become the single most important source of money that professional competition can draw on. It is written for intelligent readers who are not specialists: investors wondering whether e-sports is a real asset class, marketers deciding whether to sponsor a team, lawyers and agents drafting player contracts, students of sport business, and fans who want to understand why the teams they follow keep changing leagues, owners and names. The argument The booklet has one argument, and every chapter serves it. In e-sports, the publisher owns the field. That sentence sounds like a technicality. It is not. In traditional sport, nobody owns football. The rules of association football belong in practice to a body, the International Football Association Board, but no company can switch the game off, change its physics overnight, or decide that a particular tournament may not use it. A club that buys a stadium, signs players and builds a fan base owns something durable. Its league may be governed badly, but the underlying sport is a public good. Competitive video games are private property. League of Legends is a product owned by Riot Games, which is itself owned by the Chinese technology group Tencent. Counter-Strike 2 is owned by Valve. Overwatch and Call of Duty belong to Activision Blizzard, which Microsoft bought in 2023. Every tournament, every broadcast of every tournament, and every professional contract in those games exists because the owner permits it. The owner can change the game with a patch, license or refuse to license a tournament, ban a player, cap the prize money of community events, or stop supporting the game altogether. When the owner decides to reorganise its competitive ecosystem, everyone who invested in the old arrangement has to live with the new one. From that single structural fact follows most of what is distinctive about e-sports as a business. Franchise fees that looked like the price of a permanent asset turned out to be the price of a licence that the licensor could redraw. Sponsorship deals depend on audiences that the publisher, not the team, largely controls. Streaming rights are sold by the publisher, while the teams that provide the talent often receive nothing directly from them. Player contracts are shaped by publisher rules on salaries, transfers and conduct. Investors who valued teams as if they were young versions of the Dallas Cowboys discovered that a team's franchise could lose most of its value when a publisher changed course. And the virtual items that now fund much of the ecosystem, from weapon skins to team-branded bundles, are sold by publishers who decide how much of the revenue, if any, flows back to the competitors. The thesis of this booklet is that the first great wave of e-sports investment, roughly from 2017 to 2022, mispriced that dependency, and that the arrangements surviving in 2026 are those that face it squarely. Three kinds of arrangement have endured. The first plugs teams directly into the publisher's own revenue from digital goods, so that the publisher's interest in selling items becomes the team's interest too. The second draws on patient capital from a sovereign investor, above all Saudi Arabia, for whom e-sports serves national purposes beyond profit. The third builds on audiences of creators and fans whom the publisher cannot easily replace, so that a team or personality has bargaining power of its own. Organisations that relied on none of these, and that tried to behave as independent sports franchises with a durable asset, have mostly shrunk, merged, or disappeared. The state of play in 2026 The past three years have tested that thesis hard. The Overwatch League, the most ambitious attempt to transplant North American franchise sport into gaming, was folded in January 2024, six years after teams paid $20 million each to join. Riot Games merged its North American and Brazilian League of Legends leagues into a single League of the Americas for 2025, then reversed the decision within a year and restored both the LCS and Brazil's CBLOL for 2026. In April 2026 Riot announced that its Valorant Champions Tour, whose partnership leagues had begun only in 2023, would move to open regional qualifiers from 2027, with partner teams receiving support payments but no guaranteed places. Valve had already banned exclusive partnerships between teams and tournament organisers in Counter-Strike from 2025. The closed, permanent league is in retreat almost everywhere except Call of Duty. Meanwhile the money has shifted. Riot reported sharing more than $100 million with its Valorant partner teams in 2025, most of it from sales of in-game items. Saudi Arabia's gaming group Savvy owns ESL FACEIT Group, the largest independent tournament operator, agreed in March 2026 to buy the Mobile Legends publisher Moonton for $6 billion, and sits within a state fund that completed the $55 billion buyout of Electronic Arts in August 2026. Riot opened its leagues in the Americas and Europe to betting sponsors in 2025, reversing a long-standing refusal. And the market for Counter-Strike weapon skins, often described as the largest secondary market for virtual items in the world, lost billions of dollars of notional value in a single day in October 2025 because Valve changed one crafting rule. Each of those developments is a variation on the same theme. Where a publisher changed its mind, value moved. How the booklet is organised Chapter 1 sets out the landlord problem in detail: what it means for a publisher to own a sport, how licensing works, and how publishers have used their power in cases from Blizzard's punishment of a Hong Kong player in 2019 to Nintendo's refusal to license the largest grassroots Smash Bros. circuit in 2022. Chapter 2 examines the franchise bet: what investors paid to join the Overwatch League, the LCS, the LEC and the Call of Duty League, what they thought they were buying, and why the numbers did not work. Chapter 3 follows the models that replaced pure franchising, from Riot's revenue-sharing pools to Valve's open circuit, and asks what a team now owns. The middle of the booklet turns to the three commercial relationships that fund a team. Chapter 4 deals with sponsorship: what is actually sold, why the crypto boom of 2021 ended badly, and what the arrival of betting sponsors means. Chapter 5 deals with streaming rights and the rise of co-streaming, which has shifted value from the owners of broadcasts to the personalities who re-broadcast them. Chapter 6 deals with players: how contracts are structured, how salaries and buyouts work, what the LCS walkout of 2023 revealed, and why player bargaining power remains weak in most titles. The final chapters deal with capital. Chapter 7 tells the story of venture investment and public listings, from FaZe Clan's $725 million stock-market debut to its sale for about $17 million little more than a year later. Chapter 8 examines Saudi Arabia's role as the sector's most important patron. Chapter 9 explains the economics of virtual items, the revenue source on which the whole structure increasingly rests, and the regulatory and market risks that come with it. The conclusion draws out what follows for anyone who wants to invest in, sponsor, work in, or regulate the sector. A note on scope. The booklet concentrates on the large publisher-run ecosystems in North America, Europe and the Middle East, because that is where the business models discussed here were invented and tested. Chinese and Korean e-sports have their own histories, their own broadcasters and their own labour markets, and they appear where they illuminate the argument rather than as subjects in their own right. Mobile e-sports in South-East Asia, now a major market, appears mainly through its effect on investment flows. Figures are given in United States dollars unless otherwise stated, and they are reported figures unless the text says they are official. In a business where most companies are private and many deals are confidential, it is worth saying plainly when a number is an estimate. Chapter 1: The Landlord Problem Every sport has an owner of some kind. Football has its governing bodies, Formula One its commercial rights holder, the National Football League its thirty-two clubs acting together. But in all of those cases the thing being governed, the game itself, existed before the governors and would survive them. Nobody can revoke the right to kick a ball. In e-sports the order is reversed. The game is a piece of software, written and owned by a company, protected by copyright, and licensed to its users under terms the company writes. Competition happens inside that property, on the owner's servers or using the owner's client, and every broadcast of it is a public performance of the owner's copyrighted work. The professional scene is a tenant. The publisher is the landlord. This chapter sets out what that ownership means in practice. It identifies four distinct powers that a publisher holds, shows how each has been used, and explains why the rest of the industry's economics, from franchise fees to sponsorship rates, have to be read in their light. Four powers of the owner The first power is the power to license. A tournament that uses a commercial game, charges for tickets, sells sponsorship or streams the action is using the publisher's intellectual property for commercial gain, and it needs permission to do so. Most publishers grant that permission in two tiers. Small community events are allowed under a general licence, usually published online, which sets limits on prize money, sponsorship and scale. Larger events require a specific licence, negotiated with the publisher, which can carry fees, revenue shares, branding rules and a right of approval over the schedule. Riot Games, Blizzard, Epic Games and Nintendo all operate some version of this model. Valve, the owner of Counter-Strike and Dota 2, was for many years the most permissive, allowing almost anyone to run events. Even Valve, however, now publishes a formal rulebook for licensed Counter-Strike competition, hosted on its public code repository, which tells organisers how teams must be invited, how compensation must be disclosed and which business relationships are forbidden. The second power is the power to design. A publisher can change the game at any time. Balance patches alter which characters, weapons or strategies are strong, and a team that has built its roster around a particular style can find it obsolete overnight. More drastic changes replace the game entirely. In 2022 Blizzard replaced Overwatch with Overwatch 2, which cut team sizes from six players to five and forced every professional roster to shed a player. In 2023 Valve replaced Counter-Strike: Global Offensive with Counter-Strike 2. Activision releases a new Call of Duty title almost every year, and its professional league switches to each new game, so that players and coaches must relearn maps, weapons and movement annually. No traditional sport has ever had to cope with the ball changing shape every autumn. The third power is the power to govern conduct. Because professional play depends on the publisher's licence and often takes place in the publisher's own league, the publisher can discipline players and teams directly. It writes the rulebook, runs the integrity investigations and imposes the bans. In most traditional sports, discipline is shared between independent governing bodies, leagues and clubs, and it is usually subject to collectively bargained procedures. In e-sports, the company that sells the game is also the judge. The fourth power is the power to exit. A publisher can decide that its competitive scene no longer serves its interests and simply end it. This has happened repeatedly. In December 2018 Blizzard announced that it would end the Heroes Global Championship, the professional circuit for its game Heroes of the Storm, from the following year. Teams and players learned that their careers in the game were over in the same announcement as the public. In 2023 Blizzard and the owners of the Overwatch League teams agreed revised terms under which the league could be wound up, the owners voted in November to leave, and in January 2024 it was replaced with a smaller circuit run by a third-party operator. In both cases, investment in rosters, facilities and fan bases lost most of its value because a single company chose to stop. The powers in use These powers are not theoretical. Three episodes show how they are exercised and what they cost. The first is the Blitzchung affair of October 2019. During a post-match interview at a Hearthstone Grandmasters event broadcast from Taiwan, the Hong Kong player Ng Wai Chung, who competed as Blitzchung, put on a protest mask and called for the liberation of Hong Kong. The following day Blizzard banned him from Grandmasters for a year, took back roughly $4,000 in prize money and ended its work with the two broadcasters who had been interviewing him. Blizzard pointed to a rule that allowed it to remove any player who brought the publisher into disrepute or offended a portion of the public. The reaction was immediate. American senators and members of Congress from both parties wrote to the company, the hashtag #BoycottBlizzard trended globally, some of the company's own employees walked out, and Mitsubishi Motors withdrew its sponsorship of Hearthstone e-sports in Taiwan. Within days Blizzard's president halved the ban and restored the prize money. The episode showed how far the conduct power reaches: a publisher can penalise a player for political speech made on its broadcast, and there is no independent tribunal to appeal to. It also showed the limits of that power. Blizzard backed down because of outside pressure, not because of any procedural protection the player had. The second episode is the collapse of the Smash World Tour in November 2022. Super Smash Bros., Nintendo's fighting series, has one of the most dedicated grassroots competitive communities in gaming, and for years Nintendo tolerated it rather than supported it. In 2013 the company tried to stop the Evolution Championship Series from streaming its Super Smash Bros. Melee tournament, then relented after a public outcry. In 2020 it sent a cease-and-desist letter to The Big House, a major Smash tournament, over plans to run an online event using a community-made modification that enabled internet play. The Smash World Tour of 2022 was the most ambitious community effort yet. According to its organisers it drew together more than 6,400 offline tournaments and more than 325,000 players, culminating in a championship with a $250,000 prize pool, the largest in the game's history. Less than two weeks before the championship, the organisers announced that it was cancelled, saying that Nintendo had told them it would not grant a licence for that event or for the 2023 circuit. Nintendo later said that it had not required the finals to be cancelled. The dispute turned on the details of licensing conversations that are not public, but the lesson for everyone involved was plain: a community can build a sport for years and still have no standing if the owner of the game decides it will not license the result. In October 2024 Nintendo published community tournament guidelines under which unlicensed events are restricted to modest scale, with reported caps of around two hundred in-person participants and $5,000 in prizes. Anything larger needs Nintendo's explicit approval. The third episode is quieter but more consequential for money. In August 2023 Valve announced that, from 2025, Counter-Strike tournament organisers would no longer be allowed to have unique business relationships with the teams that played in their events. Invitations would be determined by Valve's ranking system or by open qualifiers, and all compensation paid to teams would be public and based on objective criteria. The announcement ended the partner-league arrangements that ESL and BLAST, the two dominant Counter-Strike operators, had built with selected teams, in which partners received guaranteed invitations and a share of the organisers' revenues. Those arrangements had been negotiated between private companies in good faith, and some had years left to run. Valve did not need anyone's agreement to end them. It simply wrote a new condition into its licence. Valve's stated rationale was to restore open competition, and there is a good argument that it succeeded. The point here is structural: a business model that two large operators and dozens of teams had invested in was abolished by the owner of the game with a two-year notice period. The broadcast right as the root of control The licensing power ultimately rests on copyright, and the clearest early test of it came not in America or Europe but in South Korea, the country where professional gaming first became a mass spectator sport. Through the 2000s, Korean television channels built an entire industry around StarCraft: Brood War, Blizzard's strategy game. The Korea e-Sports Association organised professional leagues, corporate-sponsored teams paid salaries, and two cable channels broadcast matches to large audiences. For years all of this happened without any formal licence from Blizzard, whose game was the product being shown. In 2010, as it prepared to launch StarCraft II, Blizzard sued the broadcasters for infringing its copyright. The Korean e-sports establishment argued that it had built the competitive scene, that the matches were created by the players and organisers rather than by the publisher, and that the broadcasts had helped sell Blizzard's game. Blizzard's position was that none of that mattered: the game was its property, and showing it to the public required its permission. The dispute was resolved through licensing agreements in 2011, and it established, in the market where e-sports had first become an industry, the principle that has governed the business ever since. However much value a community creates around a game, the publisher controls the right to broadcast it. That principle is why publishers can dictate terms to tournament organisers, why they can capture the most valuable media rights, and why a team's own content, when it uses game footage, is published at the owner's tolerance. It is also why the publisher's power extends well beyond its own leagues. A publisher that does not run any competition itself still controls who may run one and on what terms, as Nintendo's treatment of Smash and Valve's rulebook for Counter-Strike both show. Why publishers behave as they do It would be easy to read these episodes as stories of corporate caprice. They are better understood as the rational behaviour of a company whose e-sports operation is not its main business. For almost every publisher, the competitive scene is a marketing and retention tool. Its purpose is to keep players engaged with the game, to give them aspirational figures to watch and emulate, to generate content between game updates and to sell items. Riot Games has described its leagues as an investment in the long-term health of its games rather than a standalone profit centre, and in 2024 it said openly that the sponsorship-led model it had built with teams was not sustainable. When Riot laid off about 530 employees, around 11 percent of its staff, in January 2024, it also cancelled a planned e-sports streaming network of its own, which tells us something about where competition ranks when budgets tighten. A publisher will support e-sports as long as it believes the scene sells the game and its items. When that belief weakens, or when the scene becomes a source of reputational risk, it will cut back. This produces a particular pattern of incentives. Publishers want the competitive scene to be large, visible and exciting, because that is what drives engagement. They do not especially care whether independent teams make a profit, except insofar as failing teams create bad publicity or leave gaps in the league. They want control over messaging, because the league is a marketing channel. And they want flexibility, because the game itself will change and the competitive structure has to change with it. None of these incentives is malicious. All of them make the publisher an awkward partner for anyone trying to build a durable, independent business on top of the game. There is also a timing problem. Traditional sports franchises are valued on the assumption that the sport will last indefinitely. A video game has a commercial life. League of Legends, released in 2009, and Counter-Strike, whose lineage goes back to 1999, have shown that some games can last decades, but they are exceptions. Most competitive titles have a peak of a few years. An investor in a team is therefore making two bets at once: a bet on the team's ability to win and attract fans, and a bet on the game's longevity and the publisher's continued commitment. The second bet is usually the larger one, and it is the one the investor can do least to influence. The tenant's bargaining position If publishers hold such power, why do teams, sponsors and players deal with them at all? The answer is that the relationship is not entirely one-sided. Publishers need competitors. A league without recognisable teams is less watchable, and some teams bring audiences that exceed anything the league could build alone. When the League of Legends Championship Series players' association threatened to walk out in 2023, Riot had to delay the start of its summer season and ultimately negotiated an agreement, which Chapter 6 examines in detail. When the owners of Overwatch League teams collectively pushed for the league's wind-down, Blizzard had to pay attention. Popular creators and personalities, a subject for Chapter 5, can command viewing figures that rival the official broadcast. Third-party organisers such as the Esports World Cup can offer publishers large licence fees and global exposure, giving them reason to cooperate. Moreover, the publisher's power to change the rules cuts both ways. A publisher that behaves erratically will find it harder to attract sponsors, teams and investors to its next league. Blizzard's reputation in e-sports never fully recovered from the combination of the Heroes of the Storm shutdown, the Blitzchung affair and the Overwatch League's collapse. Riot's decision to open the Valorant Champions Tour to open qualifiers from 2027 was accompanied by a promise of guaranteed payments to partner teams precisely because the company understood that abrupt changes without compensation would damage trust. Still, the structural fact remains. In every negotiation, the publisher holds the one asset that cannot be replicated: the game. Teams can be replaced, as leagues have shown by selling slots from departing owners to new ones. Tournament organisers can be replaced; publishers have moved their licences from one operator to another many times. Broadcasters can be replaced. Sponsors can be replaced. The game cannot be. Every other participant in the ecosystem is, in economic terms, providing a service to the owner of that asset, and the price of their service depends on how badly the owner needs it. Reading the rest of the industry through this lens Once the landlord problem is understood, a great deal of the industry's history makes sense. The franchise fees of 2017 to 2019, examined in the next chapter, were payments for a licence to participate in a publisher's league, and their value depended on the publisher's continued commitment to that league. Sponsorship value depends on audience, but audience depends on the game's popularity and on the publisher's broadcast choices. Player contracts are constrained by publisher rules, and the best players' market value can collapse if their game declines. Investor valuations of teams must account for the chance that the publisher will reorganise or abandon the scene. And the most promising new revenue sources for teams, digital items sold inside the game, exist only because the publisher chooses to share revenue it could keep for itself. The practical question for anyone doing business in e-sports is therefore not whether the publisher has power. It does. The question is what makes the publisher want to share value with you. The rest of this booklet examines the different answers the industry has found, from franchise fees and revenue pools to sovereign sponsorship and creator audiences, and assesses which of them have held up. A useful discipline for readers is to ask of every e-sports arrangement a single question: what happens to it if the publisher changes its mind? An asset that survives a change of mind by the publisher has real value. An asset that exists only because of the publisher's current policy is a licence, and should be priced like one. Chapter 2: The Franchise Bet Between 2017 and 2020, four of the largest publishers in the world tried to turn their competitive scenes into something that looked like North American professional sport. Riot Games franchised its North American League of Legends Championship Series for 2018 and its European league the following year. Activision Blizzard launched the Overwatch League in 2018, organised around teams named after cities, and followed it with a franchised Call of Duty League in 2020. The idea was simple and, at the time, persuasive. Investors would pay a large entry fee for a permanent place in a closed league, free from the risk of relegation. In return they would receive a share of league revenues, the ability to build long-term brands and, eventually, an asset whose value would rise as e-sports grew, just as the value of a National Basketball Association team had risen over the previous three decades. This chapter examines that bet: what buyers paid, what they thought they were buying, what they actually bought and why, in most cases, it did not pay off. It is the central cautionary tale of e-sports business, and understanding it is the precondition for understanding what came after. Why franchising looked attractive Before franchising, most e-sports leagues ran on the European model of promotion and relegation. A team that finished at the bottom of the top league could be dropped to a lower tier, where sponsorship and viewership were a fraction of what they had been. For a team owner, this was an existential risk. One bad season could wipe out the value of years of investment. Sponsors were reluctant to sign long deals with teams that might not be in the top league the following year. Players could not be offered long-term security. And outside investors, particularly the owners of traditional sports teams who had begun to take an interest, found the model alien. American sports investors were used to closed leagues in which a franchise was a permanent asset. Publishers had their own reasons to change. Riot's North American league had been dogged by financial instability among teams, with owners complaining that they could not make money and that the risk of relegation made long-term planning impossible. A closed league, in which each team had paid a substantial sum to join, promised financial commitment and professional management. It also offered the publisher a large, immediate cash injection. And it allowed publishers to sell a new story to sponsors and media partners: e-sports was growing up and becoming a real sport, with stable teams, permanent rivalries and, in Blizzard's case, local fan bases in real cities. For investors, the case was built on analogy. The value of franchises in American sports had risen spectacularly. The cheapest National Basketball Association franchises had changed hands for tens of millions of dollars in the 1980s and were worth billions by the late 2010s. E-sports audiences were young, digitally native and hard to reach through television. If e-sports became even a fraction as large as traditional sport, the early buyers of permanent franchises would own the equivalent of early basketball teams. The price of entry, however high it looked, would seem cheap in retrospect. What was paid Table 1 sets out the reported entry prices and later fate of the four leading franchised leagues. The figures are as reported at the time by the leagues or by the specialist press; publishers did not publish the full terms, and fees were often payable in instalments. Table 1. The four major franchised leagues: first franchised season, reported entry costs, teams at launch and status in 2026. League Year Entry fee (reported) Teams Status in 2026 LCS 2018 $10m; $13m for new teams 10 Restored after LTA; 8 teams Overwatch League 2018 $20m; later $30m to $60m 12, then 20 Folded in 2024 LEC 2019 About 8m to 10.5m euros 10 10 partner teams Call of Duty League 2020 $25m 12 12 teams Sources: Riot Games and Activision Blizzard announcements; reporting by ESPN and others; league records. The buyers were a revealing mix. The Overwatch League's first owners included Robert Kraft, owner of the New England Patriots, whose group took the Boston franchise; Stan Kroenke's company, owner of the Los Angeles Rams and Arsenal, which took a Los Angeles franchise; and Sterling.VC, linked to the family that owned the New York Mets, which took New York. The LCS attracted the Golden State Warriors, the Houston Rockets and Dan Gilbert, owner of the Cleveland Cavaliers, who backed 100 Thieves. Alongside them came established e-sports organisations such as Team Liquid, Cloud9 and TSM, which raised venture funding in part to pay the fees. The buyers brought credibility, business discipline and, above all, the conviction that they knew how sports franchises gained value. What was bought The central flaw in the franchise bet is visible once we ask what exactly the buyers acquired. In American sport, a franchise is a share in a joint venture. The teams together own the league, negotiate its media rights, set its rules through collective governance and share its revenues. The commissioner works for the owners. No outside company can abolish the NFL, and no one owns American football. An e-sports franchise was something different. It was a licence, granted by the publisher, to participate in a league the publisher owned, run on the publisher's game, under rules the publisher wrote and could change. Teams typically received a share of league revenues, from media rights and league-level sponsorship, and they had some say through advisory councils. But ultimate control stayed with the publisher. The publisher could change the format, alter the schedule, merge the league with another, open it to new entrants, change the revenue-sharing formula or, in the extreme case, close it down. The franchise was only as permanent as the publisher's commitment. This mattered in two ways. First, it limited the upside. In American sport, franchise values rise partly because the teams collectively capture the growth of the league's media rights. In e-sports, the publisher captured much of that growth, and the most valuable revenue streams tied to the game, including sales of in-game items, stayed with the publisher unless it chose to share them. Second, it exposed the buyers to a risk that American sports owners never face: the risk that the owner of the underlying product would lose interest, change strategy or decide that the league was a cost to be cut. The difference can be put as a simple test. A traditional sports franchise is worth something even if the league office disappears, because the teams could form a new league around the same sport. An e-sports franchise is worth nothing if the publisher withdraws its licence, because the teams have no right to use the game. The arithmetic Set aside the question of permanence for a moment, and consider whether the franchise could have paid for itself even on favourable assumptions. The following worked example is illustrative, not a reconstruction of any real team's accounts. Suppose a team paid $20 million for an Overwatch League slot. Suppose its investors expected a return of 10 percent a year, modest by venture standards. The franchise would need to generate $2 million a year of profit, after all costs, just to justify its purchase price, before any return on the additional capital needed to run the team. Now consider the costs. A roster of eight to twelve players at the league's minimum salary of $50,000, plus health insurance, retirement contributions and housing that the league required, would cost well over half a million dollars a year before any star player was paid above the minimum. Coaches, analysts, managers, content staff, travel, facilities and marketing would add several million more. A realistic total for a competitive team in a top league was several million dollars a year. Against this, a team had four sources of revenue. It received a share of league revenues from media rights and central sponsorship. It could sell its own sponsorships. It could sell merchandise. And, in the Overwatch League's model, it could host home matches in its city and sell tickets. The league's media deals were large by e-sports standards: Twitch reportedly paid $90 million for two years of exclusive streaming rights from 2018, and YouTube reportedly paid about $160 million for three years of rights to Activision Blizzard's leagues from 2020. But divided among twenty teams, even after the league's share, such sums did not go far. Team sponsorship depended on audience, and audiences were concentrated in a few popular teams. Merchandise was a minor contributor for most organisations. Home matches, the distinctive feature of the city-based model, were barely tested before the COVID-19 pandemic ended live events in 2020. The result was that most teams operated at a loss. They were not recovering their entry fees through operating profit; they were betting on capital appreciation, the rise in franchise values that would let them sell to a new owner at a higher price. That is a legitimate strategy, and it worked for early owners of American sports teams. But it only works if new buyers keep arriving, willing to pay more. When the growth story faltered, the only exit was a sale at a lower price, or no sale at all. The Overwatch League's collapse The Overwatch League was the most complete test of the franchise model, and its failure was the most complete. Blizzard sold the city-based idea hard. Each team would represent a city, play home matches there and build a local fan base, just like a baseball team. The first season in 2018 had twelve teams, and eight more joined in 2019 at reportedly higher prices, with slots in Asia and Europe as well as North America. Several problems emerged quickly. The city-based model required expensive local infrastructure, venues and staff, but e-sports audiences are global and online rather than local. A fan in Texas was just as likely to support the London team as the Houston one. Home matches, when they happened, drew real crowds but at a cost that the revenues could not justify. The pandemic then made live events impossible for most of 2020 and 2021, destroying the central premise of the model before it had been properly tested. The game itself became a problem. Overwatch's player base declined after its early peak, and the long wait for Overwatch 2, released in October 2022, left the competitive scene playing an ageing game. When Overwatch 2 arrived, it changed the team size from six players to five, forcing rosters to cut a player. Viewership, never as strong as Blizzard had projected, fell further. Activision Blizzard was meanwhile engulfed by lawsuits over its workplace culture from 2021 and by Microsoft's acquisition, completed in October 2023. The owners, locked into a league that was losing money, pressed Blizzard for relief. In early 2023 Blizzard agreed to waive outstanding franchise fee payments and put in place revised operating terms. Those terms included a mechanism under which the teams could vote to leave the league, in which case each would receive a termination payment of $6 million. In November 2023 the required two-thirds majority of owners voted to leave. The total payout was reported at around $114 million. In January 2024 Blizzard formally ended the league and announced its replacement, the Overwatch Champions Series, run in partnership with ESL FACEIT Group and the Asian operator WDG Esports. The new circuit was open, regional and far smaller. The arithmetic of the Overwatch League from an owner's point of view is stark. An original owner paid $20 million, spent millions more each year operating a team and received $6 million on exit. The expansion owners, who paid more, lost more. The city-based sport that was supposed to create the next generation of billion-dollar franchises lasted six seasons. The partial survivors Not every franchised league collapsed. The Call of Duty League, launched in 2020 on similar city-based principles, has survived, though in altered form. Its home-and-away model gave way to centralised events, and many of the original slots have changed hands. By the 2026 season its twelve teams included the Saudi-owned Riyadh Falcons, a French entrant playing as Paris Gentle Mates and teams from established brands such as OpTic, FaZe, Cloud9 and G2, several of which operate slots with city names attached rather than genuinely local identities. The Call of Duty League has benefited from the game's enormous player base and from Activision's annual release cycle, which gives each season a fresh game to market. But its survival owes more to the importance of Call of Duty to Activision than to the success of the franchise model itself. Riot's leagues have survived by changing their terms. In 2024 Riot announced that it would move League of Legends e-sports from a model in which teams relied mainly on sponsorship and league revenue shares to one in which they received a fixed stipend and a share of sales of in-game digital content, as Chapter 3 describes. That change acknowledged that the original franchise economics had not worked. In North America, several original owners left. The Houston Rockets sold their slot within two years; the Golden State Warriors' Golden Guardians left after the 2023 season; TSM, once the region's most popular team, sold its place in 2023, and 100 Thieves announced its withdrawal after the 2025 season. Riot merged the LCS with Brazil's league into a single League of the Americas for 2025, then reversed the decision and restored both leagues for 2026, telling fans that it had learned they felt the deepest attachment to their original leagues. The restored LCS has eight teams, not ten, including one guest slot for a team promoted from the lower tier, a partial return of the promotion and relegation that franchising was supposed to abolish. The European league has been the most stable, with ten partner teams throughout. But even there, slots have turned over through sales and mergers. Karmine Corp, a French organisation built around a large and passionate online community, bought Astralis's place in 2023. Natus Vincere acquired Rogue's place in June 2025. The organisations that have thrived are those with strong fan bases, not those that relied on the franchise itself as a store of value. The case for the defence It would be unfair to leave the franchise era without stating the best argument in its favour, because it contains some truth. Franchising professionalised e-sports. Before 2018, many teams in top leagues were run informally, players were sometimes paid late or not at all, and contracts were often rudimentary. The franchised leagues imposed minimum salaries, standard contracts, benefits and operating requirements, and they attracted executives from traditional sport and media who brought accounting, marketing and legal discipline. The minimum salaries of $75,000 in the LCS and $50,000 in the Overwatch League, with health insurance and retirement savings, were a significant improvement for players, and they set expectations that have outlasted the leagues that introduced them. Franchising also brought in capital at a time when the industry needed it. The entry fees financed publisher investment in production, broadcasting and events, which raised the quality of e-sports as entertainment. And the credibility of owners such as Robert Kraft and the Golden State Warriors helped persuade mainstream sponsors that e-sports was a serious marketing channel. Some of the sponsor relationships formed in those years, such as Mastercard's with League of Legends, have lasted. Finally, the defenders argue, the failure was one of timing and execution rather than principle. The pandemic destroyed the Overwatch League's live-event model before it could be tested. Rising interest rates in 2022 hit every growth industry. Blizzard's own troubles, from its delayed sequel to its legal crises, would have harmed any league it ran. With better luck, the argument goes, franchising might have matured into something like the stable structures of traditional sport. There is force in these points, and the professional standards of the franchise era deserve credit. But the defence does not answer the central problem. Even the best-run franchised league, the European LEC, has seen its slots change hands at prices below the original fees, and Riot felt obliged to restructure its finances in 2024. The Call of Duty League survives because Activision values it, not because its owners have seen their investment appreciate. Bad luck explains why the Overwatch League failed so quickly; it does not explain why no franchised league has delivered the appreciation its buyers paid for. That failure is structural, and it follows from the fact that the franchise was a licence rather than a share in a jointly owned sport. What the franchise bet teaches The failure of the franchise bet was not simply a matter of bad luck, although the pandemic and the rise in interest rates from 2022 certainly did not help. It was a failure to price the landlord problem. Buyers paid for permanence and appreciation, the two things that make American sports franchises valuable, in a structure that could offer neither. The franchise was a licence to participate in a publisher's marketing programme, and it was worth what the publisher's continued commitment to that programme was worth. The lesson has been widely absorbed. Apart from Korea's LCK, which adopted franchising in 2021 on the back of a uniquely strong domestic market, no major publisher has launched a new franchised league with large entry fees since the Call of Duty League in 2020. The models that have replaced franchising, examined in the next chapter, are designed around the opposite principle: the publisher does not charge teams for entry, and instead pays them, directly or through a share of its own revenues, to take part. That reversal marks the end of the idea that an e-sports slot is an appreciating asset. It is better understood as a service contract, under which teams provide competitive content and audiences in return for a share of the value the publisher creates. Hashtags: #TheBusinessOfESports #EsportsBusiness #PublisherOwnedEcosystems #PublisherPower #GameLicensing #EsportsSponsorships #PlayerContracts #LeagueFranchising #RevenueSharing #DigitalGoodsRevenue #VirtualItems #MediaRights #CoStreaming #TournamentLicensing #EsportsTeams #FranchiseFees #OverwatchLeague #LeagueOfLegendsEsports #ValorantChampionsTour #CounterStrikeEsports #EsportsInvestment #SovereignCapital #PlayerBargainingPower #EsportsCommercialModels #FutureOfEsportsBusiness