Download the Book (PDF): Introduction Most postdoctoral fellows can describe, in some detail, the experiment they will run next week. Far fewer can describe the laboratory they will be running five years from now: what question it will be known for, who will pay for it, how many people will work in it, and why a search committee should believe it will exist at all. That asymmetry is the central problem of the postdoctoral years. Training rewards depth in the present tense. Independence is judged in the future tense. This booklet is about closing that gap deliberately. It is written for biomedical and life-science postdocs who want a tenure-track or equivalent independent research position, and for the mentors, program directors and early-career faculty who advise them. Its examples lean toward the United States and the National Institutes of Health, because that is where the largest structured bridge from trainee to independent investigator exists, but most of the reasoning transfers to other systems, and where it does not, the difference is flagged. The controlling idea The argument of this booklet can be stated in one sentence: independence is not a status you are granted at the end of a postdoc; it is a case you build during it, and every major decision of the postdoctoral years, from which project to lead to which grant to write to which offer to accept, should be judged by whether it strengthens that case. That sounds obvious, and yet most postdocs do not work this way. They work as excellent employees of someone else's program, publish well, and then, in the fourth or fifth year, try to assemble a job application from the pieces. The pieces rarely fit together. The papers answer the mentor's question rather than the applicant's. The proposed research looks like a continuation of the mentor's laboratory, which raises the uncomfortable question of why a university should fund a second copy of it. The applicant has no independent funding, or has written a bridge grant too late to receive it before the eligibility clock runs out. And the startup negotiation, when it comes, is conducted by someone who has never priced a laboratory. None of this reflects a lack of talent. It reflects a lack of design. The postdoc who treats independence as a project, with its own milestones, budget and risk register, is not more gifted than the one who does not. She simply knows earlier what evidence she will need to produce, and arranges her years so that the evidence exists when it is asked for. Why this matters more in 2026 The case for deliberate design has become stronger in the last two years, because the ground under early-career scientists in the United States has shifted. In 2025 NIH terminated the MOSAIC program, a K99/R00 variant aimed at postdocs from backgrounds underrepresented in research, and reporting by STAT in December 2025 found that roughly 104 MOSAIC awardees had their funding terminated, some later restored by court order. The same reporting counted 172 fewer postdoctoral transition awards in the first nine months of 2025 than in the same period of 2024, and 7,504 early-career researchers receiving NIH awards between January and September 2025, the lowest figure for a comparable period since 2016. Other changes compound these. NIH began obligating more research-project-grant money up front through multiyear funding in fiscal year 2025, which leaves less money for new competing awards in any given year; Congress capped that practice at fiscal 2025 levels in the 2026 appropriations act. Institutes moved away from publishing fixed paylines toward a unified funding strategy in which council weighs portfolio balance and other factors alongside percentile scores. In 2025 NIH also limited each principal investigator to six new, renewal, resubmission or revision applications per calendar year and stated that applications substantially developed by artificial intelligence would not be considered original. And in August 2026 NIH announced that the K99 itself will be consolidated with the K22 into a new "Career Transition to Research Independence Award", activity code KT1 paired with the R00, for application due dates on or after 12 October 2027. The upshot is not that the path has closed. Roughly two hundred K99 awards a year were made across the two decades before this disruption, and the mechanism survives into its new name. The upshot is that the margin for drift has narrowed. A postdoc who hopes that the right grant, the right job and the right project will line up by themselves is betting on a looser system than the one that now exists. What the booklet covers The book is organised around the order in which decisions actually arrive, not the order in which they are usually taught. Chapter 1 sets out the arithmetic of independence: how many positions there are, what search committees and study sections are actually evaluating, and how to build a realistic multi-year timeline backwards from the job you want. Chapter 2 deals with the question most postdocs postpone for too long, which is how to carve an independent research program out of a mentor's laboratory, and how to negotiate what you can take with you. Chapter 3 is a detailed treatment of the K99/R00 Pathway to Independence Award as it stands in 2026, including its eligibility window, budget, review criteria, the extensions granted in 2025 and 2026, and the coming transition to the KT1. Chapter 4 surveys the other bridges: institute-specific K99 variants, the NIH Director's Early Independence Award, private foundation awards, and the major European and United Kingdom schemes, together with the logic for choosing among them. Chapter 5 covers the faculty job market itself: the application package, the research talk, the chalk talk, and the second visit. Chapter 6 is about negotiating the startup package, with the components itemised and a worked budget. Chapter 7 addresses the first three years of running a laboratory, when most of the promises made in the job talk are tested. Chapter 8 turns to what this booklet calls the lab brand, meaning the durable scientific identity that makes colleagues, students, reviewers and funders associate a problem with a name. The conclusion draws the argument together and identifies what remains genuinely uncertain. Five tables appear where a comparison is genuinely tabular: the two phases of the K99/R00, the main bridge awards, the components of a startup package, a worked startup budget, and a first-three-years milestone plan. Everything else is prose, because most of what matters in this transition is judgement rather than data. Three assumptions worth stating First, the booklet assumes that you want an independent academic research career and have at least a reasonable chance of one. That is a choice, not a default. Many excellent scientists build careers in industry, government, research institutes and science policy, and the best decision some postdocs make is to leave academic research early and on their own terms. Several chapters, especially the first, are written to help you make that decision with clear eyes rather than to talk you out of it. Second, it assumes you are willing to treat your own career with the same discipline you apply to an experiment: stating hypotheses, collecting evidence, and changing course when the data say so. The advice here is not a set of hacks. It is an argument that careful planning, done early, removes much of the luck from a process that is often described as a lottery. Third, it assumes that rules change. Specific numbers in this booklet, such as eligibility windows, budget caps, and notice numbers, were checked against NIH guide notices and funder pages as of September 2026. Before you rely on any of them for an application, check the current funding opportunity and speak to the program officer at the institute you are targeting. Program officers are paid to answer exactly these questions, and the most common regret of unsuccessful bridge-grant applicants is that they did not call one early enough. How to use this booklet Read Chapters 1 and 2 early, ideally in the first year of a postdoc, when the choice of project is still open. Read Chapters 3 and 4 at least eighteen months before you hit any eligibility deadline, because a competitive bridge application takes most of a year to prepare and may need a resubmission. Read Chapters 5 and 6 before you apply for faculty jobs, not after you receive an offer; negotiation is much easier when you know in advance what you will need. Read Chapters 7 and 8 before your first day as an assistant professor, and again at the end of your first year. The transition from trainee to independent investigator is the most consequential career move most scientists ever make, and it happens under time pressure, with incomplete information, and usually while running experiments full time. It deserves the same planning a scientist would give to a five-year research program. That is what it is. Chapter 1. The Arithmetic of Independence Every serious career plan begins with a count. Before a postdoc decides how to spend the next four years, she should know roughly how many independent positions exist in her field, what evidence the people filling them look for, and how long the path actually takes. Most postdocs know these numbers only as rumour: that there are "hundreds of applicants per job", that "you need a Cell paper", that "nobody gets hired without a K99". Rumour is a poor basis for planning. It is usually half right, and the wrong half is where careers go astray. Counting the positions and the people The United States trains far more doctoral scientists in the biomedical sciences than it employs in tenure-track research positions, and it has done so for decades. This imbalance was the core of the 2014 analysis by Bruce Alberts, Marc Kirschner, Shirley Tilghman and Harold Varmus, "Rescuing US biomedical research from its systemic flaws", which argued that the research enterprise had come to depend on a large, cheap, and growing pool of trainees whose prospects of independence were steadily worsening. The National Academies' 2014 report The Postdoctoral Experience Revisited reached a similar conclusion from a different angle: the postdoc had drifted from a short apprenticeship toward an extended, poorly defined holding position. A decade later the structural picture is similar. A minority of biomedical PhDs end up in tenure-track faculty roles, and the median age at which a PhD scientist receives a first major independent NIH research grant has for many years sat in the early forties. Those two facts together should shape a postdoc's thinking more than any anecdote. The first says that the academic path is one option among several, and that planning for it does not mean ignoring the others. The second says that the transition is long, and that the years between the doctorate and a first R01-equivalent grant are where most of the attrition happens. Surveys of the job market itself give the same message at finer grain. Jason Fernandes, Sarvenaz Sarabipour and colleagues, writing in eLife in 2020, surveyed several hundred recent faculty applicants and found that most applied to many positions, received few interviews, and received fewer offers still; they also found that conventional markers such as first-author papers in high-profile journals were associated with success but did not guarantee it, and that many successful candidates lacked them. Analyses of faculty hiring networks, such as the 2015 study by Aaron Clauset, Samuel Arbesman and Daniel Larremore in Science Advances, show that a small number of prestigious institutions produce a disproportionate share of faculty across many fields. Prestige of training environment matters. It is not the whole story, but pretending it does not matter leads to bad planning. What does this mean in practice? It means a candidate should expect to apply to a substantial number of positions in a single cycle, often thirty or more in the life sciences, and should expect most of those applications to produce nothing. It means that the right comparison for any decision is not "Will this make me hireable?" but "Will this make me more hireable than the next forty strong candidates who applied to the same job?" And it means that the evidence a postdoc accumulates must be distinctive, not merely sufficient. What committees and study sections actually evaluate Search committees and grant review panels ask versions of the same four questions. Understanding them is more useful than any list of accomplishments to collect. The first question is whether you can do excellent science. This is the easiest to answer and the least discriminating at the top of the applicant pool. Publications, technical skill, and the quality of your training demonstrate it. At the stage of a competitive faculty search, nearly everyone on the shortlist clears this bar. The second is whether you have a question of your own. Committees want to hire a research program, not a pair of hands. They want to hear a problem that is important, that is not already owned by a large and well-funded laboratory, and that you are unusually well placed to attack. A candidate who describes a continuation of her mentor's work, however elegant, answers this question badly. The third is whether the program is fundable. A department is making an investment that commonly runs to seven figures once startup, space and salary are counted. The investment is recovered, in part, through indirect costs on external grants, and in larger part through the reputational return of a successful laboratory. Committees therefore ask, sometimes explicitly, whether they can imagine your first R01 or equivalent. The strongest evidence is that someone has already funded you: a K99, a foundation fellowship, a career development award. The fourth is whether you will be a good colleague. Departments are small societies. A candidate who will collaborate, teach competently, share equipment and mentor students well is worth more than one who will not, and committees look for signals of this in letters, in how candidates treat staff during interviews, and in the questions candidates ask. Bridge-grant study sections ask nearly the same questions, restated as review criteria. The K99/R00 is reviewed on the candidate, the career development plan, the research plan, the mentor and the environment, and the institution's commitment. The first maps to excellence, the research plan to having a question, the career plan and mentorship to the credibility of the transition, and the entire application, implicitly, to fundability. That overlap is one reason that writing a K99, even an unsuccessful one, is such good preparation for the job market. It forces a postdoc to answer the four questions on paper, a year before a search committee asks them in person. The evidence nobody lists on a CV Publications and grants are the currency that everyone counts, but a search is decided by a wider set of signals, and several of them take years to accumulate. It pays to know what they are early. The first is letters of reference. A faculty application typically requires three to five, and the most important is from the postdoctoral mentor. Committees read these letters closely and have learned to decode them. A letter that says a candidate is "hard-working and technically skilled" is a weak letter. A letter that says the candidate "conceived this project herself, against my initial advice, and it has become the most interesting thing in my laboratory" is a strong one. Mentors can only write the second kind of letter if it is true, and it is only true if the postdoc has actually led something. The other letters matter too: a letter from a collaborator outside the laboratory, or from a senior scientist who has watched the candidate present at meetings, provides independent confirmation that the candidate is known beyond her mentor's circle. The second is a public scientific voice. Committees want to know that a candidate can explain her work to people who do not already care about it. That skill is built by giving talks: laboratory meetings, departmental seminars, poster sessions, selected talks at society meetings, and eventually invited seminars at other institutions. A postdoc who has given a dozen talks outside her own laboratory before going on the market will give a much better job talk than one who has given two. Talks also generate the informal network through which many searches are actually shaped. A committee member who heard a candidate give a crisp ten-minute talk at a Gordon Research Conference two years ago is already, in a small way, an advocate. The third is mentoring and teaching. Almost every faculty position involves supervising students and, in most departments, teaching. Committees read the evidence for this in research statements, teaching statements, and letters. A postdoc who has directly supervised a rotating graduate student or a summer undergraduate, and can say what that student went on to do, has something concrete to offer. Formal teaching certificates, offered by many universities through centres for teaching and learning, are useful mainly because they produce a documented record and a coherent teaching philosophy, not because committees check them. The fourth is preprints and open work. In most of the life sciences, posting preprints on bioRxiv has become routine, and committees now read preprints as evidence of productivity, especially for work that is still in review during the application cycle. A postdoc whose anchor paper will not be published until after the application deadline should post it as a preprint in time for committees to read it. Open datasets, software and protocols play a similar role, and in some fields, such as computational biology or imaging, a widely used tool can be as strong a signal as a paper. None of these signals can be manufactured in the final six months before an application. All of them can be built steadily across a postdoc, at modest cost, if the postdoc knows from the beginning that they will be read. Deciding whether this is the right path The arithmetic above is sobering, and it should be. The right response is not despair but clarity. A postdoc who has looked honestly at the numbers and still wants an independent academic career will make better decisions than one who is pursuing it by default. A postdoc who looks at the same numbers and decides that industry, a research institute, a core facility, science policy or teaching better suits her values has not failed. She has done the analysis. A few questions help separate the two cases. Do you have a scientific question you would pursue even if nobody asked you to? Do you enjoy, or at least tolerate, writing grants, managing people, and living with uncertain funding? Are you prepared to relocate, possibly more than once, to a place you would not choose on other grounds? Would a staff scientist position in an excellent laboratory, with a stable salary and no grant pressure, feel like a defeat or a relief? None of these questions has a right answer. But the answers should be known. For those who decide to pursue independence, the most important consequence of the arithmetic is this: the scarcity of positions and the length of the path mean that time is the scarcest resource. A year spent on a project that cannot become yours, or a K99 written after the eligibility window has closed, is not merely a delay. It is a year taken out of a finite window. The rest of this booklet is about using that window well. Building the timeline backwards The most useful single exercise for a postdoc is to write the timeline backwards from the day she wants to open her laboratory. The typical sequence in the United States looks something like this. Faculty searches in the life sciences mostly advertise between late summer and early winter, interview between winter and spring, and make offers in spring, with start dates anywhere from six to eighteen months after the offer. So a candidate who wants to start a laboratory in the autumn of year N should expect to apply in the autumn of year N minus one or N minus two. A K99 application, if successful on the first submission, typically takes about nine months from submission to award, because it moves through peer review, advisory council and then award processing. If it needs a resubmission, add roughly another six to nine months. And the K99/R00 imposes an eligibility limit: applicants may have no more than four years of postdoctoral research experience at the time of the initial application or the resubmission. That means the first K99 submission should ideally go in during the second or early third year of a postdoc, which in turn means that the preliminary data for an independent aim must exist by then. Working backwards, a sensible plan for a four-to-five-year postdoc looks like this. Year one is for joining the laboratory, learning the systems, and choosing a project that can become yours. Year two is for generating the first independent result and drafting the K99. The end of year two or early year three is for submitting it. Year three is for resubmitting if needed, publishing the anchor paper, and beginning to give talks outside the laboratory. The autumn of year three or four is for the first round of faculty applications. Year four or five is for interviews, offers, negotiation, and the move. This timeline is compressed, and many excellent scientists take longer. The point of writing it down is not to meet it exactly but to see which steps are rate-limiting. For most postdocs the rate-limiting step is not publication. It is the identification, early enough, of a project that is both theirs and fundable. Chapter 2 is about that step. Early Stage Investigator status and other clocks Several other clocks run in parallel, and a sensible plan accounts for all of them. NIH defines an Early Stage Investigator (ESI) as a new investigator who is within ten years of completing the terminal research degree or the end of post-graduate clinical training, and who has not yet competed successfully for a substantial independent NIH research grant. ESI applications for research grants receive special consideration in funding decisions at most institutes, and in 2025 and 2026 institutes continued to say publicly that early-stage investigators would receive such consideration under the new unified funding strategy. The R00 phase of a K99/R00 does not remove ESI status, so a K99 awardee can still apply for her first R01 as an ESI. The ten-year clock can be extended for specific reasons, such as childbirth, family care responsibilities, medical conditions or military service, but it is otherwise unforgiving. A postdoc who spends seven years in training will enter the job market with only three years of ESI eligibility left, and that is worth knowing in advance. Visa status is another clock for international postdocs. The K99/R00, unusually among NIH career awards, is open to non-citizens, but the transition to a faculty position often requires a change of visa category, and some other bridge awards are restricted to citizens and permanent residents. International postdocs should read the eligibility section of every award they are considering in the first year, not the third. Personal clocks matter too. Partners' careers, children, ageing parents, and health all interact with a mobile, uncertain career. There is no general advice that fits these situations, except that they should be part of the plan from the start rather than an afterthought at the negotiation stage. Chapter 6 discusses how to raise them in negotiation. A worked example Consider a composite but realistic case. Dr. A completed a PhD in cell biology in 2023 and began a postdoc in a well-funded immunology laboratory in early 2024. She has two first-author papers from her doctorate and a strong technical background in live-cell imaging. Her mentor has offered her a choice of two projects. The first is a large, well-funded collaboration on a signalling pathway the laboratory has studied for a decade. It will almost certainly produce a high-profile paper within three years, with Dr. A as one of several co-first authors. The second is a riskier project applying her imaging expertise to a question at the edge of the laboratory's interests: how a particular class of tissue-resident immune cells makes decisions in intact tissue. The mentor is supportive but does not plan to pursue this direction after Dr. A leaves. Working backwards from a target start date of autumn 2028, Dr. A's K99 must go in no later than early 2026 to leave room for a resubmission within her four-year window, which closes in early 2028. She will need preliminary data for an independent aim by late 2025. The first project will not give her that, because its independent angle is unclear and the laboratory will continue it after she leaves. The second project might, and the question it addresses is precisely the kind a search committee will see as hers. The sensible plan is not to choose one project exclusively. It is to contribute to the first project at a defined level that secures a middle-authorship on a strong paper and a close relationship with the collaborators, while making the second project the core of her own work, with a clear agreement with her mentor that she will take it with her. That agreement, and how to negotiate it, is the subject of the next chapter. Dr. A's case illustrates the general principle. The choices that most affect a postdoc's chance of independence are made in the first eighteen months, often without anyone labelling them as career decisions. They should be labelled, and made on purpose. Chapter 2. Designing a Research Program You Can Take With You The hardest intellectual task of the postdoctoral years is not a technique or an experiment. It is the design of a research program that is recognisably yours, that grows out of your training without duplicating it, and that a university would pay to host for the next decade. Most postdocs arrive at this task late, and many arrive at it by accident, discovering during a job application that the story they can tell is really their mentor's story. This chapter treats the design of the program as a deliberate piece of work, with its own methods and its own negotiation. Choosing a problem that is yours Uri Alon's short essay "How to choose a good scientific problem", published in Molecular Cell in 2009, remains the clearest guide to the underlying judgement. Alon suggests placing candidate problems on two axes: how feasible they are, and how much knowledge they would generate if solved. A new laboratory needs a mix. It needs at least one problem that is feasible enough to produce results within two or three years, because students need papers and the laboratory needs preliminary data for its first grant. It also needs at least one problem that is large enough to be worth a career, because that is what defines the laboratory's identity and attracts the best students. A postdoc planning her own program should ask of each idea where it sits on those axes, and should resist the temptation to fill the portfolio with only one kind. Alon's second insight is that the problem should be chosen from the inside out: it should reflect what the scientist is curious about, not merely what is fashionable or fundable. That advice can sound naive in a competitive system, but it has a practical logic. An investigator who cares deeply about a question will keep working on it through the inevitable failures of the first years, will write about it with conviction, and will attract students who share that conviction. Search committees can tell the difference between a candidate who has found her question and one who has assembled a proposal to fit an advertisement. Several practical tests help turn curiosity into a viable program. The distinctiveness test asks whether anyone else is doing this. A postdoc can answer it more rigorously than most do. NIH RePORTER, the public database of NIH-funded projects, allows a search of active grants by keyword, method or organism, and shows who is funded to work on what, at what scale, and for how long. PubMed and bioRxiv searches over the last three years show who is publishing. If three well-funded laboratories are already attacking the same question with the same approach, the problem is not yours, however interesting it is. If nobody is working on it at all, that is worth pausing over too: the question may be unimportant, intractable, or simply undiscovered, and the postdoc should know which. The comparative-advantage test asks why you, specifically, are well placed to answer it. The strongest answers combine two things that rarely occur together: a technique from one field and a question from another, a model system nobody has applied to a problem, or a dataset nobody else has access to. Dr. A, the imaging specialist from the previous chapter, has an advantage in asking how immune cells make decisions in intact tissue because most immunologists who ask that question lack her imaging skills and most imaging specialists do not know the immunology. That combination is the kernel of a program. The fundability test asks who would pay for it. In the United States, the relevant question is which NIH institute, or which other funder, has a mission that the work serves. A program that falls between institutes, or that no study section has clear expertise to review, can be brilliant and still hard to fund. The test is not a reason to abandon an unusual idea, but it is a reason to frame it carefully, and to talk to program officers early about where it would fit. The ten-year test asks whether the problem can sustain a laboratory for a decade. A single elegant experiment is not a program. A program has a central question, several lines of attack, and a natural way to grow as results arrive. When a postdoc can describe what her laboratory will be doing in years one, five and ten, and why each follows from the one before, she has a program. Separating from the mentor's laboratory The central tension of the postdoc is that the work is done in someone else's laboratory, with someone else's money, often using someone else's reagents, animals and data, and yet the postdoc must emerge with something that belongs to her. Every field has stories of transitions that went badly because this tension was never addressed openly: a new assistant professor discovering that her former mentor is now competing directly with her, or that the mouse line at the heart of her program cannot leave the old institution, or that her K99 aims overlap with her mentor's newly funded R01. The only reliable way to avoid these outcomes is a direct conversation, early and repeated, about what will leave with the postdoc. Good mentors expect this conversation and are relieved when the postdoc raises it. Mentors who resist it are signalling something important about how the transition will go. The conversation has several components. The scientific scope of what the postdoc will pursue independently should be defined in terms both sides can recognise. A useful formulation is to agree which questions the postdoc will own, which the laboratory will continue, and where the boundary lies. For example: "I will continue working on how this cell type integrates signals in tissue; the laboratory will continue the biochemistry of the receptor complex; we will collaborate on any work that requires both." Written down, even informally in an email, this is far more durable than an understanding reached over coffee. The materials the postdoc will take require particular attention. Transgenic animals, cell lines, plasmids, antibodies, patient samples and datasets are often governed by material transfer agreements, institutional policies, data use agreements or consent restrictions that neither the postdoc nor the mentor fully controls. Genetically modified mouse lines, in particular, may need to be rederived or imported through a quarantine process at the new institution, which can take months. A postdoc planning to build her first aim around a specific line or dataset should confirm, in the first two years, that it can move, and at what cost and speed. Authorship and credit for work in progress at the time of transition should be discussed before, not after, the move. The common and sensible arrangement is that papers largely completed in the mentor's laboratory are published with the postdoc as first author and the mentor as senior author, and that the postdoc may be listed as a co-corresponding author if she has driven the project. Work begun in the mentor's laboratory and continued in the new one may be published jointly. What matters is that both sides know the plan. Future competition is the most delicate item. A mentor may reasonably continue working in the broad area the postdoc is entering, and the postdoc cannot expect a field to be vacated for her. But a mentor who deliberately launches a project directly on the postdoc's independent question, after she has left, damages the postdoc's career and the mentor's reputation. The explicit agreement on scope is the main protection. Strong mentors go further and actively steer their laboratories away from their former trainees' territory for a period, knowing that their own reputation rests in part on the success of the people they train. Bridge grants force this conversation in useful ways. The K99/R00 application requires the mentor to describe how the candidate's independent research will be distinct from the mentor's program and what the candidate will take with her, and study sections look for that statement. Writing it together is often the easiest way to reach an agreement. When the mentor will not agree Not every mentor makes this conversation easy. Some are reluctant to let any part of their program go, particularly a project that has become central to the laboratory. Some avoid the conversation rather than refuse it, repeatedly deferring it until the question becomes urgent. And some agree in principle but do not honour the agreement when it matters. A postdoc facing any of these situations has more options than she may think. The first is to make the conversation concrete. A general request to "take the project with me" is easy to deflect; a specific written proposal, listing the question, the materials, the authorship plan and the proposed boundary with the laboratory's continuing work, is much harder to ignore, and it gives the mentor something to amend rather than simply decline. The second is to involve others. Most universities now have a postdoctoral affairs office, and many departments have a faculty member or ombudsperson who handles exactly these disputes. Members of a postdoc's mentoring committee, if she has one, can raise the matter without making it personal. The third is to redesign the program so that it depends less on contested resources: using a publicly available mouse line rather than the laboratory's own, or building a new dataset rather than relying on one the mentor controls. If none of this works, the postdoc should recognise the situation for what it is. A mentor who will not allow a trainee any independent territory is a constraint that no amount of hard work will remove, and the rational response may be to move, either to a new postdoc or to a different position, while there is still time on the eligibility clocks. This is a hard decision, and it should not be made in anger. But it is better made in year two than discovered in year five. It is equally important to be fair to the mentor. Mentors have their own obligations to their funders, their other trainees and their institutions, and some of what a postdoc wishes to take may genuinely belong to the laboratory. A reasonable agreement usually involves compromise on both sides, and a postdoc who asks for everything is less likely to receive anything. The aim is not to extract the maximum but to leave with a coherent program and a mentor who remains an advocate. Structuring the program into aims Once the problem is chosen and its boundaries agreed, it needs to be structured into a form that can be funded, staffed and explained. In the NIH world that form is the Specific Aims page, a single page that states the problem, the gap, the central hypothesis and two to four aims. Even postdocs who are not writing an NIH application should draft one, because it disciplines thinking in a way that looser formats do not. A strong program for a new laboratory usually has three aims with different risk profiles. The anchor aim is the most feasible. It builds directly on preliminary data the postdoc already has, uses methods she has mastered, and will produce a publishable result within the first two to three years of the laboratory. Its function is to keep the laboratory productive while the riskier aims develop, and to give the first graduate student a project that will succeed. It should not be trivial, but it should not depend on anything going unexpectedly right. The extension aim takes the anchor result into new territory, often by applying it in a new system, a disease model, or a larger scale. It is moderately risky and is usually where the laboratory's second and third papers come from. The ambitious aim is the one that defines the laboratory's long-term identity. It may depend on a technique not yet fully working, or on a hypothesis that could be wrong. It is the aim that makes the program exciting, and it is the one that search committees remember. It should be designed so that partial success is still valuable; an aim that produces nothing unless everything works is a poor bet for a new laboratory. These three aims should not be independent. Reviewers of both grants and job applications react badly to aims that collapse if the first one fails, but they also react badly to aims that have nothing to do with each other. The best programs have aims that share a central question and a set of tools, and that would each advance understanding of that question even if the others stalled. A worked version for Dr. A might look like this. Her central question is how tissue-resident memory T cells in the skin decide whether to respond to a local signal or remain quiescent. Her anchor aim uses intravital imaging, which she already has working, to describe the behaviour of these cells in response to a defined local stimulus, and to test whether a particular receptor is required. Her extension aim applies the same approach to a model of chronic inflammation, asking whether the decision rules change. Her ambitious aim develops a genetically encoded reporter that reads out signalling in these cells in real time, allowing her to connect behaviour to signalling within single cells in living tissue. The three aims share a question and a toolkit. The first can produce a paper in two years. The third, if it works, is the kind of result that makes a laboratory's reputation. Preliminary data and the independent result The critical asset for any postdoc planning independence is what might be called the independent result: a finding, obtained during the postdoc, that is the postdoc's own idea, belongs clearly to her proposed program, and could only have come from her. It does not need to be published in a prestigious journal. It does need to exist, be convincing, and be clearly separable from the mentor's program. Many postdocs misjudge what this result needs to be. They believe it must be a finished paper, and they wait until the paper is accepted before beginning their applications. In practice, a solid preliminary result, clearly presented, is enough to anchor a K99 application and a faculty job talk. A well-constructed preprint serves the same purpose. The finished paper can arrive during the application cycle, and ideally does. Two rules help. First, the independent result should be generated early, ideally within the first two years, because it is the foundation on which the grant and the job application rest. Second, it should be protected: the postdoc should be first author, should have primary control over how it is presented, and should be free to discuss it in talks outside the laboratory. A result that the mentor presents at every meeting as the laboratory's work will not be perceived as the postdoc's, however clear the internal credit. A postdoc who, eighteen months into her position, cannot point to such a result, or cannot see how to obtain one within the next year, should treat that as a signal. It may be time to change projects, to reorganise effort, or to have a frank conversation with the mentor. The cost of adjusting course early is small. The cost of discovering in year four that there is nothing independent to present is very large. When the trajectory changes Not every trajectory survives contact with data. A key experiment fails, a competing laboratory publishes the central result, or a new finding opens a more interesting direction. Changing course is not a sign of failure, and the most successful investigators often pivot at least once during their training. What matters is to pivot deliberately. A good pivot preserves the postdoc's comparative advantage. If Dr. A's imaging approach fails in skin, the right move is probably to apply it to another tissue or cell type where it can work, not to abandon imaging for a technique in which she has no distinctive skill. A good pivot also preserves the story. Committees are comfortable with candidates whose programs evolved, as long as the evolution follows a clear logic that the candidate can explain in two sentences. A bad pivot is one driven by fashion or panic: abandoning a coherent program to join a crowded field because it looks more fundable, or chasing a new method because a competitor has adopted it. Such pivots usually cost more time than they save and produce a candidate whose story no longer hangs together. The test for any proposed pivot is simple. After the change, can you still state your question, your advantage and your ten-year vision in a paragraph that a stranger would find coherent and interesting? If so, the change is probably sound. If not, it probably is not. Chapter 3. The K99/R00 Pathway to Independence Award For biomedical postdocs in the United States, the NIH Pathway to Independence Award, known by its activity codes K99/R00, is the best-known bridge from trainee to independent investigator. It is not the only route, and many excellent faculty never held one. But it has a unique design, it is open to a wider pool of applicants than most NIH career awards, and it shapes how search committees read applicants who have one and applicants who do not. This chapter explains how the award works as of 2026, what reviewers look for, how the last two years of policy changes affect it, and how to decide whether and when to apply. How the award is built The K99/R00 has two phases, and its design is its main advantage. The K99 phase is a mentored award of one to two years that supports the postdoc while she completes her training in the mentor's laboratory. The R00 phase is an independent research award of up to three years that activates only when she accepts an independent, tenure-track or equivalent faculty position at an institution willing to host it. Together the phases can provide up to five years of support. The R00 moves with the investigator: it is not tied to the institution where she held the K99, and a K99 awardee on the job market is effectively offering any hiring department three years of independent NIH funding in advance. Table 1 summarises the two phases under the parent announcement for applications not involving an independent clinical trial, PA-24-194, which with its sister announcements PA-24-193 and PA-24-195 governs most K99 applications as of 2026. Individual institutes set their own salary and research-support limits within the parent announcement, so the K99 figures should be checked against the relevant institute's table before an application is budgeted. Table 1. The two phases of the K99/R00 (parent announcement PA-24-194 and institute guidance, as of 2026). Feature K99 phase R00 phase Status Mentored postdoctoral Independent faculty Duration 1 to 2 years Up to 3 years Effort At least 75% (9 person-months) Set by the research plan Direct support Salary plus research development funds, capped by each institute Up to $249,000 total costs per year Indirect costs 8% of modified total direct costs Host institution's negotiated rate, within the cap Host Mentor's institution Any institution offering an independent position The R00 figure is a total-cost ceiling, which means indirect costs are included within it. At an institution with a high negotiated indirect rate, the direct research budget available to the new investigator is correspondingly smaller. That arithmetic matters in the startup negotiation discussed in Chapter 6, because an R00 at a high-overhead institution buys less science than the headline figure suggests, and the startup package must make up the difference. Eligibility, the four-year clock, and the extensions The central eligibility rule is the four-year limit. Under the parent announcements, applicants may have no more than four years of postdoctoral research experience at the time of the initial application or of the resubmission. The clock generally counts research experience after the doctoral degree is conferred, in any laboratory and under any job title, not only time in the current position; the announcement defines exactly what counts, and time in clinical training may be treated differently from research time. Extensions are possible for specified reasons, including childbirth, parental leave, family care, medical conditions and military service, and are requested from the institute before submission with supporting documentation. The practical consequence is that a postdoc who wants the possibility of a resubmission should submit her first application well before the end of her third postdoctoral year. NIH application cycles for K awards have standard due dates in February, June and October, and a typical first submission receives its score about four months later. A resubmission then requires another cycle, and there is only one resubmission permitted per application. Working backwards, a postdoc whose clock expires in month 48 should submit her first K99 by around month 30 to 33 to leave room for review, revision and resubmission within the window. The unusual feature of the K99 among NIH career awards is its openness to non-citizens. Temporary visa holders are eligible, which makes it an important route for the large number of international postdocs in American laboratories. Applicants who are not citizens or permanent residents should check the institution's documentation requirements for each phase and plan the visa transition that the move to a faculty position will require. Candidates must be in a mentored postdoctoral position at the time of application and at the time of the K99 award. A postdoc who has already accepted a tenure-track faculty position is no longer eligible for the K99, which is why the grant should be written before the job search, not during it. Applicants who hold other substantial independent funding or an independent faculty-level appointment are also excluded. The disruptions of 2025 led NIH to adjust the clock twice. In June 2025, notice NOT-OD-25-121 granted a temporary extension to applicants whose four-year window would have expired between October 2024 and January 2026, recognising that delays in application processing, peer review and awards between January and May 2025 had disproportionately affected early-career applicants; eligible applicants received up to three additional receipt cycles, with a final deadline in October or November 2025. In January 2026, notice NOT-OD-26-021 automatically extended eligibility again for applicants whose windows closed around the federal appropriations lapse of autumn 2025, giving applicants whose final deadline fell in September to November 2025 until the February or March 2026 cycle, and those with deadlines between January and March 2026 until the June or July 2026 cycle. These were one-time measures. They illustrate a general point: when the system is disrupted, NIH has so far been willing to protect the eligibility of early-career applicants, but only through specific notices that must be read and applied carefully. What reviewers are looking for K99 applications are reviewed by study sections that assess the candidate, the career development plan, the research plan, the mentor and co-mentors, and the environment and institutional commitment. Each reviewer provides an overall impact score, and the application then goes to the institute's advisory council. The written application has a combined page limit, currently twelve pages, that covers both the candidate's career development plan and the research strategy, plus a one-page Specific Aims page, a mentor's statement, letters of reference, and an institutional commitment letter. Several features of successful applications recur. The candidate section tells a coherent story of scientific development that explains why this person, at this moment, is ready to become independent. It is not a chronological CV. The strongest candidate sections identify the specific skills the candidate has, the specific gaps she still needs to close, and how the K99 period will close them. A candidate who claims to need nothing further is not a mentored applicant; one who needs everything is not ready. The career development plan is concrete. It names the courses, workshops, conferences, techniques and mentoring meetings the candidate will undertake, with a timeline and measurable outcomes. It includes training in the responsible conduct of research, which is required. It explains how the candidate will prepare for the job market during the K99, including the timing of applications. Vague plans to "attend seminars and develop leadership skills" are a common weakness. The research strategy is typically organised with a K99 aim or aims to be completed in the mentor's laboratory and R00 aims to be carried into the independent position. Reviewers want to see that the K99 aim provides genuine new training, not simply more of what the candidate already knows, and that the R00 aims form the kernel of an independent program distinct from the mentor's. The design principles from Chapter 2 apply: a feasible anchor, an extension, and an ambitious aim that could define a laboratory. The mentor's statement is read closely. It should describe the mentor's record of training postdocs who became independent, the specific support the mentor will provide, and above all what the candidate will take into the independent position. The mentor should say explicitly that the candidate's R00 program is hers and will not be continued as a competing project in the mentor's laboratory. Reviewers are alert to mentor statements that sound as though the mentor expects to retain the program, and they penalise them. Co-mentors and advisory committees are common and useful, especially when the K99 aim involves new training outside the primary mentor's expertise. A co-mentor with a clearly defined role, meeting schedule and relevant expertise strengthens the application. A long list of distinguished names with no defined role does not. The institutional commitment letter from the department chair or dean should confirm the candidate's protected time and access to resources during the K99 phase. It does not commit the institution to hiring the candidate, and reviewers do not expect it to. What the data say about outcomes The most thorough recent analysis of K99/R00 outcomes is by Nicole Woitowich, Rebecca Hengel, Anna Solis, Tauras Vilgalys, Joel Babdor and Daniel Tyrrell, published in eLife in 2024. Using NIH records from 2006 to 2022, they counted 3,456 extramural K99 awards and 2,842 R00 awards, about 230 K99 awards per year on average, and found that roughly 78 percent of K99 awardees activated an R00. Their data show several patterns that postdocs should know. Awards are concentrated. About half of K99 awards went to about a tenth of the institutions receiving them, with Stanford, UCSF and Massachusetts General Hospital receiving the most. R00 awards were less concentrated, because many awardees moved from elite training institutions to a broader set of hosting institutions. Moving matters. Awardees who took their R00 to a different institution activated it at a much higher rate, about 94 percent, than those who stayed at their K99 institution, about 69 percent. And those who stayed took longer to win their next major independent award. The authors estimated that the time from R00 to a subsequent R01, DP2 or R35 was a median of about 4.6 years in the most advantaged combination of factors and about 7.4 years in the least, with staying at the same institution among the largest single disadvantages. Gender did not change the conversion rate from K99 to R00 in their data, though men received about 58 percent of awards and women about 40 percent. Awards to postdocs at historically Black colleges and universities were essentially absent. These figures are descriptive, not causal, and the authors are careful about that distinction. But they reinforce two practical conclusions. First, the K99 is a strong predictor of successful transition, though not a guarantee of it. Second, the R00 is most valuable when used to move to a new institution where the investigator can establish a clearly independent identity. The 2025 to 2027 policy landscape The K99 has operated in an unusually turbulent environment since early 2025, and a postdoc planning an application in 2026 or 2027 should understand the main changes. The MOSAIC program, which since 2019 had offered a K99/R00 variant for postdocs from backgrounds underrepresented in biomedical research, was terminated in 2025. Its awards had provided K99-phase support and up to $249,000 a year in the R00 phase, along with cohort-based career development. STAT reported that about 104 awardees had their funding terminated, some later restored by court order for plaintiffs covered by specific rulings. No new MOSAIC applications are accepted. Postdocs who would previously have applied to MOSAIC should apply through the parent K99/R00, and those whose MOSAIC awards were affected should speak with their program officers about their status. Application limits now apply. Notice NOT-OD-25-132, issued in July 2025, limited each principal investigator to six new, renewal, resubmission or revision applications per calendar year across most activity codes, and stated that NIH will not consider applications substantially developed by artificial intelligence to be original ideas of the applicant. For most K99 applicants six applications is not a binding constraint, but the AI provision is relevant: the application must be the candidate's own work, and a candidate who drafts with AI assistance should understand the policy and her institution's guidance before doing so. Funding strategy has also changed. In 2025 and 2026 NIH moved away from published paylines toward a unified strategy in which councils weigh scientific merit alongside portfolio balance and other factors, and several institutes stated that early-stage investigators and trainees would continue to receive special consideration. The practical effect for K99 applicants is that a percentile score no longer tells an applicant with any precision whether she will be funded. The program officer's view, requested after the summary statement arrives, matters more than it once did. The National Cancer Institute runs its own K99/R00 announcements outside the parent program, with terms that have at times differed from the parent announcements in eligibility window and budget. Postdocs whose work falls within NCI's mission should read the current NCI announcement rather than the parent one. The biggest structural change is still ahead. In August 2026, notice NOT-OD-26-098 announced that NIH is consolidating its career development awards into a smaller set of parent announcements. The K99 and K22 activity codes will be combined into a new NIH Career Transition to Research Independence Award, activity code KT1, paired with the R00, for application due dates on or after 12 October 2027. The current K99 announcements are due to expire in July 2027. Under the new structure, K awards will no longer require a Data Management and Sharing Plan, and K award funds may not be used to conduct a clinical trial, though awardees may gain trial experience within a mentor's trial. The notice did not announce changes to eligibility or budget, but postdocs planning to apply after the transition should expect the first KT1 announcements to contain details that matter, and should read them as soon as they appear. Deciding whether and when to apply Not every postdoc should apply for a K99, and some who should apply do not. A few questions help. Is there an independent program? If the candidate cannot yet describe R00 aims that are clearly her own, the application will be weak, and the effort might be better spent generating the independent result. But the act of drafting the aims page is itself one of the best ways to discover whether such a program exists. Is there time? A postdoc near the end of her eligibility window, without preliminary data, may be better advised to apply for a foundation award or go directly to the job market. A postdoc early in her window has time to submit, revise and resubmit, and should use it. Does the mentor support it? The application needs a strong, specific mentor's statement, and the mentor must accept that the R00 program will leave with the candidate. A mentor who will not write that statement is telling the candidate something. What will the award buy? For some postdocs, particularly those moving to institutions with modest startup budgets, three years of R00 funding may be the difference between a viable laboratory and a precarious one. For others at wealthy institutions it is valuable mainly as a signal. Either way, it is also the best available rehearsal for writing the R01 that will follow. The strongest general advice is to start early. The best K99 applications are drafted over six months or more, read by the mentor, by co-mentors, by colleagues who have held K99s, and by at least one person outside the field, and discussed with a program officer before submission. A program officer can say whether the science fits the institute, whether the candidate's timeline is plausible, and what recent study sections have liked and disliked. That call costs twenty minutes, and there is no better investment in the application. Hashtags: #ThePostdoctoralRoadmap #PostdoctoralTraining #ResearchIndependence #IndependentInvestigator #CareerDevelopment #AcademicCareerPlanning #FacultyJobMarket #IndependentResearchProgram #ResearchProgramDesign #K99R00 #PathwayToIndependence #NIHCareerDevelopment #KT1 #EarlyStageInvestigator #SpecificAims #PreliminaryData #MentorTransition #ResearchMentorship #ScientificIdentity #FacultyApplications #JobTalk #StartupNegotiation #LabStartup #GrantStrategy #FutureOfResearchIndependence

Download the Book (PDF): Introduction A principal investigator I once worked alongside kept a note taped inside a desk drawer where only she could see it. It read: The experiment is not the only thing under your control. She had written it during a year when two of her doctoral students left within four months of each other — one to a job in consulting, one simply away, without a plan — and she had spent the months afterwards trying to work out whether she had caused it. Her conclusion, arrived at slowly, was that the question was badly framed. She had not caused it in the way one causes a car crash. She had caused it the way a badly designed building causes people to get lost: not through a single act, but through a set of arrangements that made certain outcomes far more likely than others, and which she had never thought of as arrangements at all. That is the argument of this booklet. The distress, attrition, and quiet disengagement that run through academic research are not, in the main, a story about individuals who lack resilience. They are a story about how research groups are organised, and they are therefore a leadership problem — which means they are tractable. This is an uncomfortable reframing for most of us who run laboratories, because it moves responsibility toward the person holding the budget and the reference letter. It is also, in my experience, the only reframing that leads anywhere useful. If burnout among early-career researchers is fundamentally a matter of personal constitution, then the leader's role is limited to sympathy and signposting. If it is substantially a matter of workload design, feedback structures, role clarity, credit allocation, and the predictability of a career path, then the leader's role is to build something different. The evidence, which we will look at properly in the first chapter, points firmly toward the second reading. What the evidence actually says The picture assembled over the last decade is consistent enough that arguing about it has become unproductive. Levecque and colleagues, surveying doctoral researchers in Flanders, found that roughly one in two showed signs of psychological distress and about one in three was at risk of a common psychiatric disorder — rates markedly higher than in comparably educated employees outside academia (Levecque and colleagues, Research Policy, 2017). Evans and colleagues, surveying graduate students internationally, reported moderate to severe anxiety and depression at rates several times higher than in the general population (Evans and colleagues, Nature Biotechnology, 2018). Nature's periodic surveys of doctoral researchers and postdocs have returned similar findings, along with persistent reports of bullying and of career uncertainty as a dominant source of strain. What matters for a leader is not the headline prevalence but what predicts it. Levecque's analysis is the crucial one here, because it went looking for the correlates and found them in the structure of the work rather than in the characteristics of the person: high job demands combined with low job control, an unsupportive or laissez-faire supervisory style, a perceived lack of decision-making input, and career uncertainty. These are not personality traits. They are design parameters. Every one of them is something a principal investigator or a department head sets, whether deliberately or by default. The World Health Organization's revision of the International Classification of Diseases made a related move in 2019, classifying burnout not as a medical condition but as an occupational phenomenon — a syndrome arising from chronic workplace stress that has not been successfully managed. The wording is precise and worth dwelling on. Burnout is defined by its relationship to the workplace. It is not a diagnosis you carry into a job; it is something a job does to you over time. What this booklet is, and is not This is a practical guide for people who run research groups and the departments that contain them. It assumes you are technically excellent — you would not have the job otherwise — and that nobody ever trained you to manage people, because almost nobody in this profession is trained to manage people. It assumes you have limited time, competing obligations, and a funding environment that rewards output on a timescale shorter than the one on which human development actually happens. It is not a clinical text. Where mental health appears in these pages, it appears from the vantage point of a manager, not a clinician, and the guidance is correspondingly bounded: notice, respond humanely, adjust what is within your control, and connect people with professionals who are qualified to help. A principal investigator who tries to become their trainee's therapist has made an error of role that usually ends badly for both. Chapter 8 addresses this boundary directly, because getting it wrong in either direction — intruding, or ignoring — is one of the more common ways that well-meaning leaders cause harm. Nor is it a book about being nice. Several of the most protective things a leader can do are uncomfortable: telling a fourth-year student plainly that the project is not working, declining to let a postdoc take on a fifth side collaboration, saying out loud in a lab meeting that the probability of a faculty position is low and that this is not a judgement on anyone's worth. Kindness that withholds information is not kindness. Much of what follows is about how to be structurally generous and conversationally honest at the same time. The shape of the argument The booklet moves from diagnosis to design. Chapter 1 sets out the case that the laboratory is a system with knowable failure modes, and introduces the framework — the areas of working life along which strain accumulates — that organises much of what follows. Chapter 2 examines the peculiar job of the principal investigator, and the transition that nobody prepares for: from being the person who does the science to being the person who makes the science possible. The middle chapters are the working core. Chapter 3 is about making expectations explicit — handbooks, onboarding, development plans, and the hidden curriculum that advantages students who already know how universities work. Chapter 4 treats mentorship as a function to be distributed rather than a relationship to be heroically supplied by one overstretched person, and takes seriously what inclusive mentorship requires in practice. Chapter 5 concerns psychological safety: what it is, what it is not, and why it is simultaneously a wellbeing measure and a research integrity measure. Chapter 6 confronts workload, hours, and the durable myth that long hours are what produces good science. Chapter 7 addresses the things that go wrong between people — authorship disputes, conflict, and the misuse of the considerable power a supervisor holds. Chapter 8 concerns noticing distress and responding well without stepping outside your competence. Chapter 9 argues that a lab should be built for departure: that the measure of a training environment is what happens to people afterwards, and that career honesty is a wellbeing intervention. Chapter 10 turns to the department head, because a great deal of what constrains a well-intentioned PI is set above their head, and some problems cannot be solved one laboratory at a time. Throughout, I have tried to give specific practices rather than principles. Principles are easy to agree with and impossible to act on. A statement like "foster a culture of psychological safety" is not advice; it is a wish. What a person can actually do on Monday is change how the lab meeting runs, change what the first question after a presented result is, change who speaks first and who speaks last. Those changes are small, cheap, and they compound. A word on incentives One objection deserves answering at the outset. It runs: all of this is admirable, but the system does not reward it. Grants are won on preliminary data and publications, not on the wellbeing of the people who generated them. Promotion committees ask about impact factors. A PI who invests heavily in mentorship is investing in something invisible to every committee that decides their future. There is real force in this. It would be dishonest to pretend that good lab leadership is straightforwardly rewarded, and Chapter 10 is largely about why department heads bear a specific obligation to change that. But the objection also overstates its case in a way that is worth naming, because it functions as permission to do nothing. A lab that loses a fourth-year student loses three to four years of accumulated tacit knowledge, an unfinished paper, and usually the momentum of whoever inherits the project. A postdoc who spends their final eighteen months disengaged and job-hunting in secret produces a fraction of what they otherwise would. Recruitment is expensive and slow. Reputational effects in a subfield are real and travel fast; graduate students talk to each other with a candour they never use with faculty, and a group's reputation among prospective students is set long before any of them apply. The claim that good management is a costly indulgence rests on an accounting that only counts one side of the ledger. There is also the plain fact that most people entered this profession partly because they wanted to make something, and partly because they wanted to help others make something. Running a group in which capable people are ground down is not what anyone signed up for. That is not a strategic argument. It does not need to be. How to read this The chapters are ordered, but they are not sequential in the sense that you must read them in order to make sense of them. If your immediate problem is a conflict over authorship, start at Chapter 7. If you are about to take on your first doctoral student, Chapters 2 and 3 are where the leverage is. If you are a newly appointed head of department wondering what levers you actually hold, Chapter 10 will be the most direct, though it will make more sense against the background of what comes before. Nothing here requires a budget line. Almost everything here requires attention, which is the scarcer resource. The single most useful thing a leader can do — noticing the difference between someone who is working hard and someone who is coming apart — costs nothing but the willingness to look. Chapter 1: The Lab Is a Designed System Every research group is designed. The design is rarely written down, rarely deliberate, and rarely recognised as design at all, but it exists. It consists of who reports to whom, how often, and in what setting; what counts as an acceptable reason to be away; who gets the good samples, the working instrument, the shared postdoc's time; what happens to a result that contradicts the lab's published position; whose name goes where on the paper, and who decides; what the leader visibly rewards and what they merely say they value. These arrangements produce outcomes as reliably as a protocol produces a readout. The difference is that nobody writes them down, so when they produce bad outcomes, we look for an explanation in the person they happened to. The consequence is a persistent misattribution. A student who stops making progress is described as having lost motivation. A postdoc who becomes withdrawn is described as not handling pressure well. A technician who leaves is described as having wanted something different. Each of these descriptions may be locally true. Collectively, they constitute a way of not seeing the system. When the same outcome appears in the same lab across different people, the variable that is constant is not the person. Where the strain actually comes from The most useful framework for locating strain comes from Christina Maslach and Michael Leiter, whose work on burnout has run for four decades and who converged on a model that treats burnout as a mismatch between a person and six areas of their working life: workload, control, reward, community, fairness, and values. The framing matters because each area names a specific lever rather than a mood. It is possible to ask, concretely, where the mismatch sits. Consider each in a laboratory setting. Workload is the obvious one, and the one most often mistaken for the whole story. But the mismatch is rarely simply "too much." It is more often workload that is unpredictable, or that consists largely of tasks the person cannot see the point of, or that arrives without any corresponding removal of something else. A doctoral student asked to take on a collaborator's assay on top of a full project is experiencing a workload problem; so is one whose experiments require attendance at eleven at night three times a week with no recognition that this is unusual. Control is the variable Levecque's data implicates most sharply, and it is the one academia handles worst. Karasek's demand–control model, formulated in the late 1970s and validated many times since, holds that high demands are tolerable — even energising — when paired with high control, and corrosive when paired with low control. Research work is superficially high-control: nobody is standing over a student telling them which pipette to pick up. But the decisions that actually matter — what question to pursue, when a result is good enough, whether a project continues, when to write up — are frequently held entirely by the supervisor, and often held without ever being discussed. The trainee has enormous autonomy over means and almost none over ends. This is precisely the configuration the model predicts will hurt. Reward in a laboratory is mostly not money. It is authorship, conference presentations, invited talks, a supervisor's public acknowledgement, the sense that work was seen. These are cheap to give and are frequently withheld through simple inattention. A PI who praises in private and never in the lab meeting has created a reward mismatch without intending anything. Community covers the texture of daily relationships — whether people eat together, whether the group is composed of collaborators or competitors, whether there is anyone to ask a stupid question of. Labs that run on internal competition for resources or for the PI's attention generate a particular kind of exhaustion that no amount of reduced hours will fix. Fairness is about whether the rules are applied consistently. It is where favouritism, opaque authorship decisions, and unequal enforcement of expectations do their damage. Fairness mismatches are especially corrosive because they undermine the rationale for effort: if outcomes are not connected to contribution, the incentive to contribute collapses. Values is the mismatch between why someone came and what they find themselves doing. A person who entered science to understand something and spends three years generating figures for a grant renewal is experiencing a values mismatch. So is one who is asked to present data more confidently than the data support. The practical use of this list is diagnostic. When a group member is struggling, the question "which of the six?" is far more productive than "what is wrong with them?" It usually produces an answer, and the answer usually points at something the leader can change. Demand, control, and the particular shape of research work It is worth being precise about why research work is unusually prone to this failure, because the diagnosis determines the remedy. First, the feedback interval is long and the failure rate is high. In most professions, effort produces visible output on a timescale of days or weeks. In research, a person can work competently for eight months and produce nothing publishable, not because they did anything wrong but because the hypothesis was false or the technique did not transfer. This is normal and it is also psychologically punishing, particularly for people whose entire prior education rewarded them reliably for effort. The transition from a world in which hard work produces an A to a world in which hard work produces a negative result is one of the most common sites of early doctoral distress, and it is almost never named explicitly. Second, the boundary between work and self is unusually thin. Research is chosen, often at real financial cost, by people who find it meaningful. That is a resource — meaning buffers strain — but it is also a vulnerability, because a failed experiment is easily experienced as a verdict on the person rather than on the hypothesis. A group culture that distinguishes clearly between the quality of the work and the worth of the worker is doing genuine protective work. Third, the career structure is a funnel that most people do not know they are in. The number of doctoral graduates produced substantially exceeds the number of permanent academic positions available, and has for decades. This is not a secret, exactly, but it is frequently left unspoken in individual groups, so trainees calibrate their expectations against the visible example of their supervisor — a survivor of the funnel — and interpret their own uncertain prospects as personal shortfall. Alberts, Kirschner, Tilghman and Varmus set this out plainly in 2014 in their analysis of the systemic flaws in US biomedical research: a system dependent on a large, cheap, temporary workforce with poor prospects of advancement produces predictable hypercompetition and predictable damage. Fourth, the supervisor holds an unusual concentration of power. In most employment relationships, a bad manager is survivable: you can move teams, appeal to HR, leave with a reference from someone else. A doctoral supervisor controls the project, the funding, the timeline, the assessment, the reference letter, and often the visa. There is no realistic exit that does not cost years. This asymmetry is the single most important structural fact about the relationship, and Chapter 7 returns to it, because it changes the meaning of behaviours that would be unremarkable between equals. The failure modes are recognisable Groups fail in patterned ways. A leader who can recognise the pattern early has a considerable advantage, because most of these are far cheaper to correct at month three than at year three. The vacuum. The PI is absent — travelling, writing grants, running a second lab, or simply not thinking about people. Supervision happens by exception, usually when something goes wrong. Trainees fill the vacuum with guesswork about what is expected, and the guesses are systematically pessimistic. This is the laissez-faire supervisory style that Levecque's data identified as a risk factor, and it is worth stressing that it is a risk factor: not neutral, not merely suboptimal. An absent supervisor is actively harmful, and many absent supervisors believe they are being respectfully hands-off. The furnace. The opposite configuration: constant presence, high demands, visible displeasure at insufficient output, an explicit or implicit norm of long hours. This produces impressive short-run productivity and a characteristic profile of attrition — people leave at the end of a project, exhausted, and the group's alumni do not recommend it. Furnace labs frequently have excellent publication records, which is why they persist. The court. Access to the PI is the scarce resource, and it is unevenly distributed. There is an inner circle, usually people who are similar to the PI in background or style, and an outer one. Members of the inner circle get the good projects, the timely feedback, the introductions. This mode is rarely deliberate and is very often invisible to the person at the centre, who experiences it simply as some people being easier to work with. It is a fairness failure and, because similarity tracks demographics, it is frequently also an equity failure. The museum. Everything is calm, nothing changes, projects extend without clear endpoints, and nobody is quite sure when a student will finish. Museums are pleasant places to be for a while and destructive over years, because the drift consumes exactly the career stage at which people most need momentum. The archipelago. Each person works on an isolated project with no scientific overlap. There is no one in the group who understands what anyone else is doing, so peer support is impossible and the PI is the only possible source of technical help. Archipelago labs generate loneliness and a hard bottleneck at the supervisor. Most groups are not purely one of these. Most are a vacuum on Tuesdays and a furnace before a deadline, or a court with an archipelago in the corner. The value of naming them is that each suggests a different intervention: the vacuum needs scheduled contact, the furnace needs explicit limits from the top, the court needs structured rotation of attention, the museum needs milestones, the archipelago needs overlapping subgroups. What "resilient" should mean The word resilience has been badly used. In much institutional language it has come to mean a capacity individuals are supposed to acquire, usually through a workshop, so that they can withstand conditions that remain unchanged. Used that way it is an instrument of blame with a therapeutic accent, and researchers hear it as such. When a university responds to a workload crisis with a resilience training module, the message received is: the conditions are fixed, and your inability to tolerate them is a deficit. The engineering sense of the word is better, and it is the sense used in this booklet. A resilient system is one that continues to function when a component fails. It has redundancy, it degrades gracefully, it does not have single points of failure, and it recovers. Applied to a laboratory, that means something quite specific. A resilient lab is one where a student's progress does not halt because the PI is travelling for three weeks, because someone else can advise them. It is one where a failed project does not end a career, because the milestones were set so that failure was detected at eighteen months rather than forty. It is one where a person going through a crisis — a bereavement, an illness, a family emergency — can step back without the project collapsing and without the group's opinion of them changing. It is one where a bad result can be reported immediately, because reporting it is safe, so that four months are not wasted building on it. It is one where the departure of any single person, including the PI, does not destroy anyone else's work. None of that is about individual toughness. All of it is about how the group is built. The leader's actual leverage It is worth being honest about the limits. A principal investigator does not control the funding climate, the number of faculty positions, the immigration rules that constrain an international postdoc's options, the university's mental health service waiting times, or the fact that publication remains the currency of the field. Pretending otherwise produces a guilt that is both unearned and paralysing. But the list of things a PI does control is longer than most PIs behave as though it is. The frequency and format of one-to-one meetings. Whether expectations exist in writing. Who has a second mentor. What happens in the first ninety seconds after someone presents a negative result. Whether authorship criteria are stated before the work starts or negotiated after it. Whether holidays are taken. Whether the lab's norm of hours is set by the PI's own visible behaviour, which it invariably is. Whether a struggling person is spoken to at week three or at month nine. Whether the group knows where the counselling service is and that using it is unremarkable. Whether career conversations happen annually or never. Each of those is a design choice. Most of them cost nothing beyond attention and a small amount of discomfort. Their aggregate effect on whether a capable person flourishes or fractures is, on the evidence, larger than anything else within reach. The rest of this booklet is about making those choices deliberately. A concrete case An example makes the abstraction usable. Consider a structural biology group of nine: the PI, two postdocs, four doctoral students, a technician, and a rotating master's student. Over three years the group publishes well. Over the same three years, one student transfers to another supervisor, one postdoc leaves science entirely, and the technician resigns. From inside, each departure had its own explanation. The student "was not suited to the technical demands." The postdoc "wanted a more stable life." The technician "got a better offer." Now read the arrangements rather than the people. The group's only working cryo-EM access was two overnight sessions a week, allocated by the PI, and allocation was never explained; two of the four students had twice the beam time of the other two. That is a fairness mismatch and a control mismatch in one arrangement. The lab meeting was structured so that presenting a week of failed optimisation was visibly unwelcome — the PI's habitual first question was "so what worked?" — so nobody reported problems until they were unfixable. That is a safety failure that also cost the group months of duplicated troubleshooting. The technician was the only person who knew how to service the ancillary equipment, and had been for four years, with no cover and no development. That is a workload and reward mismatch, and also a single point of failure in the engineering sense. None of this required a villain. The PI was decent, well liked, and would have been horrified at the reading above. The arrangements simply accumulated, each one locally sensible, and together they produced a group in which three of nine people departed under strain in three years. Read individually the departures look like three separate stories about three particular people. Read structurally they look like one story about a system with two allocation problems, a reporting problem, and a redundancy problem — all four of which could have been fixed in an afternoon by someone who was looking. The discipline worth cultivating is exactly that switch of frame. When something goes wrong with a person, ask what arrangement made it likely. The answer is not always there, but it is there often enough to be worth the habit. Why this is not the same as excusing anyone A structural account sometimes attracts the objection that it dissolves responsibility — that if the system produces the outcome, nobody is accountable for it. The objection has things backwards. Individual accounts are the ones that dissolve responsibility, because they locate the cause in a person who has already left. Structural accounts locate the cause in arrangements, and arrangements have owners. If a group's allocation of instrument time is generating resentment, someone decides that allocation, and that someone can decide differently on Monday. If reporting a negative result is unsafe, someone sets the tone in the meeting. The structural frame is more demanding of leadership, not less, because it removes the comfort of attributing every bad outcome to the idiosyncrasies of the person it happened to. It also does not deny that individuals differ. People arrive with different histories, different vulnerabilities, different resources. Two people in identical conditions will not have identical experiences of them. But that variation is mostly not something a leader can change, whereas the conditions are. A leader who concentrates on the modifiable variable is not denying the existence of the others; they are doing the part of the job that is theirs. Chapter 2: The Accidental Manager The career path into laboratory leadership is unique among professions in its indifference to whether the person can lead. A surgeon is assessed on operating. A pilot is assessed on flying. A principal investigator is selected on the strength of their science and then spends most of their working life doing something else — hiring, budgeting, allocating, mediating, developing people, deciding whose project lives and whose dies — with no training, no supervision of their own, no feedback loop worth the name, and no professional norm that treats any of this as a skill. It is worth pausing on how strange that is. A newly appointed assistant professor in a mid-sized institution will, within eighteen months, typically be responsible for the professional development of three to six adults, the deployment of several hundred thousand in grant funding, the safety of a physical space, and the scientific direction of a programme. In any other sector, a person taking on that scope would receive management training as a matter of course. In academia the expectation is that it will be absorbed by osmosis from one's own supervisor, who absorbed it the same way, back through a chain of untrained people to the beginning. Osmosis is a poor transmission mechanism for anything, and a particularly poor one here, because the sample is biased. The PIs you learned from are, by definition, the ones whose approach produced you. You did not observe the counterfactual: the students who left, the ones who stayed and quietly stopped trying. Nor did you observe the approaches of the fifty other labs in your field. Most of us are running a management style inherited from one or two people, unexamined, and mistaking familiarity for validation. Four jobs in one appointment The role decomposes into four jobs that draw on different skills and compete for the same hours. The first is scientific director: choosing the questions, judging the quality of evidence, deciding when a line is exhausted. This is the job the appointment was made for, and the one most PIs are genuinely good at. The second is people manager: recruiting, onboarding, allocating work, giving feedback, developing capability, managing underperformance, handling conflict, and ending relationships when they must end. This is the job this booklet is mostly about, and the one almost nobody was prepared for. The third is resource acquirer: grants, fellowships, equipment, industrial partnerships, institutional negotiation. It is relentless, largely thankless, and consumes an amount of time that early-career PIs consistently underestimate. The fourth is institutional citizen: committees, teaching, review, editorial work, outreach. It expands to fill whatever space is left and is distributed unevenly, falling disproportionately on women and on researchers from underrepresented groups — the phenomenon sometimes called the minority tax, where those least numerous in a department are asked most often to represent it. Nobody does all four well simultaneously. The question is not how to excel at everything but which failures you are choosing. The default failure, in my observation, is that the people-management job is the one that gets cut, because it is the only one of the four with no external deadline. A grant has a submission date. A committee has a meeting. A paper has a referee waiting. A conversation with a struggling second-year has nothing forcing it, and so it slides, week after week, until it becomes a conversation about leaving. The single most consequential management decision an early-career PI makes is therefore an unglamorous one: to give the people job a fixed place in the calendar so that it stops being the residual. What actually changes at the transition Several things invert at the move from postdoc to PI, and the inversions catch people out. Your output is now other people's output. The instinct of a good scientist under pressure is to do the work themselves, faster and better. As a PI this instinct is destructive twice over: it consumes the time you needed for the things only you can do, and it removes from a trainee the experience of struggling through something, which is the mechanism by which they become capable. Taking the keyboard is the commonest failure of new PIs and the hardest to stop, because it feels like diligence. Everything you say carries weight you did not intend. A speculative remark — "I wonder whether we should be doing more on the kinase side" — is heard by a second-year as an instruction, and they will reorganise six weeks around it. A flicker of visible disappointment at a result is remembered for a year. Silence is read as displeasure. This amplification is a direct function of the power asymmetry, and it does not switch off because you are informal, young, or on first-name terms. The remedy is not to become guarded but to become explicit: label speculation as speculation, and say what you actually mean rather than relying on tone. You are now the weather. The group's norms about hours, about weekends, about whether it is acceptable to be stuck, about how a mistake is received — all of these are set by what the PI does rather than what they say. A PI who emails at 2 a.m. while assuring everyone that work–life balance matters has communicated only one of those two things. If you must work at unusual hours, delayed-send exists and costs nothing. Your feedback loop disappears. As a trainee, you knew where you stood: supervisors told you, referees told you, results told you. As a PI, almost nobody is positioned to tell you that you are managing badly. Your trainees cannot, because of the power asymmetry. Your peers do not see it. Your department head sees only outputs. You can go a decade without honest feedback on the half of your job that most affects other people's lives. Building an artificial feedback loop — anonymous group surveys, exit conversations conducted by someone else, a mentoring committee that meets trainees without you — is not a nicety. It is the only way to know. Time, honestly accounted A common and useful exercise is to track where the hours actually go for two weeks, in fifteen-minute blocks, without trying to improve anything. The result is almost always a surprise. Most new PIs believe they are spending a third of their time on their people and discover it is closer to a tenth, most of it reactive, most of it with the two or three group members who seek them out most. That last point deserves emphasis. Attention, left to itself, flows to whoever demands it. The people who demand it are typically the confident, the extroverted, and those culturally accustomed to approaching senior people directly. The people who do not demand it include the shy, the overwhelmed, those from educational backgrounds where approaching a professor is not done, and — critically — those who are struggling and ashamed. Unmanaged attention is therefore systematically maldistributed toward the people who need it least. Scheduling is the correction. A standing thirty-minute meeting with every group member, in the diary, that happens whether or not anything is wrong, is the highest-return management practice available to a laboratory leader, and the one most frequently abandoned under pressure. It should happen even — especially — when there is nothing to report. A meeting that only occurs when there is a problem teaches people that requesting a meeting is an admission of failure. Delegation as development New PIs delegate badly in two opposite directions. Some delegate nothing, and become the bottleneck through which every decision must pass, which exhausts them and infantilises the group. Others delegate by abandonment: hand over a problem with no specification, no check-in, and no stated standard, then express disappointment at what returns. Useful delegation states four things explicitly: what the outcome should be, what the constraints are, what decisions the person may take alone, and when you will next look at it. The third of those is the one usually omitted and the one that causes most distress. A trainee who does not know whether they are allowed to change a protocol will either freeze or proceed anxiously, and either way will spend cognitive effort on a question that thirty seconds of clarity would have removed. It also helps to be explicit about the level of finish expected. "I want a rough sense by Friday whether this is worth pursuing" and "I want an analysis I could put in a grant" are different requests, and a person who guesses wrong wastes days. Saying which one you mean is nearly free. Delegation is also the main instrument of development. A trainee who has never written a section of a grant, never reviewed a manuscript under supervision, never presented to an external audience, never trained a junior person, has been protected into unemployability. Each of those is a delegable task that costs the PI a little quality and a little time and returns a more capable person. The calculation almost always favours delegation if the horizon is longer than the current paper. Hiring as the first act of culture-setting The composition of the group determines more of its culture than any policy. Two principles do most of the work. First, hire for the group, not only for the project. Technical skill is real but it is also the most trainable attribute on the list. Curiosity, candour about not knowing things, a disposition to help others, and an ability to take a critique of the work without hearing a critique of the self — these are far harder to teach and far more determinative of whether the lab is a good place. A brilliant, corrosive recruit will cost more in other people's productivity than they contribute in their own. This is not a soft judgement; it is arithmetic. Second, make the process legible. Ad hoc recruitment — a friend's student, someone who emailed at the right moment — is how labs become homogeneous. A written role description, consistent questions, more than one person in the room, and a structured way of comparing candidates all reduce the influence of similarity bias. The evidence that unstructured evaluation in science is susceptible to bias is robust: Moss-Racusin and colleagues, in a well-known randomised study published in 2012, found that faculty rated an identical application less favourably and offered a lower starting salary when a female name was attached. The study is about laboratory manager hiring specifically, which makes it uncomfortably close to home. Structure is the practical defence, because it constrains the space in which unexamined preference operates. It is also worth telling candidates the truth about the group during recruitment: how you supervise, what hours look like in practice, what happened to the last four people who left, what the funding situation is. Candidates who would be unhappy self-select out, which is a service to everyone. A PI who oversells a lab acquires a disappointed employee within six months. Learning the job on purpose Because the profession supplies no training, the only reliable route is to seek it deliberately, and there is more available than most PIs realise. Many research funders and institutions run laboratory leadership programmes. The Burroughs Wellcome Fund and the Howard Hughes Medical Institute jointly produced Making the Right Moves, a practical guide to scientific management for new faculty, which is free and remains the most concrete starting point in print. Cold Spring Harbor Laboratory Press publishes Kathy Barker's At the Helm and Carl and Suzanne Cohen's Lab Dynamics, both written specifically for this audience. Formal mentor-training curricula exist — the Entering Mentoring family of materials developed by Christine Pfund and colleagues, now supported through the Center for the Improvement of Mentored Experiences in Research, has been evaluated in randomised trials and is widely adopted in the United States. Reading a book will not make anyone a good manager, any more than reading a protocol makes anyone a good experimentalist. What it does is supply vocabulary and a sense that these are known problems with known approaches, which is enough to stop the most common error: treating each people problem as unprecedented and improvising a response under stress. The deeper shift is one of self-conception. Most scientists who become PIs continue to think of themselves as scientists who unfortunately have to do some management. The ones who run good groups think of themselves as people who build the conditions for science to happen, of which doing science is one part. That is not a demotion. Over a career, a PI's total scientific contribution runs mostly through other people — through what thirty trainees go on to do, over forty years, in laboratories the PI will never see. Managed well, that is a far larger output than anything one pair of hands produces. Managed badly, it is a long trail of people who are worse at science, or no longer doing it, than they would have been. That is the actual job. It is worth taking seriously enough to learn. The habits that separate the two kinds of PI Over enough years one notices that the difference between groups that people thrive in and groups they survive is rarely a matter of grand philosophy. It is a small number of repeated behaviours. They separate the work from the worker, out loud. "This figure isn't convincing yet, and here's why" is a comment about a figure. "You haven't thought about this properly" is a comment about a person. The content may be identical; the effect is not. Good PIs are specific about which they are doing, and they do the second one rarely and privately. They say what they are unsure about. A supervisor who projects certainty at all times teaches a group that uncertainty is shameful, which is catastrophic in a profession made of uncertainty. Saying "I genuinely don't know whether this is the right approach, here's how I'd decide" models the actual epistemic behaviour of science and lowers the cost of a trainee admitting the same. They close loops. A draft returned in four days rather than four weeks is worth more than better comments delivered late, because the trainee's project is blocked and their anxiety is compounding. If a PI cannot read something promptly, saying when they will is nearly as good; silence is what does the damage. They notice the second-order effects of their own convenience. Moving a one-to-one is trivial for the PI and significant for the trainee who reorganised a week around it. Cancelling twice running communicates a ranking. A PI who protects those slots the way they protect a grant deadline sends a message no policy document can. They ask questions whose answers they cannot predict. "How's it going?" reliably produces "fine." "What's the thing you've been stuck on longest?" or "What's taking more time than it should?" or "What would you do differently if you were running this project?" produce information. The quality of a supervisory relationship is largely a function of the quality of the questions asked in it. None of these is difficult. All of them are easy to stop doing under pressure, which is why the useful unit of management improvement is the habit rather than the intention. The first year: what to get right For a PI in their first eighteen months, the temptation is to defer all of this until the lab is established, the first grant is in, and there is space to think about culture. The sequencing is exactly wrong. A group's norms set in the first year and are extremely hard to change afterwards, because the first two or three recruits become the transmitters of the culture to everyone who follows. A lab that starts with an implicit weekend norm will still have it in year eight, transmitted by people who joined in year four from people who joined in year one. Three things are worth doing before the first recruit arrives. Write down how you intend to supervise — meeting cadence, feedback expectations, authorship approach, hours — even if the document is two pages and imperfect; Chapter 3 goes into what belongs in it. Decide what your own visible working pattern will be, on the understanding that it will become the group's. And identify one person, ideally outside your department, whom you will talk to about management problems, because you will have them and there is otherwise nobody to ask. None of this requires the lab to be figured out. It requires only the recognition that a culture is going to form whether or not you attend to it, and that attending to it early is much cheaper than repairing it late. Chapter 3: Making the Implicit Explicit Ask ten members of a research group what is expected of them and you will get ten different answers, most of them wrong in the same direction. People overestimate the hours expected, underestimate the tolerance for failure, and are uncertain on nearly everything that matters: how often they should meet the PI, whether they may take a holiday, what a first draft is supposed to look like, what happens if an experiment does not work for six months, who will be an author on what. This uncertainty is not neutral. It consumes cognitive effort continuously, it produces anxious over-work in conscientious people and disengagement in others, and it falls unevenly. Trainees whose parents were academics, who did their undergraduate degree at a research-intensive institution, or who share a cultural background with their supervisor arrive already knowing much of the unwritten code. Those who do not arrive at a disadvantage that has nothing to do with their scientific ability and everything to do with information. Educationalists call this the hidden curriculum: the body of norms, expectations, and tacit rules that a setting transmits without teaching. In research training it is enormous. How to email a senior person you have never met. What it means when a supervisor says "that's interesting." Whether to say you are stuck, and when. How conference networking actually works. Whether the deadline is the deadline. Every one of these is learnable, and a group that teaches them explicitly rather than leaving them to be inferred converts a source of inequity into a source of competence. The whole of this chapter is one argument: write it down. Almost every problem in this booklet is smaller when the expectations that govern it were stated in advance, by both parties, in a document either can point at. The lab handbook The single highest-return document a research group can have is a handbook: a plain description of how the group works, given to everyone on arrival and revised annually. It is not a legal document, it is not a policy manual, and it does not need to be long. Ten to fifteen pages, written in the PI's own voice, is ample. What earns its place in one: · The group's scientific mission, in a paragraph — what the lab is trying to find out, and why it matters. People who understand the arc are better at judging what to do when a decision falls to them. · How supervision works: the cadence of one-to-ones, who else people can go to, how to get help urgently, what happens in lab meeting and how presenting rotates. · Working hours and availability: what the actual expectation is, in plain words. If the honest answer is "a full working week, occasionally more around a deadline, and I do not expect weekends," say that. If experiments require overnight or weekend attendance, say how that is compensated in time. · Holidays and time off: the entitlement, how to book it, and — critically — an explicit statement that taking the full entitlement is expected rather than merely permitted. · Communication norms: expected response times, what channel is for what, whether out-of-hours messages require a reply. Stating "I sometimes send email at night because that is when I write; I do not expect a response before the next working day" resolves a common and entirely avoidable source of strain. · Authorship and credit: the criteria the group uses, when authorship is discussed, and who decides. Chapter 7 treats this in detail; the handbook is where the principle lives. · Data management: where data live, in what format, backed up how, and what happens to it when a person leaves. This is an integrity requirement and, incidentally, a wellbeing one, because a person who cannot find their own data from eighteen months ago suffers for it. · Safety and reporting: how to report an incident, an error, a near-miss, or a concern about someone's conduct, including a route that does not go through the PI. · Support resources: where the counselling service is, what the student union or postdoc association offers, who the ombudsperson is, what the harassment reporting routes are. Listing these in a neutral document, alongside the fire procedure, normalises them. · What good looks like at each stage: what a first-year should be able to do by the end of the year, a third-year, a postdoc at eighteen months. Vague developmental expectations are a major source of private panic. What does not belong: aspirational language about excellence, a mission statement written in the vocabulary of a corporate away-day, or anything the PI does not actually intend to honour. A handbook that says one thing while the lab does another is worse than no handbook, because it teaches the group that written commitments from the leader are decorative. The exercise of writing one is itself valuable. Most PIs discover, drafting it, that they do not know their own answer to two or three of these questions. Better to find that out at a keyboard than in a dispute. Onboarding as a designed process The first month sets more than it should. A person who spends their first two weeks unsure where to sit, unable to access the instrument booking system, and afraid to ask has learned something about the group that six months of good intentions will not unteach. A workable onboarding structure has four elements. A practical checklist — access, accounts, safety training, keys, payroll — assigned to a named person who is not the PI, because it will otherwise be forgotten. A buddy, a peer one or two years ahead, whose explicit job for the first two months is to answer the questions people are embarrassed to ask the supervisor. A structured first meeting with the PI in the first week, which covers the handbook, the project, and expectations in both directions. And a first small project that can be completed in six to eight weeks and will probably work — a technical replication, a small analysis, a contribution to someone else's paper — so that the person's first experience is of competence rather than of a two-year unsolved problem. That last point is underrated. New researchers frequently arrive from an educational context of constant, rapid feedback and are placed immediately into a project whose first positive signal may be a year away. Manufacturing an early success is not coddling; it establishes that the person can do the work, which is the ground they will stand on later when nothing is working. The other half of onboarding is the conversation about what the person wants. Asking, in week one, why they came, what they want to be able to do by the end, what they are worried about, and what a good outcome looks like for them, costs half an hour and changes the supervisory relationship. It also surfaces mismatches early — the student who came for the technique rather than the question, the postdoc who intends to leave academia and has not said so — at a point when both parties can still act on them. Individual development plans An individual development plan is a short, revisited document that records what a person is trying to become, not just what they are trying to produce. The distinction matters. Project milestones track the science; a development plan tracks the scientist. The most widely used tool in the life sciences is myIDP, produced by Science Careers at AAAS, which combines a self-assessment of skills, interests, and values with a structured exploration of career paths and a goal-setting framework. Several funders now require something of this kind, and a number of institutions have their own. The specific instrument matters much less than the fact of having one and revisiting it. A functional plan covers four things: current skills and gaps, assessed honestly and by both parties; goals for the next twelve months, in the form of concrete, checkable commitments; what the trainee will do to get there; and what the supervisor will do. The fourth is the one usually missing, and its absence is what turns the exercise into a performance-management ritual rather than a two-way agreement. A plan that says "attend a scientific writing course, draft the methods section by March, present at the national meeting in June" should sit alongside "I will fund the meeting, return draft comments within one week, and introduce you to two people working on adjacent problems." Reviewing it twice a year is enough. The review should be a separate meeting from any project discussion, because the two crowd each other out: the urgent state of the experiment always displaces the slower question of what the person is becoming. Mentoring agreements Distinct from a development plan is a mentoring compact — a short written agreement between supervisor and trainee about how they will work together. Compacts of this kind have been advocated by professional bodies for some years and are used routinely in some fields, and the practice is worth adopting generally. A compact is one or two pages. It states what each party commits to: meeting frequency, feedback turnaround, how disagreements will be handled, how authorship will be decided, what happens if the project has to change direction, what notice each side gives of a significant change. Both sign it. It is revisited annually. Two objections usually arise. The first is that it is bureaucratic. It is one page, reviewed once a year, and it forestalls disputes that otherwise consume weeks. The second is that it formalises a relationship that ought to be based on trust. This confuses trust with vagueness. Written agreements are what allow trust to survive disagreement — a person who can point at a sentence does not have to rely on the goodwill of someone far more powerful than them at the worst moment in the relationship. The genuinely important function of a compact is that it gives the less powerful party standing to raise a problem. A trainee who says "you haven't returned my draft" is making a complaint about their supervisor. A trainee who says "we agreed two weeks, could we look at that" is referring to a joint document. The second is a conversation almost anyone can have; the first is one that many trainees, especially those with a precarious visa or a difficult history, will never have. The elements below summarise what these documents cover and how they differ, as Table 1 sets out. Table 1. Three written instruments and what each is for. Instrument Scope Who writes it Revisited Primary function Lab handbook The whole group PI, with group input Annually Removes guesswork about norms Mentoring compact One relationship Supervisor and trainee jointly Annually Gives both parties standing to raise problems Development plan One person's growth Trainee, reviewed with supervisor Twice yearly Separates becoming from producing Stage-appropriate expectations Expectations should differ by stage, and saying how they differ prevents a great deal of misery. A first-year doctoral student should be expected to learn techniques, read widely, make mistakes, and produce little. Treating a first-year's low output as underperformance is a category error that does real damage. What they should be assessed on is engagement, care, and rate of learning. A mid-stage student should be expected to run their own experiments, diagnose their own failures, and begin to shape the direction of the project. This is the stage at which control should be visibly transferred, and the stage at which many supervisors fail to transfer it, producing a capable third-year who is still waiting to be told what to do next. A final-year student should be expected to write, to defend positions independently, and to plan what comes after. The supervisor's job shifts to editing, connecting, and protecting the timeline. A postdoc is a different category and is frequently mismanaged as a senior student. A postdoc is an employee with a career to build, and the relationship should be one of managed independence: they should be developing their own line, building their own network, writing their own funding applications, supervising juniors, and accumulating a record that is legibly theirs. A postdoc whose supervisor's name is the only one on everything, who has never written a grant, and who has no independent project is being used rather than developed — and will discover this at exactly the moment they go on the job market. A technician or staff scientist has a career track that is chronically neglected, in part because they are the group's continuity and it suits everyone to keep them where they are. They need development plans too: training, authorship where merited, opportunities to present, and a route to advancement that does not require leaving. What to do when expectations were never set Most readers of this chapter are not starting a lab. They already have one, running on five years of accumulated implicit understanding, and the prospect of introducing a handbook to people who have been there for three years feels awkward — as though it implies a criticism of how things have been, or announces a crackdown. The awkwardness is real and it is manageable. The move that works is to make the group write it. Put the question to the lab meeting: what do people wish they had known when they arrived? What do new people always ask? What are we actually inconsistent about? Take the answers, draft from them, circulate the draft, and revise. This does three things at once. It produces a better document, because the people who joined most recently remember the confusions that the PI has long forgotten. It makes the exercise visibly collaborative rather than disciplinary. And it surfaces, in a low-stakes format, the places where the group's understanding of its own rules diverges — which is precisely the information a PI most needs and least often gets. Expect at least one uncomfortable discovery. In a group I know, the exercise revealed that every doctoral student believed the supervisor expected a first draft of a paper to be essentially publishable, and had therefore been sitting on drafts for months rather than submitting something rough. The supervisor had believed for years that his students were slow writers. One sentence in a handbook — "send me the ugly version; that is what I want" — recovered an enormous amount of stalled time. There is a related question about enforcement. A handbook that states a norm nobody follows degrades into decoration, and the norm most often ignored is the one about holidays and hours. If the document says people should take their full leave and nobody does, the PI must do more than restate it: ask each person in their one-to-one when they are taking it, put your own leave in the shared calendar visibly, and decline to schedule anything that assumes someone's absence is a problem. Norms about rest are set entirely by what the most senior person visibly does, and a written commitment contradicted by observed behaviour teaches only that the writing is not to be trusted. Reviewing rather than assuming None of this works if the documents are written once and filed. The mechanism that makes it work is a short annual review in which supervisor and trainee read the compact and the plan together and ask: is this still true? What did we each commit to, and did we do it? What has changed? That conversation takes forty minutes and does something no amount of informal goodwill achieves. It creates a scheduled, low-stakes occasion at which a problem can be raised — which means problems get raised at month eight rather than month thirty, when they are still small enough to fix. The commonest reason serious supervisory breakdowns reach the point of formal complaint is not that nobody noticed. It is that there was never an ordinary, expected moment at which the quieter party could say that something was wrong. Building that moment into the calendar is most of the work. Hashtags: #TheResilientLaboratory #LaboratoryLeadership #ResearchCulture #STEMMentorship #BurnoutPrevention #ResearcherWellbeing #PsychologicalSafety #LabManagement #PrincipalInvestigatorLeadership #MentoringCulture #WorkloadDesign #DemandControlModel #MaslachBurnoutFramework #CareerUncertainty #LabHandbook #MentoringCompact #IndividualDevelopmentPlans #InclusiveMentorship #AuthorshipFairness #ResearchIntegrity #SupervisorTraineeRelationship #LaboratoryResilience #CareerDevelopment #HealthyResearchEnvironments #FutureOfResearchCulture

Download the Book (PDF): Introduction In August 2015, a consortium of 270 researchers published the results of an unusual experiment. They had not studied memory, or prejudice, or decision-making. They had studied psychology itself. Working from a sample of one hundred studies published in three leading journals in 2008, they had attempted to repeat each one with high statistical power, using the original materials wherever they could obtain them and, in most cases, with the original authors' review of the protocol before data collection began. Ninety-seven of the hundred original studies had reported a statistically significant result. Thirty-six of the replications did. The average effect size in the replications was about half the size reported in the originals. The Open Science Collaboration's report did not claim that sixty-four findings were false. Replication is probabilistic; some of those studies were surely true effects that the replication attempt missed, and some of the successful replications were surely flukes. What the report established was narrower and more disturbing: from the published literature alone, a reader had no reliable way to tell which was which. The published record was not a map of what was known. It was a map of what had been reported, filtered through a set of practices that systematically favoured surprising, statistically significant, publishable claims over accurate ones. The decade since has confirmed that this was not a peculiarity of social psychology. Attempts to repeat experimental economics studies, social science papers from Nature and Science, preclinical cancer biology experiments, and analyses in political science and epidemiology have produced broadly similar patterns: replication rates well below what the literature's confidence implied, effect sizes that shrink on repetition, and — in the computational fields — a startling inability even to re-run the original analysis on the original data and obtain the original numbers. When Nature surveyed more than 1,500 researchers in 2016, more than seventy per cent said they had tried and failed to reproduce another scientist's experiment, and more than half said they had failed to reproduce one of their own. This booklet takes that diagnosis as settled and moves past it. The crisis literature has been written, argued over, and absorbed. What has been less well served is the practical question that follows: what, concretely, does a researcher, a lab head, an editor, or a research office do on Monday morning? The argument The controlling claim here is simple, and it runs against the moral framing that dominated the first decade of this debate. Reproducibility is not a virtue. It is an infrastructure. The early response to the replication crisis was largely hortatory. Researchers were urged to be more rigorous, more honest, more self-critical; bad actors were named; the field acquired a vocabulary of sin — p-hacking, HARKing, the file drawer — and a corresponding tone of confession and reproach. Some of that was necessary. Very little of it was sufficient, because the problem was never mainly one of individual bad faith. Almost all of the practices that generated the unreliable literature were, at the time, entirely normal. They were what competent, well-trained, well-intentioned researchers did, because the surrounding system — journals, funders, hiring committees, statistical training, the physical impossibility of publishing a null result — made them the rational thing to do. Systems that produce bad outcomes from well-intentioned people are not fixed by asking the people to try harder. They are fixed by changing what the system makes easy, what it makes hard, and what it makes verifiable. That is what open science practices actually are, when you strip away the movement rhetoric. A pre-registration is not a purity ritual; it is a timestamped specification that converts an unfalsifiable claim about the researcher's intentions into a checkable public record. A Registered Report is not a gesture of humility; it is a redesign of peer review that removes the reviewer's ability to condition acceptance on the result, thereby removing the author's incentive to produce a particular one. A version-controlled analysis pipeline in a container is not craftsmanship for its own sake; it is the difference between a claim that can be independently checked in an afternoon and one that cannot be checked at all. Each of these is a piece of engineering with a specific failure mode it addresses. The practical consequence of taking the infrastructural view seriously is that the work becomes tractable. You do not have to reform science. You have to install, in your own project, a small number of specific mechanisms, each of which closes a specific leak. That is a to-do list, and this booklet is largely about how to work through it. What this book covers, and what it does not The chapters that follow move from diagnosis to machinery to practice to institutions. The first two chapters establish what actually broke and why, with enough specificity that the fixes make sense as fixes rather than as rituals. The third settles vocabulary, which matters more than it sounds: "reproducibility" and "replicability" are used in opposite senses in different fields, and a policy written in one field's vocabulary can be meaningless in another's. The middle chapters are the operational core. Pre-registration, Registered Reports, computational transparency, and data archiving each get a chapter of their own, written as implementation guidance rather than advocacy: what to specify, in what order, with what tools, and what the common failure modes look like when the practice is adopted in form but not in substance. That last point recurs. Every one of these practices has a hollow version — the pre-registration so vague it constrains nothing, the data availability statement that promises data the authors will never send, the code repository that no one can run — and the hollow version is now common enough to be a problem in its own right. The final chapters deal with verification and with incentives: how errors are actually found in published work, what replication looks like as a routine activity rather than an act of aggression, and how labs, journals, funders, and hiring committees can change the defaults so that reliable work is also the easiest work to do. Three things are deliberately out of scope. This is not a statistics textbook; questions of estimation, power, and inference appear only where they bear directly on transparency. It is not a treatment of research misconduct; fabrication and paper mills are real and worsening, but they are a different problem with different remedies, and conflating them with ordinary irreproducibility has muddied both discussions. And it is not a survey of the open access publishing debate, which overlaps with open science institutionally but is a separate argument about business models. The intended reader is someone with skin in the game: a doctoral student designing a first study, a principal investigator deciding what her lab's standard practice will be, an editor considering a format change, a research integrity officer writing policy. The guidance is written to be usable in social science and biomedical settings alike, and where those settings genuinely diverge — particularly around human-subjects data, where "share everything" is not merely difficult but sometimes unlawful — the differences are treated explicitly rather than waved away. A final word on tone. It is easy, writing about this subject, to slide into either despair or evangelism. Neither is warranted. The literature is less reliable than it was assumed to be in 2010, and it is more reliable than it was in 2010, because a great deal of the machinery described in these pages did not exist then and now does. Trial registration transformed what cardiovascular trials report. Registered Reports have moved the rate of positive findings in the journals that use them from near-universal to something like a coin flip. Data and code archiving has gone from eccentric to mandatory across much of the funding landscape. None of it is finished, and some of it is being adopted in the hollow form rather than the real one. But the direction is clear, the tools exist, and the remaining work is mostly a matter of doing it. Chapter 1: The Anatomy of a Crisis The word "crisis" is doing a lot of work in the phrase "replication crisis," and it is worth being precise about what the evidence does and does not show. The claim is not that most published science is wrong. It is that the published literature contains a substantially higher proportion of findings that will not hold up than its own statistical apparatus advertises, that this proportion varies enormously by field and by study design, and that readers currently have very poor tools for telling the durable findings from the fragile ones. That claim rests on several independent lines of evidence, gathered by different communities using different methods over roughly fifteen years. They converge, which is why the conclusion has proved durable even as individual studies within the crisis literature have themselves been contested. The theoretical warning The modern phase of the argument opens in 2005, with John Ioannidis's essay "Why Most Published Research Findings Are False" in PLoS Medicine. The title was provocative; the content was arithmetic. Ioannidis's point was that the probability a published positive finding is true depends not only on the study's statistical properties — its significance threshold and its power — but on the prior probability that the tested hypothesis is true in the first place, and on the amount of flexibility and bias in how the study was conducted and reported. Run the numbers. In a field where researchers test many speculative hypotheses, with small samples, using flexible analytic methods, and where only positive results get published, the positive predictive value of a published significant finding can fall below fifty per cent even when every individual researcher is behaving honestly and every p-value is computed correctly. This is the crucial move, and the one most often missed. The arithmetic does not require anybody to cheat. It requires only that hypotheses be speculative, samples small, analyses flexible, and publication selective. Those four conditions describe a great deal of twentieth-century biomedical and social science. Ioannidis was not the first to make the point — Jacob Cohen had been complaining about statistical power since 1962, and Robert Rosenthal named the "file drawer problem" in 1979 — but his paper crystallised it, and it became the most-downloaded article in the journal's history. For several years it remained a theoretical warning. What changed the conversation was empirical demonstration. The industry replications The first hard empirical shock came from the pharmaceutical industry, and it came from a place that had no ideological interest in embarrassing academia: drug discovery. In 2011, researchers at Bayer HealthCare published an analysis in Nature Reviews Drug Discovery reporting on their internal attempts to validate published targets in oncology, women's health, and cardiovascular medicine. In only about a quarter of cases were the in-house findings consistent enough with the published data to justify continuing the project. The following year, C. Glenn Begley and Lee Ellis, writing in Nature, described Amgen's attempt to confirm fifty-three "landmark" preclinical cancer studies. Six were confirmed. These reports have well-known limitations. Neither disclosed which studies were attempted, so neither can be independently checked — an irony that the open science movement has not been shy about pointing out. But their significance was less evidential than motivational. They came from organisations that lose hundreds of millions of dollars when a preclinical finding fails to translate, and they signalled that the reliability problem had a price tag. That got attention from funders in a way that methodological argument had not. The systematic replication projects The next decade produced something better than industry anecdote: large, pre-specified, transparently reported replication projects, run by consortia, with protocols reviewed in advance and results published regardless of outcome. These are the backbone of the empirical case, and their findings are summarised in Table 1. Table 1. Major systematic replication projects and their headline results. Project Field Studies attempted Replicated Notable detail Open Science Collaboration (2015) Psychology 100 36 of 97 significant originals Mean replication effect size about half the original Klein et al., Many Labs 1 (2014) Psychology 13 effects, 36 samples 10 of 13 Effect sizes stable across sites; variation was not the explanation Camerer et al. (2016) Experimental economics 18 11 Replication effects about 66% of original magnitude Camerer et al. (2018) Social science in Nature/Science 21 13 Prediction markets forecast which would fail Errington et al. (2021) Preclinical cancer biology 50 experiments, 23 papers Mixed by criterion Median replication effect size 85% smaller than original Two features of this table deserve more attention than the headline percentages. The first is that these are not adversarial exercises. The Open Science Collaboration invited original authors to review replication protocols before data collection; many did, and many endorsed the designs. The Reproducibility Project: Cancer Biology went further, submitting each replication design as a Registered Report reviewed before any data were collected. The failures are not the product of incompetent replicators using wrong methods, which was the first and most natural defence offered when the results appeared. The second is the cancer biology finding about protocols, which is in some ways more damning than the replication rate itself. The project began by attempting to design replications of 193 experiments drawn from 53 high-profile papers. Not one of those experiments was described in enough detail in the published paper for the replication team to design the experiment without going back to the original authors. Not one. And when they did go back, roughly a third of authors were helpful, a third gave minimal or unhelpful responses, and a third did not reply at all. The project had to abandon the majority of its planned replications, finishing fifty experiments from twenty-three papers rather than the 193 it set out to do. That is a finding about the published record as a communication medium. A paper that cannot be used to repeat the experiment it describes is, whatever else it is, not a scientific report in the sense the word has carried since the seventeenth century. It is an announcement of a result. The computational failures Running alongside the replication projects is a second, quieter literature, and its findings are in some ways stranger. This one does not ask whether a new sample would produce the same result. It asks whether the same data, run through the same analysis, produce the same numbers. The answer, distressingly often, is no — or rather, unknown, because the data and code are not available to check. Economists have been at this longest. Beginning in the 1980s, the Journal of Money, Credit and Banking ran a project attempting to reproduce published results from author-supplied data and programs; the success rate was dismal. Three decades later, Andrew Chang and Phillip Li attempted to reproduce results from sixty-seven papers in thirteen economics journals, with author assistance where they could get it. They succeeded for fewer than half. Their working paper's subtitle — "Sixty Published Papers from Thirteen Journals Say 'Usually Not'" — was blunter than the profession was used to. In psychology, Tom Hardwicke and colleagues examined what happened after Cognition adopted a mandatory data-sharing policy. The good news was that data were now usually available. The less good news was that when the team attempted to reproduce the main findings from the shared data, they could do so without contacting the authors in only a minority of cases; author assistance raised the success rate substantially but did not get it close to complete. The obstacles were rarely fraud. They were missing code, missing documentation, undocumented exclusions, software version drift, and files whose relationship to the reported analysis was simply unclear. Victoria Stodden and colleagues, examining a journal that required code and data sharing, found that a substantial fraction of requested artifacts could not be obtained at all, and that among those obtained, independent reproduction succeeded in only a minority of cases. Every such study has methodological quibbles available to it — what counts as reproduction, how hard the team tried, whether "close enough" counts. But no study of this kind has ever reported a high success rate, in any field, in any decade. Analytic variability A third line of evidence arrived more recently and is, to my mind, the most conceptually important, because it identifies a failure mode that none of the standard remedies for fraud or incompetence would touch. In 2018, Raphael Silberzahn, Eric Uhlmann, Brian Nosek and colleagues gave twenty-nine analyst teams the same dataset and the same question: are football referees more likely to give red cards to dark-skinned players than light-skinned ones? Every team was competent. Every team's approach was reviewed by the other teams. Twenty of the twenty-nine found a statistically significant positive effect; nine did not. The estimated effect sizes ranged from essentially nothing to a very large effect. The variation came entirely from defensible analytic choices: which covariates to include, how to handle repeated observations of the same player, which model family to use. The finding has since been reproduced in other domains. Seventy teams analysing the same functional MRI dataset produced no two identical pipelines and substantial disagreement about which brain regions showed significant activation. Seventy-three teams testing whether immigration reduces support for social policy, using the same survey data, produced results spanning both directions. What these studies show is that the analysis of a dataset is not a deterministic procedure that a competent researcher applies to data to extract the answer. It is a long sequence of judgement calls, each individually reasonable, whose joint effect on the final number is large and largely invisible to the reader of the resulting paper. A published analysis is one path through a branching structure of thousands. The paper reports the path; it does not report the structure. That is why analytic transparency — pre-specification before the fact, full code disclosure after it — does different work from ordinary rigour. No amount of care in executing a single analysis tells you how much the result depended on the choices that defined it. The positive result problem Step back from any individual replication project and look at the literature in aggregate, and one statistic dominates everything else: the published scientific record is almost entirely composed of successes. Daniele Fanelli, surveying thousands of papers across disciplines, found that the proportion reporting support for the tested hypothesis was high everywhere and rising over time, reaching above ninety per cent in several social science fields. Anne Scheel and colleagues, comparing a sample of standard psychology papers with a sample of Registered Reports — articles accepted for publication before results were known — found positive results in around ninety-six per cent of the standard literature and around forty-four per cent of the Registered Reports. Same field, same kinds of question, wildly different hit rates, with the only systematic difference being when the publication decision was made. There is no plausible world in which researchers' hypotheses are correct ninety-six per cent of the time. If they were, the hypotheses would not be worth testing. A literature with that hit rate is not reporting what happened; it is reporting a filtered subset of what happened, and the filter is applied at several points at once — by the researcher who abandons a null finding as unpublishable, by the reviewer who reads a null result as a failed study, by the editor who knows that null results are not cited. The cleanest natural experiment on this comes from clinical trials. Robert Kaplan and Veronica Irvin examined large randomised trials of drugs and dietary supplements funded by the US National Heart, Lung, and Blood Institute, comparing those published before and after the institute began requiring registration of the primary outcome on ClinicalTrials.gov. Of the trials conducted before the requirement, around fifty-seven per cent reported a significant benefit for the treatment. Of those conducted after, around eight per cent did. The interventions had not become less effective. What had changed was that the primary outcome could no longer be selected after the data were in. That single comparison is worth more than a shelf of methodological argument, because it isolates the mechanism. Nothing about the science changed. One piece of bureaucratic infrastructure changed, and the reported success rate fell by a factor of seven. What researchers say they do The replication and publication-bias evidence is indirect: it infers behaviour from outcomes. There is also direct evidence, from surveys that asked researchers what they actually do, under conditions designed to elicit honest answers. The best known is Leslie John, George Loewenstein and Drazen Prelec's 2012 survey of over two thousand academic psychologists, which used an incentive-compatible design to encourage truthful reporting. Large majorities admitted to at least one questionable research practice: failing to report all of a study's dependent measures, collecting more data after checking whether the result was significant, selectively reporting studies that worked, excluding data after looking at the impact of doing so, reporting an unexpected finding as though it had been predicted. Most respondents did not consider these practices indefensible. Many were taught them. That last point matters more than the prevalence figures, which have been contested on methodological grounds. In 2010, none of these practices was clearly prohibited by any journal, funder, or institution. Adding participants until a result stabilised was widely regarded as efficient. Dropping a measure that "didn't work" was regarded as good writing. Constructing a narrative in which the obtained result was the predicted one was regarded as good exposition — journals explicitly asked for clean, coherent papers, and a clean narrative is what you get when the hypothesis is written after the data. The surveys, in other words, are not a catalogue of misconduct. They are a description of normal practice under a set of incentives, and they tell you that the problem sits at the level of professional norms rather than individual character. Power, and the small-sample trap One more strand, because it explains why so many published effects are both significant and wrong. Katherine Button and colleagues, analysing meta-analyses in neuroscience, estimated the median statistical power of studies in the field at around twenty per cent. Other estimates across psychology and biomedicine land in a similar range. Power that low has two consequences, and the second is not intuitive. The obvious consequence is that most true effects will be missed. The less obvious one is that the true effects that are detected will be overestimated, often grossly. If a study is underpowered, the only way a real effect can clear the significance threshold is if sampling noise happens to push the estimate well above the true value. Published significant results from low-powered studies are therefore a biased sample of inflated estimates — a phenomenon Andrew Gelman and John Carlin call a "type M", or magnitude, error. Add the possibility of getting the sign wrong as well, and you have studies that are more likely to mislead than to inform even when everything is done correctly. This is the arithmetic behind the shrinking effect sizes in Table 1. When a well-powered replication estimates an effect at half the published magnitude, the usual explanation is not that the original was fabricated. It is that the original was small, and small studies that reach significance are inflated by construction. What the crisis is not Three clarifications, because the argument gets muddled without them. It is not a claim that science does not work. The replication projects themselves are science working: measurement applied to a system, results published whatever they showed, methods open to inspection. The fields that discovered their own problem were, in general, the fields methodologically self-aware enough to look. Silence from a discipline is not evidence of health. It is not primarily about fraud. Fabrication exists, it appears to be increasing in certain publishing ecosystems, and industrial-scale paper mills are a genuine and worsening threat to the integrity of the literature. But the replication failures documented above are overwhelmingly not fraud. They are the predictable output of small samples, flexible analysis, and selective publication operating on speculative hypotheses. The remedies for fraud — detection, investigation, sanction — do almost nothing about them. And it is not uniform. Reliability varies enormously with design. Large pre-registered randomised trials with pre-specified primary outcomes replicate well. Small, unregistered, exploratory studies of subtle behavioural effects replicate badly. Genome-wide association studies, after a disastrous candidate-gene era in which perhaps ninety-five per cent of published associations proved spurious, became one of the most reliable literatures in biomedicine — not by exhorting geneticists to be careful but by imposing genome-wide significance thresholds, mandatory independent replication, and enormous consortium samples. That transformation is the single most useful precedent in this whole debate, and it happened because the field changed its rules rather than its attitude. Which is the theme this booklet develops. The crisis is not a moral failure requiring moral correction. It is a design failure in the system that produces and certifies knowledge, and design failures are fixed by redesign. The next chapter examines the specific mechanisms through which the old design generated the results we have been looking at, because you cannot redesign what you have not diagnosed. Chapter 2: The Machinery That Produces Unreliable Findings A fault diagnosis is only useful if it identifies mechanisms. "The literature is unreliable" is a symptom. This chapter describes the specific processes that generate it, because each remedy in the rest of the book is aimed at one or more of them, and a remedy applied to the wrong mechanism is worse than useless — it consumes effort and produces the appearance of reform. There are four mechanisms, and they compound. Taken one at a time each looks survivable. Operating together on the same literature, they are sufficient to produce the pattern documented in the previous chapter without any individual acting in bad faith. Mechanism one: the garden of forking paths Start with the analysis, because this is where the largest and least visible damage occurs. The standard interpretation of a p-value assumes a specific counterfactual: that if the null hypothesis were true and the study were repeated many times, results this extreme would occur no more than five per cent of the time. That interpretation holds only if the analysis was fixed in advance. If the analysis was chosen after seeing the data — even in small ways, even reasonably — the guarantee dissolves, and the true false positive rate can be several times the nominal one. In 2011, Joseph Simmons, Leif Nelson and Uri Simonsohn demonstrated this in the most vivid way available: they used it to prove something false. Their paper "False-Positive Psychology" reported a real experiment, honestly conducted, showing that listening to the Beatles' "When I'm Sixty-Four" made participants a year and a half younger — not feel younger, but be chronologically younger, as measured by their reported date of birth. The finding was significant at conventional levels. It was also, necessarily, false. They obtained it by exploiting four ordinary degrees of freedom: choosing among several dependent measures, deciding when to stop collecting data, including or excluding a covariate, and reporting a subset of experimental conditions. Their simulations showed that these four flexibilities together raise the probability of at least one false positive in a study from five per cent to over sixty per cent. Their term for it, "researcher degrees of freedom," entered the field's vocabulary immediately. Two years later, Andrew Gelman and Eric Loken made the argument subtler and considerably more uncomfortable, in an essay titled "The Garden of Forking Paths." The Simmons account, they noted, suggests a researcher trying many analyses and picking the best — which sounds like something you would notice yourself doing. But the inflation does not require that. It requires only that the analysis you would have run had the data looked different is different from the one you ran. A researcher who runs exactly one analysis, in complete good faith, and would honestly deny fishing, still gets the inflated error rate, because the choice of that analysis was data-dependent. Consider a concrete case. You hypothesise that a framing manipulation affects risk-taking. The data arrive. The main effect is weak, but there is a clear difference among women. You report the interaction with gender, and you write it up as a finding about gender-differentiated framing effects, with a plausible theoretical rationale that you genuinely believe. Had the effect appeared among men instead, you would have written the corresponding paper. Had it appeared among older participants, you would have written that one. Had the main effect been strong, you would have reported the main effect and never looked at subgroups at all. You ran one test; the garden contained dozens of paths; your p-value is calibrated for a world in which only one path existed. This is why exhortations to honesty do not solve the problem. The researcher in this example is honest. The fix has to be structural: the path must be chosen before the data exist. That is precisely and only what pre-registration does. Hypothesising after the results are known A close relative of forking paths deserves its own name, and has one. In 1998, Norbert Kerr described the practice of presenting a post hoc hypothesis as though it had been specified in advance, and called it HARKing — Hypothesising After the Results are Known. HARKing is not lying, exactly, and this is what makes it so persistent. The researcher genuinely believes the theoretical account she writes; she arrived at it by thinking hard about a real pattern in real data. Inductive reasoning from data to theory is a legitimate and indispensable scientific activity. The damage is done by the presentation, which converts an exploratory finding into a confirmatory test it never was. A hypothesis generated from a dataset cannot be tested by that dataset; the fit is guaranteed. Reporting it as a prediction offers the reader a test that has already been passed by construction. Kerr's paper listed the reasons HARKing was rewarded: it makes papers cleaner, it satisfies reviewers who want a coherent story, it flatters the author's apparent insight, and it makes the introduction easier to write. Journals have spent decades teaching authors to produce exactly this shape of paper. Ask an editor for an "integrated narrative" or a manuscript that "tells a clear story," and you have asked, without meaning to, for the hypothesis to be written after the analysis. The remedy is not to ban exploration. It is to label it. A paper that says "we explored the data and found X; here is a pre-registered confirmatory test of X in a new sample" is doing excellent science. A paper that says "we predicted X and found X," when it did not, is not. Mechanism two: selective publication The second mechanism operates one level up, on which results enter the record at all. Rosenthal's file-drawer metaphor from 1979 still describes it best: for every published significant finding, some unknown number of null results sit in filing cabinets. If the unpublished studies were a random sample of all studies, this would merely waste effort. They are not random. They are selected precisely on the variable that determines what the meta-analytic estimate will be. A literature assembled this way is biased upward by construction, and the bias is invisible from inside the literature. The selection is not imposed by a censor. It is applied at several independent points, each of which feels locally sensible: · The researcher. A null result is harder to write up, harder to place, less cited, and worth less at review time. Writing it up costs weeks that could go into the next study. Most nulls die here, without anyone deciding they are wrong. · The reviewer. A null result invites the response that the study was underpowered, the manipulation weak, the measure unreliable. These objections are often correct, and they are unanswerable, because absence of evidence in a single study genuinely is ambiguous. Positive results do not face the symmetric challenge. · The editor. Journals compete for citations and attention. Null results generate neither. An editor who published them in proportion to their occurrence would preside over a less-read journal. · The field. Even published nulls are undercited, and a finding that fails to replicate often remains cited as established for years after — a phenomenon documented repeatedly in the medical literature, where retracted or refuted findings continue to accumulate citations long after the correction. The consequence is that the literature reports, roughly, the upper tail of the distribution of conducted studies. Meta-analysis, which was supposed to be the corrective, inherits the bias: it can only synthesise what was published. Statistical tools for detecting publication bias — funnel plot asymmetry, trim-and-fill, p-curve, selection models — help, but they are estimating the shape of something they cannot observe, and they perform poorly when heterogeneity is high or when bias operates through analytic flexibility rather than outright suppression. The only complete fix is to make the publication decision independent of the result. There are exactly two ways to do that: register the study in advance so that its existence is known whatever happens, and make the acceptance decision before the results exist. Chapters 4 and 5 are about those two mechanisms. Outcome switching, and what happens when someone checks Clinical medicine offers the best-documented instance of selective reporting operating inside a single paper rather than across the literature, because clinical medicine is the one field where the pre-specified plan is publicly available for comparison. Since 2005, the International Committee of Medical Journal Editors has required trials to be registered before enrolment as a condition of publication, with a declared primary outcome. That declaration makes a specific form of checking possible: take the published paper, take the registry entry, and see whether the outcomes reported are the outcomes promised. Ben Goldacre's COMPare project did exactly this for trials published over a period in five leading medical journals. In the great majority of papers examined, the reported outcomes diverged from the registered ones — pre-specified outcomes silently dropped, new outcomes added without being flagged as post hoc. When the team wrote correction letters to the journals documenting each discrepancy, the responses ranged from prompt correction to flat refusal, and some of the editorial replies revealed a striking lack of familiarity with the journals' own stated policies. Two lessons follow. The first is that registration only works if somebody compares the registration with the paper, and the comparison is currently done almost nowhere as a matter of routine. The second is that a policy which exists on paper and is not enforced generates a false sense of security: readers of those journals reasonably assumed that the ICMJE requirement meant reported outcomes were pre-specified, and for most of the trials examined it did not. Mechanism three: incentives Underneath both of the preceding mechanisms sits a reward structure, and it is worth stating plainly rather than gesturing at. Academic careers are decided by a small number of evaluations — hiring, tenure, promotion, grant renewal — conducted by committees who cannot possibly read the candidate's work in depth. Faced with fifty applications and limited time, committees use proxies: number of publications, prestige of venue, citation counts, grant income, h-index. These proxies are not stupid. They correlate with quality. But they are cheap to game relative to the thing they proxy, and Goodhart's law applies with full force. Now consider which behaviours the proxies reward, as Table 2 sets out. Table 2. What the conventional reward structure pays for, and what it costs. Behaviour Career payoff Effect on reliability Many small studies High — more publications Negative — low power, inflated estimates One large definitive study Low — one publication, slow Positive Novel surprising claim High — prestige venues, citations Negative — low prior probability Replication of others' work Very low — hard to publish, seen as unoriginal Positive Publishing a null result Negative — time cost, little credit Positive Sharing data and code Neutral to negative — effort, exposes errors Positive Flexible analysis until significant High — guarantees publishable result Strongly negative Every row in that table points the same way: the practices that make a literature reliable are the ones the reward structure penalises, and the practices that corrupt it are the ones it pays for. A young researcher who adopts the reliable column wholesale, in an environment that has not changed its evaluation criteria, will produce fewer papers in lesser venues than a competitor who does not, and will lose the job. This is the strongest argument against the purely moral framing of the reproducibility problem. You are asking individuals to absorb a personal cost for a collective benefit, in a competitive selection process, at the career stage where they have the least security. Some will do it anyway; the open science movement is substantially built by people who did. But a reform that depends on sustained altruism under selection pressure is not a reform, it is a hope. The durable fixes are the ones that change the payoffs — which is why Chapter 9 deals with hiring committees and funders rather than with individual conscience. Mechanism four: the opacity of the artifact The fourth mechanism is different in kind. The first three concern how results are produced; this one concerns how they are communicated, and it is the reason errors survive. A scientific paper is a compressed narrative account of work that was, in reality, an enormous and messy computational and physical process. A typical empirical paper in a social or biomedical field represents thousands of decisions: instrument settings, exclusion rules, data cleaning steps, variable transformations, model specifications, software versions, random seeds. The paper describes perhaps twenty of them, in prose, in a methods section constrained by word limits and by conventions that reward brevity. This was a reasonable design when papers were printed on paper and the alternative to compression was nothing. It is indefensible now. The full record — the raw data, the cleaning scripts, the analysis code, the computing environment, the protocol, the materials — exists on the researcher's machine, costs almost nothing to store, and would answer the great majority of questions a sceptical reader might have. It is simply not transmitted. The consequences show up everywhere in the evidence from the last chapter. The Reproducibility Project: Cancer Biology could not design a single replication from a published methods section. Computational reproduction attempts fail on missing code and undocumented exclusions rather than on disagreement about substance. Analytic variability studies find enormous spread precisely because the published account of an analysis does not pin the analysis down. There is also a quieter cost: errors go undetected. Michèle Nuijten and colleagues built a tool called statcheck that recomputes the p-values in psychology papers from the reported test statistics and degrees of freedom. Applied to tens of thousands of articles, it found that around half contained at least one internally inconsistent statistical result, and that roughly one in eight contained an inconsistency large enough to change whether a result was significant at the conventional threshold. These are not fraud; they are typos, copy-paste errors, and last-minute changes not propagated through the manuscript. They persisted because nobody could check. The moment a machine could check, they became visible in bulk. Opacity, in other words, is not a neutral property of the publication format. It is an active cause of error persistence, and the remedy — transmitting the artifact rather than a summary of it — is the subject of Chapters 6 and 7. How the mechanisms compound Taken separately, each mechanism looks like something a careful field could absorb. Their interaction is what produces the observed failure rate. Flexible analysis raises the probability that any given study yields a significant result. Selective publication ensures that the significant ones are the ones that enter the record. Low power ensures that those that do enter are inflated in magnitude. Career incentives ensure that researchers run many small studies rather than few large ones, maximising the number of draws from this biased process. And opacity ensures that none of it can be detected from the published paper, so the literature accumulates without correction. Run that system for thirty years and you get exactly what the replication projects found: a literature of significant, surprising, well-told findings, a substantial minority of which do not exist. Notice what this analysis implies about remedies. A practice that addresses only one mechanism will produce disappointing results, because the others route around it. Pre-registration without a commitment to publish regardless of outcome fixes forking paths but not the file drawer; researchers simply abandon the registered studies that fail, and the registry fills with dead entries. Data sharing without code sharing fixes nothing, because the data alone do not specify the analysis. Open access without transparency of method makes unreliable findings more widely readable. This is the single most common failure of reproducibility policy: adopting one mechanism, observing that the literature has not improved, and concluding that the whole enterprise is theatre. The corollary is more encouraging. Because the mechanisms compound, so do the fixes. A study that is pre-registered, published as a Registered Report, with data and code archived and a protocol available, is protected at every point in the chain — and the marginal effort of adding each practice once the others are in place is small, because they share infrastructure. That is the practical case for treating open science as a stack rather than a menu, and it is the organising principle of the chapters that follow. Before building the stack, though, we need to be precise about what we are trying to achieve, because the central terms of this field are used in incompatible ways by the people using them. That is the next chapter's business. Chapter 3: A Working Vocabulary There is a peculiar obstacle in the middle of this field, and anyone writing policy will hit it within a week: the two central words mean opposite things depending on who is speaking. In computational science, following Jon Claerbout's usage from the early 1990s, reproducible means that someone else can take your data and your code and obtain your numbers, and replicable means that an independent study with new data reaches the same conclusion. In much of the biomedical and psychological literature, the terms are reversed: reproduce means to repeat the experiment and get the same answer, and replicate is sometimes used for the narrower computational check. Some fields use them interchangeably. The Association for Computing Machinery has used both conventions at different times, and at one point revised its own definitions to switch them. This is not pedantry. A funder that mandates "reproducibility" without saying which one it means has issued an instruction that different applicants will implement differently — some by depositing code, some by running a second experiment, at a cost difference of three orders of magnitude. The 2019 US National Academies report Reproducibility and Replicability in Science settled the question for English-language policy purposes, and this booklet follows it: · Reproducibility is computational: obtaining consistent results using the same input data, computational steps, methods, and code as the original study. · Replicability is empirical: obtaining consistent results across studies aimed at answering the same question, each of which has obtained its own data. Whichever convention you adopt, state it. A single sentence defining terms at the top of a policy document prevents a great deal of confusion downstream. The ladder It helps to think of the relevant concepts as a ladder rather than a pair, because there are more rungs than two and the practical demands differ sharply between them. Table 3 lays them out. Table 3. The reliability ladder: five distinct questions a reader might ask about a finding. Rung Question answered What must be shared Typical cost Computational reproducibility Does the author's code on the author's data give the author's numbers? Data, code, environment specification Hours Analytic robustness Does the conclusion survive different defensible analytic choices? Data, code, documented alternatives Days Direct replication Does a new sample, same protocol, give the same result? Protocol, materials, measures Months, a new study Conceptual replication Does the underlying claim hold with different operationalisations? Theory, construct definitions Months to years Generalisability Does the effect hold in other populations, settings, and times? Everything above, plus new contexts Years, programmes of research Reading down the ladder, two things change. The cost rises steeply. And the meaning of failure changes with it. A failure of computational reproducibility is unambiguous and is always a defect. If the author's own code on the author's own data does not produce the author's own reported numbers, something is wrong: a typo, a stale output file, an undocumented manual step, an unrecorded exclusion. There is no interpretation under which this is acceptable, and no scientific judgement is required to adjudicate it. This is why computational reproducibility is the right first target for any reform effort — it is cheap, binary, and uncontroversial. A failure of direct replication is ambiguous. It might mean the original was a false positive. It might mean the replication was underpowered. It might mean an unrecognised moderator differs between the two settings. Adjudicating requires judgement, more data, and usually a long argument. This is why replication disputes turn acrimonious in a way that reproduction disputes do not. A failure of generalisability is often not a failure at all. An intervention that works in Finnish schools and not in Kenyan ones has told you something valuable about both. The mistake was in the original claim's implied scope, not in either study. Confusing these rungs produces bad arguments in both directions. Defenders of a contested finding often respond to a failed direct replication by invoking generalisability — "the effect is context-sensitive" — when the replication was conducted in the same context with the same materials. Critics sometimes treat a failure of conceptual replication as though it falsified the original, when a different operationalisation is a different test. One further dimension cuts across the whole ladder: independence. A result repeated by the same person, on the same machine, in the same week is weaker evidence than the same result obtained by a different team, in a different laboratory, with its own equipment and its own analyst. Internal replication — the "study 1, study 2, study 3" structure common in psychology papers — is genuinely useful for establishing that a finding is not a one-off fluke of a single sample, but it shares every systematic bias the lab carries: the same materials, the same subject pool, the same analytic habits, the same investigator expectations. The strongest form is adversarial: a replication designed jointly by the original authors and by researchers who doubt the finding, with the protocol, the analysis and the success criterion agreed in advance by both parties. Adversarial collaborations are rare because they are uncomfortable and slow, but they resolve disputes that no amount of unilateral replication does, because neither side can afterwards claim the design was stacked. Where a disagreement has become entrenched enough to consume years of journal space, this is usually the cheapest way out of it. Robustness, and the multiverse The second rung deserves more attention than it usually gets, because it is the one that has most changed how careful empirical papers are written. Analytic robustness asks: given this dataset and this question, how much does the answer depend on the choices the analyst made? The analytic variability studies described in Chapter 1 established that the answer is often "a great deal." What followed was a set of methods for measuring it rather than hoping it was small. Specification curve analysis, developed by Simonsohn, Simmons and Nelson, enumerates the reasonable analytic specifications — every defensible combination of covariates, exclusions, outcome definitions and model forms — runs all of them, and plots the resulting estimates in order. What the reader sees is not one number but the distribution of numbers the data support, with the specifications that produce each. Sara Steegen and colleagues' multiverse analysis does the analogous thing for data processing choices, which are usually made before the analysis proper and are usually invisible. The results are informative in both directions. Sometimes the curve is flat: whatever you do, the effect is there, and the finding is robust in a way no single specification could demonstrate. Sometimes it straddles zero, and the honest report is that the data do not settle the question. Occasionally the published specification sits at the extreme edge of the curve, which is diagnostic. These methods do not replace pre-specification; they complement it. Pre-registration prevents the analyst from choosing the specification after seeing the results. A specification curve shows the reader what would have happened under other choices. The first is a commitment device, the second a disclosure — and the strongest empirical papers now do both, reporting the pre-registered primary analysis and a robustness surface around it. What counts as "the same result"? A replication project cannot report a success rate without a definition of success, and there is no consensus definition. This is not a technicality; the choice of criterion can move a project's headline number by twenty percentage points, which is why arguments about replication rates so often turn out to be arguments about arithmetic. The main candidates each capture something real and each fail in a characteristic way. Statistical significance in the same direction. The simplest rule: did the replication obtain p below the threshold, with the effect pointing the same way? It is easy to apply and easy to communicate, which is why it dominates headlines. It is also the worst criterion available, because it inherits every problem of dichotomised inference. A replication that estimates an effect at 0.19 with a confidence interval just crossing zero is scored as a failure, while one that estimates 0.02 with a very large sample is scored as a success even though it has effectively refuted the original's magnitude. The original estimate falls inside the replication's confidence interval. Better, because it compares magnitudes rather than verdicts. It is biased by sample size in an awkward way: a small replication has wide intervals and will "succeed" against almost anything. The replication estimate falls inside the original's prediction interval. Better still, since it accounts for uncertainty in both studies, but it requires the original to have reported enough to compute one. Meta-analytic combination. Pool the two and ask what the combined estimate says. Sensible for accumulating evidence, but it lets a large original dominate a replication designed precisely to test it. Small telescopes. Simonsohn's proposal: ask whether the replication can reject an effect large enough that the original study would have had reasonable power to detect it. If the answer is yes, the original could not have provided meaningful evidence for the effect it claimed, whatever the truth of the matter. This reframes the question usefully — it asks about the evidential value of the original rather than about the existence of the effect. Subjective assessment by the replication team. Used, alongside other criteria, by the Open Science Collaboration. Less rigorous-sounding, but it captures things no formula does, such as a replication that technically succeeded on a measure everyone agrees was the wrong one. Serious replication projects now report several criteria side by side and let the reader see the spread, which is the right answer. For anyone designing a replication, the operational implication is concrete: state the success criterion in the pre-registration, before the data exist. Otherwise the choice of criterion becomes one more forking path, with the added hazard that a replicator who has an opinion about the original finding will, entirely unconsciously, prefer the criterion that confirms it. Transparency is not the same as reproducibility A further distinction matters in practice, because conflating these two is the most common way that good-faith reproducibility policy turns into paperwork. Transparency is the disclosure of what was done: the protocol, the analysis plan, the code, the data, the deviations, the conflicts of interest. It is a property of the reporting. Reproducibility is a property of the result: that the reported numbers can in fact be obtained again. Transparency is necessary for reproducibility but does not guarantee it. You can share everything and still have made an error; indeed the point of sharing is that the error becomes findable. Conversely, a result can be reproducible in principle — the author could produce the numbers again — while being entirely opaque to everyone else, which is the situation the literature was in for most of its history. The practical implication is that transparency is the operational target. It is entirely within the author's control, it can be verified at submission, and it is the precondition for everything else. You cannot require that a result be true. You can require that it be checkable. This also clarifies what open science practices do and do not promise. Pre-registration does not make a finding true. It makes the distinction between prediction and postdiction checkable. Data archiving does not make an analysis correct. It makes the analysis contestable. The gain is not that individual papers become right; it is that the error-correction machinery of the field, which has always been the actual source of scientific reliability, is given something to work with. Open science, defined narrowly "Open science" is used as an umbrella for a set of practices that are related but separable, and it pays to keep them distinct, since institutions frequently adopt one and claim the whole. Open access concerns who can read the published paper. It is about business models and paywalls, and it is genuinely important, but a paywalled paper with full data and code is more scientifically useful than a free one with neither. Open data concerns whether the underlying observations are available for inspection and reuse, under conditions that respect participant confidentiality and legal constraints. Open code concerns whether the computational steps from raw data to reported result are available and executable. Open materials concerns the instruments: questionnaires, stimuli, protocols, reagent identifiers, cell line provenance, software configurations. Open evaluation concerns the review process: whether reports are published, whether reviewers are named, whether review happens before or after the result is known. Pre-registration concerns the timing of commitments: what was planned before the data existed. These are largely orthogonal. A field can have near-universal open access and near-zero analytic transparency, which describes several. A lab can pre-register everything and share nothing, or share everything and pre-register nothing. Institutional policy tends to pick up whichever is cheapest to mandate and easiest to audit — often open access, because it is a publishing contract rather than a change in practice — and then report progress on reproducibility. The distinctions in this list are what let you tell whether a policy is doing the work it claims. The TOP framework If there is a single document worth knowing in this area, it is the Transparency and Openness Promotion guidelines, published by Brian Nosek and a large group of co-authors in Science in 2015 and since adopted in some form by well over a thousand journals and organisations. TOP is not a manifesto. It is a specification with eight standards — citation of data and materials, data transparency, analytic code transparency, research materials transparency, design and analysis transparency, study pre-registration, analysis plan pre-registration, and replication — each of which a journal can adopt at one of three levels, plus level zero for no policy. The levels are the useful part. Level 1 is disclosure: authors must state whether data are available and where. Level 2 is requirement: data must be deposited in a trusted repository, with stated exceptions. Level 3 is verification: the journal or a third party checks that the deposited materials actually reproduce the reported results. This graduation matters because it names the gap between policy and practice. Most journals that claim a data policy are at Level 1, and Level 1 policies are widely satisfied by statements that turn out to be false. Studies of data availability statements have repeatedly found that when someone actually requests the data promised in those statements, the large majority of requests go unfulfilled — authors do not reply, or reply that the data are unavailable after all, or impose conditions not mentioned in the statement. One examination of nearly two thousand articles with data availability statements found that only a small percentage of authors actually provided data when asked. A field that has moved from Level 0 to Level 1 has changed its paperwork. A field that has moved to Level 3 has changed its literature. The American Journal of Political Science, which since 2015 has required successful independent verification of the computational results before publication, is the standing proof that Level 3 is achievable in practice: manuscripts are accepted conditionally, the replication package goes to a verification service, and errors found there are fixed before the article appears. Several economics journals now run comparable processes, and the effect on what gets published is not subtle. What to take from this Three practical rules follow from the vocabulary. First, when you write or read a reproducibility requirement, identify which rung of the ladder it addresses. Most institutional policies address only the first, and that is a defensible priority — but it should be a choice rather than an accident of wording. Second, aim policy at transparency, which is verifiable at submission, rather than at reliability, which is not. The first is enforceable; the second is an outcome you hope follows. Third, treat the Level 1 / Level 2 / Level 3 distinction as the real measure of whether an institution has done anything. Disclosure requirements are almost free and produce almost nothing on their own. Verification requirements are expensive and work. Most of the disappointment in this field over the last decade comes from expecting Level 1 results from Level 1 effort. With the terms fixed, the rest of the book is implementation. The next chapter starts where a study starts: with writing down, before anything happens, what you intend to do. Hashtags: #TheReproducibilityBlueprint #Reproducibility #Replicability #OpenScience #ReplicationCrisis #Preregistration #RegisteredReports #ComputationalReproducibility #AnalyticRobustness #OpenData #OpenCode #OpenMaterials #VersionControl #ContainerizedWorkflows #DataArchiving #ResearchTransparency #SpecificationCurveAnalysis #MultiverseAnalysis #HARKing #ResearcherDegreesOfFreedom #PublicationBias #TOPGuidelines #ReproducibilityVerification #ResearchIntegrity #FutureOfOpenScience

Download the Book (PDF): Introduction Most scientists who end up leading a research consortium never set out to become managers of an organization. They wrote a strong proposal with colleagues they admired, the funder said yes, and one morning they found themselves responsible for forty or eighty or three hundred people spread across a dozen institutions, several time zones, four or five disciplines and at least two legal systems. Nothing in their training prepared them for this. A doctorate teaches you to formulate a question, design a study, defend an argument and supervise a handful of students. It does not teach you how to reconcile the data-sharing clauses of three university contracts offices, how to tell a senior co-investigator that her work package is holding everyone else up, or how to decide whose name goes last on a paper that eleven groups made possible. This book is written for that person, and for the people around them: the co-investigators, project managers, postdoctoral coordinators, program officers and research administrators who together determine whether a large collaborative grant turns into something more than the sum of its separately published parts. It draws on a body of research that has grown up over the past two decades under the somewhat self-conscious name "the science of team science", on older and deeper work in organizational behavior and the sociology of science, and on the practical experience of consortia in fields from genomics and high-energy physics to ecology, public health and climate modeling. The argument in brief The central claim of this book is simple to state and hard to act on. A research consortium is not a large laboratory. It is an organization, and its core product is integration: the combining of knowledge, data, methods and people that would otherwise remain apart. Integration does not happen as a by-product of excellent science done in parallel. It fails in predictable places, and those places can be named. Disciplines fail to understand one another, or believe they understand one another when they do not. Leadership shared between several principal investigators drifts into ambiguity or quiet rivalry. Credit for joint work is assigned by conventions that reward some contributors and erase others, which in turn shapes who is willing to do the integrative work at all. And communication across institutional and disciplinary boundaries thins out, so that a consortium that looked like a network on the organizational chart behaves, in practice, like a set of islands exchanging annual reports. Because these failure points are predictable, they are designable. The best-run consortia do not rely on goodwill and brilliance to carry them across these gaps. They build explicit structures: shared vocabularies and objects that disciplines can work on together, leadership agreements written before trouble starts, authorship policies that recognize the full range of contributions, and deliberate roles and rhythms for communicating across boundaries. None of this is glamorous. Much of it looks like administration. But the research evidence, and the accumulated experience of people who have run these projects, point the same way: the difference between consortia that produce genuinely integrated science and those that produce a bundle of loosely related papers lies less in the quality of the individual scientists than in how deliberately the collaboration itself is organized. Why this matters now Collaboration is no longer an optional mode of doing science. Over the second half of the twentieth century and into the twenty-first, the share of research produced by teams rather than individuals rose steadily in almost every field, including the humanities and mathematics, where solo authorship long remained the norm. Team size grew too. The largest projects in physics, genomics and astronomy now list authors in the hundreds or thousands. Funders have encouraged this shift through center grants, multi-site networks, "grand challenge" programs and framework programs that require partners from several countries. The practical problems, in other words, are not confined to a handful of megaprojects. Anyone who holds a multi-institution grant is running a small organization, whether or not they think of it that way. At the same time, there is good evidence that collaboration across boundaries is costly and frequently disappointing. Studies of multi-university projects have found that each additional institution adds coordination burdens that erode productivity unless actively managed. Interdisciplinary proposals have been shown to fare worse in competitive funding. Researchers whose work spans fields can find themselves prominent but less productive by conventional measures, and early-career scientists who invest in integrative roles often discover that tenure and promotion committees do not know how to evaluate what they did. These are not arguments against collaboration. They are arguments for taking its costs seriously and managing them with the same rigor that scientists bring to their own experiments. What the book covers The chapters move from the general to the specific and then back out to the life of a consortium as a whole. Chapter 1 sets out what is known about the rise of team science and what the research field that studies it has learned: where teams outperform individuals, where they do not, and which features of collaboration matter most. Chapter 2 treats the consortium explicitly as an organization, looking at structure, governance, coordination costs and the uneasy fit between project-management methods and open-ended research. The next two chapters take up translation across disciplines. Chapter 3 examines why disciplines misunderstand each other, drawing a distinction between problems of vocabulary, problems of meaning and problems of interest, each of which calls for a different response. Chapter 4 turns to the practical instruments of translation: the shared datasets, models, ontologies, protocols and physical spaces that let people from different fields work on something in common without first having to agree on everything. Chapter 5 addresses co-leadership, now the default arrangement for large grants in many funding systems, and asks what makes shared leadership work rather than dissolve into a committee without a chair. Chapter 6 deals with credit: authorship conventions, contributor-role taxonomies, the special problems of papers with hundreds of authors, and the career consequences of how credit is assigned. Chapter 7 is about communication across boundaries, the roles and routines through which information actually crosses between groups, and the conditions of trust and psychological safety that make people willing to share half-formed ideas and bad news. Chapter 8 follows a consortium through its life cycle, from formation through midlife to its end, which is the phase most consortia plan for least. The conclusion draws out what follows from all of this for the people who lead consortia, the institutions that host them and the agencies that fund them. A note on evidence The science of team science is a young field and an uneven one. Some of its findings rest on large bibliometric datasets covering tens of millions of papers; others rest on careful case studies of a few projects; still others draw on the much larger literature on teams in business, the military and healthcare, whose applicability to academic research is plausible but not always tested. Throughout, the book tries to be clear about which kind of evidence is being used and how far it can be pushed. Where the research is thin, the argument leans on well-documented cases and on reasoning from established organizational principles, and says so. Nor is the book a template. Consortia differ enormously. A ten-year particle-physics collaboration built around a single detector has little in common, organizationally, with a three-year network of social scientists and clinicians studying loneliness in older adults. What they share is the set of problems this book is organized around, and a set of principles for handling them that travel reasonably well across contexts. The aim is to give leaders a way of seeing their collaboration clearly enough to make good decisions about it, rather than a checklist to follow. One last point about tone. Much writing on collaboration is relentlessly upbeat, full of synergy and shared vision. The people who have actually run consortia tend to talk differently. They talk about the partner who never delivered, the dataset that took three years to harmonize, the co-lead who stopped answering email, the authorship dispute that ended a friendship. This book tries to keep faith with that experience. Collaboration across disciplines and institutions is one of the most powerful tools science has, and it is also hard, frequently frustrating and easy to do badly. Treating it as a craft that can be learned is the first step toward doing it well. Chapter 1: Why Teams Took Over Science, and What We Know About Them In 2007 three researchers at Northwestern University, Stefan Wuchty, Benjamin Jones and Brian Uzzi, published a short paper in Science that has since become one of the most cited reference points in any discussion of collaborative research. They examined nearly twenty million papers indexed over roughly five decades, together with more than two million patents, and asked a plain question: who produces knowledge, individuals or teams? Their answer was unambiguous. In science and engineering, in the social sciences and, to a lesser degree, in the arts and humanities, the share of work produced by teams had risen steadily throughout the period. Teams were getting larger. And papers written by teams were, on average, more highly cited than papers written by individuals, with the advantage growing over time. The lone genius, if he had ever been the main engine of discovery, was now the exception. That finding is the starting point for this chapter, but only the starting point. The more useful questions for anyone leading a consortium are why the shift happened, what it costs, and what two decades of research have established about which collaborations work and why. The burden of knowledge The most compelling explanation for the rise of teams comes from the economist Benjamin Jones, who in 2009 set out what he called the "burden of knowledge" hypothesis. The argument runs as follows. As a field accumulates knowledge, it takes longer for a new researcher to reach its frontier. Scientists respond by specializing more narrowly, because no one can master everything. But narrow specialists cannot, on their own, tackle problems that require several kinds of expertise. So they combine into teams. Jones found evidence consistent with this picture in patent data: the age at which inventors produce their first invention had been rising, specialization had been increasing, and team size had been growing. The burden-of-knowledge idea explains a great deal. It explains why teams have grown fastest in fields with rapidly expanding technical demands, such as genomics, where a single study may need molecular biologists, bioinformaticians, statisticians, clinicians and ethicists. It explains why collaboration has spread even to fields with a strong solo tradition. And it explains a feature of modern consortia that leaders often experience as a nuisance but which is in fact their reason for existing: the members genuinely do not understand one another's work in detail, because the whole point is that no individual could. A second driver is the nature of the problems funders and societies now want solved. Climate change, antimicrobial resistance, pandemic preparedness, the social determinants of health and the governance of artificial intelligence do not map onto any single discipline. Public funders have responded with programs explicitly designed to force cross-disciplinary work: center grants in the United States, networks and "synergy" grants in Europe, challenge-led programs in the United Kingdom and elsewhere. A third driver is infrastructure. Some science is only possible with instruments so expensive that they must be shared: particle accelerators, gravitational-wave observatories, large telescopes, national cohort studies, research vessels. The collaboration is built around the instrument, and its size follows from the instrument's scale. When the ATLAS and CMS collaborations at CERN published a combined measurement of the Higgs boson mass in 2015, the author list ran to more than five thousand names. A fourth driver, less often acknowledged, is the digital environment. Email, shared repositories, video calls and cloud computing have lowered the cost of working with people in other places. They have not eliminated that cost, as later chapters argue, but they have made it possible to sustain collaborations that would have been impractical a generation ago. The emergence of a field As teams became the default, a group of researchers began to treat collaboration itself as an object of study. The phrase "science of team science" came into wide use in the mid-2000s, associated particularly with work supported by the United States National Cancer Institute, which had funded large transdisciplinary center initiatives on tobacco use and on obesity and cancer and wanted to know whether they were working. In 2008 the American Journal of Preventive Medicine published a supplement on the topic, introduced by Daniel Stokols, Kara Hall, Brandie Taylor and Richard Moser, which set out the field's aims: to understand and improve the processes and outcomes of collaborative, team-based research. An annual Science of Team Science conference followed, and later an international professional association, INSciTS. The most authoritative synthesis came in 2015, when the United States National Research Council published Enhancing the Effectiveness of Team Science, a consensus report edited by Nancy Cooke and Margaret Hilton. The committee reviewed the evidence on teams in science and elsewhere and identified seven features that make team science distinctively challenging. These are worth stating because they recur throughout this book: high diversity of membership; deep knowledge integration; large size; goal misalignment with other teams or with members' home institutions; permeable team boundaries, with members joining and leaving; geographic dispersion; and high task interdependence. Most consortia exhibit all seven at once. The report's broad conclusion was measured. There was, it said, a substantial body of research on teams in general, much of it from organizational psychology, that very likely applied to science teams: findings about the value of clear goals, shared mental models, team processes such as reflection and feedback, psychological safety, and leadership that attends to both task and relationships. But direct evidence from science teams themselves was thinner, and evaluation of interventions designed specifically to improve team science was limited. The committee recommended that universities and funders invest in leadership training, revise promotion and tenure policies to recognize collaborative work, and support research on what actually improves outcomes. A later review by Kara Hall and colleagues, published in American Psychologist in 2018, reached a similar position. The field had matured, with a growing body of studies on team composition, processes, leadership and institutional support, but it still relied heavily on cross-sectional and case-study designs, and the causal links between particular practices and scientific outcomes were often inferred rather than demonstrated. This caution matters. A consortium leader should take the science of team science seriously but not literally. It offers well-grounded principles and a vocabulary for noticing problems early. It does not yet offer a validated manual. What teams do better, and what they do worse The first thing the evidence shows is that teams are not uniformly better. Wuchty, Jones and Uzzi found a citation advantage for team papers on average, and later work by Uzzi and colleagues found that the papers most likely to become highly cited combined deeply conventional foundations with unusual combinations of prior work, a pattern teams produced more often than individuals. Teams, in other words, seem to be good at bringing together knowledge that does not usually meet. But in 2019 Lingfei Wu, Dashun Wang and James Evans published a study in Nature that complicated the picture. Analyzing more than sixty-five million papers, patents and software products, they developed a measure of how far a piece of work disrupted its field, meaning how far later work cited it instead of, rather than alongside, the work it built on. Small teams, they found, tended to produce more disruptive work; large teams tended to develop and consolidate existing ideas. Both kinds of work are valuable. Large teams are well suited to taking a promising direction and pushing it to scale, testing it thoroughly and building the infrastructure around it. They are less well suited to overturning it. For consortium leaders this finding has a practical edge. A large consortium that wants to do genuinely novel work needs to protect small-team spaces within itself: pilot projects, exploratory working groups, seed funding for unexpected pairings. Several major programs do exactly this, reserving part of their budget for internal competitions that let two or three members pursue an idea the original proposal never anticipated. Without such spaces, the gravitational pull of a large collaboration tends toward consolidation. The second thing the evidence shows is that crossing boundaries is costly. The most influential study here is by Jonathon Cummings and Sara Kiesler, who in 2005 examined several dozen research projects funded under the National Science Foundation's Knowledge and Distributed Intelligence initiative, a program specifically intended to encourage collaboration across fields and institutions. They found that projects spanning more disciplines were not, in themselves, less productive. But projects involving more universities were: they used fewer coordination mechanisms per participant and reported fewer outcomes. Organizational distance, not disciplinary distance, was the bigger problem. In later work, Cummings and Kiesler found that the projects that succeeded despite multi-institution structures were those that invested in coordination, through face-to-face meetings, shared workshops, student exchanges and co-supervision, and that principal investigators who had worked together before coordinated more effectively than those who had not. This finding runs against a common intuition. People worry most about the intellectual difficulty of working with other disciplines, and that difficulty is real. But the grinding, everyday obstacles in consortia tend to be organizational: different academic calendars, different budget systems, different ethics review boards, different expectations about who owns data, the simple absence of chance encounters in a corridor. The disciplinary gap is visible and interesting; the institutional gap is invisible and boring, and so it is systematically neglected. The third thing the evidence shows is that cross-disciplinary work carries career and funding penalties that shape who is willing to do it. Lindell Bromham, Russell Dinnage and Xia Hua, analyzing proposals to the Australian Research Council's main discovery program, found in 2016 that the more interdisciplinary a proposal was, the lower its chance of funding. Erin Leahey, Christine Beckman and Taryn Stanko, studying a large sample of academic scientists, found in 2017 that interdisciplinary researchers achieved greater visibility but lower productivity, measured in publications, than their more specialized peers. These are average effects, and there are many exceptions. But they describe an environment in which the people a consortium most depends on, those willing to learn another field's language and build the bridges between groups, are taking a personal risk. A leader who ignores this will find that integrative work is always somebody else's job. From evidence to a working model Pulling these threads together yields a working model of the consortium that the rest of this book elaborates. The first element is that a consortium exists to integrate. If its members could do the work independently, there would be no reason for the collaboration beyond administrative convenience or funder requirements. Some consortia are in fact of this kind: umbrella grants that fund separate projects under a common title, with a shared annual meeting and little else. There is nothing dishonest about that, but its leaders should recognize what they are running. The harder and more valuable form of consortium is one in which the science genuinely depends on combining what the members know. Every structural and managerial choice should then be judged by whether it makes that combination more or less likely. The second element is that integration happens at specific points. It happens when a hydrologist and an economist sit down with the same dataset and discover that "water use" means different things to each of them. It happens when a lead investigator decides whether a junior modeler from a partner institution will be first author. It happens when a project manager notices that two work packages are building incompatible pipelines and gets their leads on a call. Most of the time in a consortium is spent not integrating: members are doing their own specialized work, as they should. The leader's job is to make sure the integration points exist, are staffed and are not left to chance. The third element is that the obstacles to integration fall into a small number of recurring types. There are problems of translation, when people from different fields do not share language, methods or standards of evidence. There are problems of leadership, when authority is shared, ambiguous or contested. There are problems of credit, when the reward system does not recognize the work that integration requires. And there are problems of communication across boundaries, when the channels between groups are too thin, too formal or too slow to carry what needs to cross them. Underlying all four is the organizational fact of dispersion: people who are not in the same place, not employed by the same institution and not accountable to the same people. The fourth element is that these problems change over time. The obstacles that dominate when a consortium is being formed, such as choosing partners, agreeing goals and negotiating contracts, differ from those that dominate in its middle years, when the risk is drift and fragmentation, and from those at its end, when the questions are about data, legacy and the careers of people whose funding is about to stop. A leader needs a sense of where the collaboration is in its life, and what that stage requires. A fifth element, which follows from the others, concerns how a consortium should judge its own success. The easiest measure is publication count, and funders' reporting templates tend to encourage it. But a count of papers says little about integration. A consortium can publish prolifically while its member groups work almost entirely apart, each producing the papers it would have produced anyway. More revealing questions are harder to answer from a spreadsheet. How many papers have authors from more than one discipline in the consortium, and from more than one institution? Did any member adopt a method, dataset or concept that originated with another group? Are there results that no single group could have produced? Have early-career researchers acquired skills that span fields? Have shared resources, such as datasets, software or protocols, been used by people outside the consortium? Stokols and colleagues, reviewing the transdisciplinary center initiatives in the late 2000s, argued for exactly this kind of multi-level evaluation, attending to changes in collaborators' thinking and working practices as well as to outputs. A leader who tracks integrative outcomes, even informally, will notice fragmentation early; one who tracks only volume will discover it at the final review. Much of what is known about teams comes from settings quite unlike academic research: aircraft cockpits, surgical theaters, military units, product-development groups in companies. It is tempting to import findings wholesale, and many of them do transfer. Clear goals help everyone. Teams that reflect on their own processes improve. Psychological safety, the shared belief that one can speak up without being punished or humiliated, predicts learning in science teams as it does in hospital wards. But academic consortia differ from most organizational teams in at least four ways that limit the analogy. First, their members are not employees of the consortium. They are employed by their home institutions, which have their own priorities, and they typically belong to several collaborations at once. A consortium leader has very little formal authority over anyone except, perhaps, the staff paid from the central budget. Second, their goals are often deliberately open-ended. A product team knows what it is building; a research consortium may discover halfway through that its original question was the wrong one, and that discovery may be its most valuable result. Third, the reward system is individual. Scientists are hired, promoted and honored as individuals, on the basis of individually attributed publications and grants, even when their work is collective. Fourth, the membership is highly autonomous by professional norm. Senior researchers are used to setting their own agendas and are, by temperament and training, skeptical of being managed. These differences explain why techniques that work in business settings sometimes fail badly in consortia. A Gantt chart imposed from the center can become a work of fiction that everyone updates the night before the funder's review. A formal performance-management process can alienate the very people whose voluntary effort the collaboration depends on. The science of team science is useful precisely because it has begun to identify what is distinctive about research collaborations, and what therefore needs to be adapted rather than copied. The next chapter takes up the first of these adaptations: how to think about a consortium as an organization, with a structure, a governance system and a set of coordination costs, while respecting the autonomy and open-endedness that make it a scientific enterprise rather than a firm. Chapter 2: The Consortium as an Organization Ask the principal investigator of a large research consortium to describe its structure and you will usually be shown a diagram of boxes: a steering committee at the top, a scientific advisory board off to one side, a set of work packages or research cores below, and perhaps a management office tucked somewhere in the middle. The diagram is almost always accurate and almost always misleading. It shows who reports to whom on paper. It says little about where decisions are actually made, where information actually flows, or where the collaboration is quietly failing. This chapter looks beneath the boxes. It treats the consortium as what organizational scholars would recognize it to be: a temporary, loosely coupled, multi-organizational system held together by a shared grant, a shared purpose and a web of personal relationships. Understanding that kind of system requires attention to three things: how the work is divided and recombined, how decisions are made and enforced, and what it costs to coordinate people who do not share an employer, a building or a discipline. Dividing the work and putting it back together Every consortium begins by dividing its work. The division is usually driven by the grant proposal, which assigns aims, work packages or cores to particular groups, and it tends to follow institutional and disciplinary lines, because that is how the partners came together. The epidemiologists get the cohort study, the statisticians get the methods core, the lab scientists get the assays, the policy researchers get the stakeholder engagement. There is nothing wrong with this, but it creates a structural problem that the organizational theorist James Thompson described in his 1967 book Organizations in Action. Thompson distinguished three kinds of interdependence among units. In pooled interdependence, each unit contributes separately to a common whole, and the units barely need to interact: think of branch offices of a bank. In sequential interdependence, one unit's output is another's input, as on an assembly line. In reciprocal interdependence, units feed outputs back and forth to one another, each adjusting to what the other produces. Thompson argued that these require increasingly costly forms of coordination: standard rules for pooled work, plans and schedules for sequential work, and continuous mutual adjustment for reciprocal work. Research consortia frequently plan for pooled or sequential interdependence and then discover that the science is reciprocal. A consortium studying the health effects of urban heat might plan for climate modelers to produce neighborhood-level temperature projections, which epidemiologists then link to hospital admissions, which economists then translate into costs. That is a sequential design, and it can be scheduled. But in practice the epidemiologists find they need the projections at a different spatial resolution, or for different time windows, or with uncertainty estimates the modelers had not planned to produce. The economists find they need the health outcomes categorized differently. Each group has to revise its work in light of the others, repeatedly. The plan assumed a pipeline; the reality is a conversation. The practical lesson is to identify, early and explicitly, which interfaces in the consortium are genuinely reciprocal, and to invest coordination effort there. A useful exercise at the start of a project is to draw not the organizational chart but the dependency map: for each work package, what does it need from others, in what form, and by when; and what might it need to send back? Interfaces where the answer to the last question is "quite a lot, and we are not sure what" are the places where integration will either happen or fail. They deserve joint working groups, shared personnel, regular technical meetings and, above all, someone whose job it is to watch them. A second lesson concerns the choice between organizing by discipline and organizing by problem. Many consortia divide their work by method or field, because that is how expertise is distributed. Others organize by question or case, with cross-disciplinary teams each tackling one part of the problem. The first design makes it easy to maintain methodological quality and hard to integrate; the second makes integration natural and risks spreading specialist expertise thin. A common compromise is the matrix: disciplinary cores that provide methods and maintain standards, crossed with problem-oriented projects that draw on several cores. Matrices are effective and notoriously hard to manage, because every person has two lines of accountability. They work when the consortium is explicit about which line governs which decisions, and they fail when that is left vague. Governance: who decides what Governance in a consortium means the arrangements by which decisions are made, disputes are resolved and resources are allocated. It is the part of consortium life that scientists most often treat as a formality, and the part that most often comes back to hurt them. Formal governance structures tend to follow a common pattern. There is usually a coordinator or lead institution, which holds the prime award and bears legal responsibility to the funder. There is a steering or executive committee, typically made up of the principal investigators or work-package leads, which makes strategic and scientific decisions. Many European consortia also have a general assembly with one representative per partner organization, which handles matters such as changes to the consortium's membership, budget reallocations and the consortium agreement itself; the model consortium agreements widely used in European framework programs, such as DESCA, are organized around exactly this division between an assembly of partners and an executive body. There may be an independent scientific advisory board, which offers external perspective and sometimes serves as a court of appeal. And there is often a management office, headed by a professional project or program manager, which handles reporting, finances, meetings and communications. The largest physics collaborations have developed more elaborate forms. Collaborations at CERN and the LIGO Scientific Collaboration, for example, elect a spokesperson to represent the collaboration and lead it for a fixed term, maintain collaboration boards with a representative from each member institution, and operate detailed internal rules for publication review and authorship. These arrangements evolved over decades to manage thousands of people with no single employer, and they have the character of a small constitutional democracy. Most consortia are far smaller and do not need anything so formal. But the underlying insight carries over: when members are autonomous and belong to different institutions, legitimacy has to be built through procedure, representation and transparency, because it cannot be derived from employment hierarchy. Three governance questions deserve particular attention because they are so often left unanswered until they become urgent. The first is how money moves. In most consortia the budget is divided among partners at the proposal stage, and each partner then treats its share as its own. This is administratively simple and strategically rigid. When the science changes direction, as it should, there is no mechanism for moving resources to where they are now needed without painful renegotiation. Well-designed consortia reserve a portion of the budget, sometimes a tenth or more, for allocation by the steering committee during the project, and agree in advance how that allocation will be decided. Some funders build this into their programs; others permit it if asked. The second is what happens when a partner underperforms. Every consortium of any size will face this at some point: a group that misses deadlines repeatedly, a lab that loses its key postdoc and cannot replace her, an investigator whose attention moves to another project. Without an agreed procedure, the choice is between tolerating the problem, which demoralizes everyone else, and confronting it personally, which can turn a scientific issue into a feud. A consortium agreement that sets out, in neutral terms, how performance concerns are raised, what steps follow and who decides on remedies, up to reallocating tasks and budget, gives the leadership a path that does not depend on personal courage or personal relationships. The third is how disputes are resolved. Disagreements over direction, data, authorship and resources are normal. What distinguishes healthy consortia is not the absence of conflict but the existence of a recognized process for handling it, with escalation steps that end somewhere other than the funder's program officer. Some consortia designate an ombudsperson or ask a member of the advisory board to play that role. The NIH's field guide on collaboration, written by L. Michelle Bennett, Howard Gadlin and colleagues, recommends that collaborators discuss in advance how they will handle disagreements, and many research offices now offer templates for exactly this kind of "collaboration agreement" or "prenuptial agreement" among investigators. The cost of coordination Coordination is the work of aligning the activities of people who are doing different things toward a common result. In a single laboratory much of it happens invisibly: people overhear each other, share equipment, eat lunch together, notice when something is going wrong. In a consortium spread across institutions and countries, almost all of that ambient coordination disappears, and whatever replaces it has to be deliberately arranged and paid for. Gary Olson and Judith Olson, in a much-cited 2000 paper titled "Distance Matters", examined why distributed collaborations so often underperform even when good communication technology is available. They identified four conditions that make remote collaboration workable. The first is common ground: the extent to which collaborators share knowledge, vocabulary and context, so that they can understand each other without constant clarification. The second is coupling of work: loosely coupled work, which can be divided into parts with clear interfaces, tolerates distance much better than tightly coupled work, which requires frequent, rapid and ambiguous exchanges. The third is collaboration readiness: whether the people and institutions involved actually want to collaborate and have the incentives to do so. The fourth is technology readiness: whether participants have the tools and the habits needed to use them effectively. Where any of these is missing, distance exacts a heavy price. The Olsons' framework remains useful because it explains a common pattern. Consortia that work well at a distance tend to be those where the tightly coupled work is kept within sites and the work that crosses sites is loosely coupled with clean interfaces. Consortia that struggle are often those where the tightly coupled work, typically the integrative analyses on which the whole project depends, has been split across institutions, because each partner wanted a share of the interesting part. It follows that one of the most consequential design decisions is where to locate the integrative core. Some consortia put it at the coordinating institution; others create a dedicated analysis or synthesis center; others rotate personnel so that people from different partners spend extended periods working in the same place. Face-to-face contact remains disproportionately important, despite everything technology has made possible. The Cummings and Kiesler studies described in Chapter 1 found that coordination mechanisms involving direct contact, such as workshops, visits and co-supervised students, were associated with better outcomes in multi-university projects. The experience of consortia during and after the COVID-19 pandemic, when in-person meetings stopped almost entirely, seems to have reinforced rather than overturned this. Many leaders report that established collaborations continued reasonably well online, drawing on trust built earlier, but that new relationships and genuinely new ideas were much harder to generate. A consortium budget that cuts travel to fund more personnel is often a false economy. Project management without pretending Research funders increasingly ask for the apparatus of professional project management: work plans, Gantt charts, milestones, deliverables, risk registers, key performance indicators. Many scientists regard this as bureaucratic theater, and some of it is. But the underlying tools are not the problem. The problem is using tools designed for predictable projects on work that is fundamentally uncertain. Classic project-management methods, of the kind codified in the Project Management Institute's body of knowledge, assume that the scope of a project can be defined in advance, broken down into tasks, sequenced and scheduled. That assumption fits some parts of a consortium well: building a database, running a multi-site recruitment, organizing a training program, procuring equipment, producing a report. It fits the research itself poorly, because the point of research is to find out things that were not known when the plan was written. A useful approach is to separate the two explicitly. The consortium's infrastructure and service activities, the parts that other groups depend on and that have knowable deliverables, can and should be managed with conventional discipline: clear owners, dates and dependencies. The research activities can be managed with lighter, iterative methods more akin to those used in software development: short planning cycles, regular reviews of what has been learned, and permission to change direction when the evidence warrants. Milestones for research are best framed as decision points rather than guaranteed results: by month eighteen, we will know whether approach A is viable, and we will decide whether to continue with it or switch to approach B. This framing has an important side effect. It makes it legitimate to report that something did not work. Consortia that frame every milestone as a promised result create an incentive to hide failure, which is exactly the information the rest of the consortium most needs. Consortia that frame milestones as decisions make failure reportable and therefore useful. The single most important project-management decision, however, is not about method. It is about people. The consortia that run best almost always have a dedicated, competent, adequately senior professional manager: someone whose job is the collaboration itself, who has the confidence of the leadership, who attends every steering meeting, who knows every work-package lead personally and who notices problems before they become crises. Research management has become a recognized profession, with its own associations and qualifications, and the best research managers combine an understanding of the science with genuine organizational skill. Yet many consortia still budget for this role at a junior level, or split it among several part-time administrators, or leave it to a postdoc who also has research to do. That is a false economy of the most damaging kind. The manager is often the person who holds the consortium together as an organization, and a consortium that under-invests in that role will pay for it in delays, disputes and missed integration. Most consortia are required to sign some kind of legal agreement among partners, covering intellectual property, confidentiality, liability, data ownership and the distribution of funds. These agreements are usually negotiated by university contracts offices, often slowly, and scientists tend to experience them as an obstacle to getting started. A better way to see the consortium agreement is as the consortium's first and most durable design document. It is the one place where the partners are forced to answer, in writing and in advance, questions that will otherwise be answered later under pressure: Who owns data generated jointly? Who may use it, and for what, and when? How are publications reviewed before submission? What happens if a partner leaves, or a new one joins? How are budget changes decided? What happens to shared resources when the grant ends? Leaders who leave all of this to the lawyers miss an opportunity. The legal text will protect the institutions; it will not necessarily reflect how the scientists want to work together. The most effective approach is for the principal investigators to agree the substance first, in plain language, often as a short memorandum of understanding or collaboration charter, and then ask the contracts offices to embody it. Where the legal agreement cannot cover everything, as it rarely can, a separate set of consortium policies, on publication, data sharing and authorship for example, can be adopted by the steering committee and revised as the collaboration learns. Chapter 6 returns to authorship policy in detail, and Chapter 8 to the questions of how agreements evolve across a consortium's life. The larger point of this chapter is that a consortium's organization is not a neutral container for its science. It shapes what science gets done. A structure that divides work along institutional lines will produce institutional outputs. A governance system that has no way to move money will lock in the original plan. A coordination budget that assumes pooled work will starve the reciprocal interfaces where integration happens. Designing the organization is part of designing the research, and the chapters that follow take up the specific points where that design matters most, starting with the problem of making different disciplines intelligible to one another. Chapter 3: Translation Across Disciplines Consider a composite case, assembled from situations that recur across interdisciplinary projects. A group of researchers studying antibiotic resistance in farming communities spends the better part of a year talking past each other about the word "exposure". To the microbiologists, exposure means the presence of resistant organisms or resistance genes in a sample taken from an animal, a person or a stream. To the epidemiologists, it means a measurable variable, defined in a study protocol, whose relationship to an outcome could be estimated. To the anthropologists, it means something closer to a lived relationship with animals, water and medicines, shaped by work, household arrangements and local understandings of illness. Each group is using the word precisely, by the standards of its own field. Each assumes the others mean the same thing. It takes a workshop at which they try to jointly design a single survey question before anyone realizes how far apart they are. Stories of this kind are so common in interdisciplinary research that they have become a genre. They are usually told as comedy, with the moral that people should define their terms. But the problem runs deeper than definitions, and treating it as merely verbal leads consortia to underestimate it. Disciplines differ not only in vocabulary but in what counts as a good question, what counts as evidence, how uncertainty is expressed, what a finished piece of work looks like and what their members stand to gain or lose from a given result. This chapter sets out a way of distinguishing among these kinds of difference, because each calls for a different response. Disciplines as cultures The sociologist Karin Knorr Cetina, in her 1999 book Epistemic Cultures, compared the ways that high-energy physicists and molecular biologists produce knowledge. She found that the two fields differed not merely in subject matter but in the whole machinery of knowing: how laboratories were organized, how individuals related to the collective, how experiments were understood, how errors were handled. High-energy physics, working with enormous shared detectors, had developed a culture in which the individual scientist was largely submerged in the collaboration and much effort went into understanding the limits and failings of the instrument itself. Molecular biology, working at the bench, was organized around individual laboratories and a more hands-on, trial-and-error engagement with material. The idea of disciplines as epistemic cultures helps explain why interdisciplinary collaboration is harder than it looks. Researchers from different fields are not simply specialists in different topics who can pool their knowledge like contributors to an encyclopedia. They are members of different communities with different tacit assumptions about how research should be done, and those assumptions are largely invisible to the people who hold them. A statistician trained in hypothesis testing may regard qualitative interview data as anecdote. An ethnographer may regard a statistician's variables as crude abstractions that strip away everything important. A physicist may regard a biologist's tolerance of messy, context-dependent findings as a lack of rigor; the biologist may regard the physicist's search for general laws as naive about living systems. None of these reactions is simply wrong, and none can be dissolved by a glossary. It helps to be clear about what kind of cross-disciplinary relationship a consortium is aiming for. In an influential 1992 paper, Patricia Rosenfield distinguished three levels. In multidisciplinary research, specialists work in parallel on a common problem from their own disciplinary perspectives, and their results are combined at the end. In interdisciplinary research, specialists work jointly, integrating their approaches around the shared problem. In transdisciplinary research, they go further, developing a shared conceptual framework that transcends their separate disciplines; later usage, especially in Europe, also applies the term to research that includes non-academic partners such as practitioners, communities and policymakers. These are not grades of virtue. Many problems are best tackled in a multidisciplinary way, and pretending otherwise wastes everyone's time. But a consortium that promised interdisciplinary or transdisciplinary integration in its proposal and has organized itself for multidisciplinary parallel work will not deliver what it promised, however good the individual components. Three kinds of boundary The most useful framework for diagnosing translation problems comes not from studies of science but from studies of product development. In papers published in 2002 and 2004, Paul Carlile, an organizational scholar then at MIT, examined how engineers, designers, manufacturing specialists and salespeople in a company developing new products managed to work together. He argued that knowledge boundaries between specialist groups come in three kinds, of increasing difficulty, each requiring a different response. As Table 1 sets out, the distinction maps well onto the experience of research consortia. Table 1. Three kinds of knowledge boundary and how consortia can respond to each, adapted from Carlile's framework. Boundary What differs Typical symptom Useful response Syntactic Terms, formats, codes Data will not merge; files are unreadable across groups Shared standards, data dictionaries, common formats Semantic Meanings and interpretations Same word, different concept; results are misread Joint problem framing, worked examples, translators Pragmatic Interests and stakes Resistance to changes that would cost one group its methods or credit Negotiation, shared goals, redesign of incentives At a syntactic boundary, the groups share enough understanding that the problem is simply one of transferring information in a common format. If two genomics centers use different file formats or naming conventions, the solution is a standard. Syntactic problems are real and can be expensive, but they are tractable: once the right standard is agreed, information flows. Much of the work of large data consortia consists of solving syntactic problems, and it should not be underestimated. At a semantic boundary, the groups use terms and representations that look shared but carry different meanings. This is the "exposure" problem. A standard alone does not solve it, because both groups can fill in the same field of the same template while meaning different things. Semantic boundaries require translation: people must work together on concrete problems until they discover where their interpretations diverge, and then develop shared understandings, or at least mutual understanding of the difference. This takes time, face-to-face contact and, very often, a specific person who understands both sides. At a pragmatic boundary, the groups understand each other perfectly well, but integrating their knowledge would require one or both to change, and change is costly. Carlile emphasized that specialists have invested heavily in their existing knowledge. If the consortium's integrated model requires the field ecologists to collect data in a way that makes their long-term time series incompatible with the past, or requires the modelers to abandon a framework they have spent a decade building, the resistance is not a misunderstanding. It is a rational defense of real interests. Pragmatic boundaries cannot be crossed by better communication alone. They require negotiation, compromise and sometimes a change in the incentives facing the people involved. Carlile's key insight for consortium leaders is that these boundaries are nested and that people routinely misdiagnose them. A consortium that treats a pragmatic conflict as a semantic misunderstanding will organize more workshops to clarify terms, which will fail, because the parties already understand each other. A consortium that treats a semantic gap as a syntactic one will adopt a common data standard, which will be filled in inconsistently, because the terms mean different things. And a consortium that treats every difficulty as a pragmatic conflict of interest will generate suspicion where there was only confusion. The first step in managing a translation problem is to ask which kind it is. A practical test helps. If the groups, once shown each other's usage, say "oh, you mean that", the boundary was semantic. If they say "yes, we understood, but we cannot do it your way", the boundary is pragmatic. If they say "we just need the file in a different format", it is syntactic. Trading zones and interactional expertise Two ideas from the history and sociology of science illuminate how translation actually happens. The first is Peter Galison's concept of the trading zone, developed in his 1997 history of twentieth-century physics, Image and Logic. Galison observed that theorists, experimentalists and instrument builders in physics worked in different subcultures with different languages, yet managed to coordinate closely. He borrowed from anthropology and linguistics to describe how. When different groups trade, he noted, they do not need to share a full language or worldview. They develop local coordinating languages, which begin as limited pidgins sufficient for specific exchanges and can, over time, grow into full creoles with their own grammar and communities of native speakers. Radar research during the Second World War, Galison argued, produced exactly such a zone, in which physicists and engineers developed a shared language that later helped shape postwar physics. The trading-zone idea is liberating for consortium leaders, because it suggests that disciplines do not have to reach deep agreement before they can work together productively. They need a shared zone, a set of problems, objects and terms around which exchange is possible, even if each side interprets the exchange differently in its own terms. Many successful consortia have created such zones deliberately: a shared model to which each group contributes a module, a joint field site, a common dataset, a recurring problem-solving session in which everyone works on the same concrete case. Chapter 4 takes up these shared objects in detail. The second idea is interactional expertise, a term developed by the sociologists Harry Collins and Robert Evans. Collins, who spent decades studying the gravitational-wave physics community, noticed that he could converse fluently with physicists about their field and even pass informal tests of expertise, without being able to do the physics himself. He called this interactional expertise, as distinct from contributory expertise, the ability to contribute to a field's research directly. Interactional expertise is acquired by immersion in a field's discourse, through extended conversation with practitioners, rather than by doing its practical work. For consortia, the concept identifies exactly what boundary-crossing members need. It is unrealistic to expect an economist to become a hydrologist. It is realistic, and enormously valuable, for an economist in a water consortium to acquire enough interactional expertise in hydrology to know what hydrologists care about, what their results can and cannot support, and what questions to ask them. Such people are the translators through whom semantic boundaries are crossed. They are made, not born, and they are made by sustained contact, which is one more reason why consortia that economize on travel, joint workshops and cross-site visits struggle to integrate. It also suggests a useful piece of self-knowledge for leaders. Most principal investigators have contributory expertise in one field and some interactional expertise in one or two others. A consortium leader who lacks interactional expertise in a field central to the project will find it very hard to judge that field's contributions or to mediate between it and the rest. The honest response is either to acquire the expertise, which takes time, or to ensure that someone in the leadership team has it. Building integrative capacity The research on how teams integrate diverse knowledge points to several practices that consortia can adopt deliberately. The first is structured dialogue about assumptions. The Toolbox Dialogue Initiative, which grew out of work by Sanford Eigenbrode and colleagues published in 2007, developed workshops in which members of cross-disciplinary teams respond to a set of prompts about their research assumptions, such as whether scientific research must be hypothesis-driven, whether values have a role in science, whether reductionism is appropriate, or what counts as confirmation, and then discuss their answers together. The aim is not to reach agreement but to surface differences that otherwise remain tacit. Participants frequently report discovering that colleagues they had worked with for years held assumptions they had never suspected. A few hours spent this way at the start of a consortium can save months of confusion later. The second is joint work on concrete cases early. Abstract discussions of disciplinary difference tend to produce polite agreement and little insight. Asking a mixed group to design one survey instrument, specify one model component or analyze one small shared dataset together reveals semantic boundaries quickly, because the differences show up in decisions that have to be made. The "exposure" consortium described at the start of this chapter discovered its problem only when it tried to write one question together. The lesson is to arrange such encounters deliberately and early, rather than waiting for them to happen when the stakes are higher. The third is investing in people who can bridge. Research on team integration, including work by Maritza Salazar, Theresa Lant, Stephen Fiore and Eduardo Salas on what they called integrative capacity, emphasizes that diverse teams innovate when they have the social and cognitive processes to combine their knowledge, and that individual members with experience across fields contribute disproportionately to this capacity. Consortia can cultivate such people: by funding postdoctoral positions explicitly shared between two groups, by co-supervising doctoral students across disciplines, by running summer schools in which members teach each other their fields' basics, and by rotating early-career researchers between sites. The researchers who emerge from this kind of training are often the consortium's most durable legacy. The fourth is a shared glossary, used carefully. A living document that records how key terms are used in each field, and where the usages differ, can be a valuable tool, provided it is understood as a record of difference rather than a set of imposed definitions. The goal is not to make the anthropologists use "exposure" the way the epidemiologists do; it is for everyone to know that they do not, and to be precise in writing about which sense is meant. The fifth concerns pragmatic boundaries specifically. When integration requires one group to pay a cost, whether in abandoned methods, delayed publications or lost credit, the leadership must acknowledge it openly and, where possible, compensate for it. That might mean guaranteeing the group a leading role in the resulting publications, funding the work needed to maintain continuity with its past data, or sequencing the project so that the group's own disciplinary outputs are secured before it is asked to adapt. Pragmatic boundaries are crossed by bargains, and bargains are fairer and more durable when they are made explicitly. It would be comforting to conclude that every disciplinary gap can be bridged with enough goodwill and effort. It cannot. Some differences reflect genuinely incompatible views about what is true or what matters, and forcing a synthesis produces something weaker than either original. A consortium studying the social dimensions of a new technology may include researchers who believe the technology's risks can be quantified and others who believe the quantification itself obscures the most important questions. Neither will convert the other, and neither should have to. In such cases the best outcome may be a productive disagreement, clearly articulated, rather than a forced consensus. Some of the most valuable outputs of interdisciplinary consortia are papers that set out, carefully and respectfully, why different fields reach different conclusions about the same problem, and what each view captures that the other misses. Leaders should be prepared to recognize when integration has reached its useful limit, and to make space for pluralism rather than demand a single framework. Translation is a means to better science, not a goal in itself, and sometimes better science means knowing precisely where two languages cannot be reconciled. Hashtags: #ScienceOfTeamScience #TeamScience #CrossDisciplinaryResearch #ResearchConsortia #InterdisciplinaryResearch #TransdisciplinaryResearch #KnowledgeIntegration #ConsortiumGovernance #ResearchLeadership #CoLeadership #SharedLeadership #BoundarySpanning #KnowledgeBoundaries #SyntacticBoundaries #SemanticBoundaries #PragmaticBoundaries #TradingZones #InteractionalExpertise #IntegrativeCapacity #PsychologicalSafety #ResearchCoordination #AuthorshipCredit #ResearchCommunication #ConsortiumLifecycle #FutureOfTeamScience Pasted markdown

Download the Book (PDF): Introduction Every working scientist carries two kinds of record. The first is the laboratory notebook, the electronic lab book, the folder of raw instrument files and the scripts that turned them into numbers. It is chronological, exhaustive and honest in the way a diary is honest: it records what was done, in the order it was done, including the dead ends, the reagent that turned out to be contaminated and the three weeks lost to a miscalibrated pipette. The second record is the published paper. It is short, selective and ordered not by time but by logic. It tells a reader what was learned and why they should believe it. Most of the difficulty of scientific writing lives in the gap between these two records. Early-career researchers often assume that a paper is the notebook tidied up: the same story, told more formally, with the embarrassing parts removed. That assumption produces a recognisable kind of manuscript. It opens with a long history of the field, describes experiments in the order they happened, reports every result with equal weight, and ends with a discussion that restates the results and adds that more work is needed. Reviewers call such papers "descriptive" or "lacking a clear message". Editors decline them without sending them for review. The work inside may be excellent, but the paper does not let anyone see that. The central claim of this booklet is simple and, once absorbed, changes almost every decision a writer makes. A scientific paper is an argument, not a record. It exists to establish one principal claim, supported by evidence, qualified by honest limits, and placed in relation to what others already believe. Every structural element of the paper, from the title to the last sentence of the discussion, is there to serve that argument. Every stage of publication after the writing, from choosing a journal to answering a hostile reviewer to reading an editor's letter, is a continuation of the same argument with a new and more sceptical audience. This view is not a rhetorical trick imposed on science from outside. It follows from what a paper is for. A reader of a research article is almost never trying to relive the researcher's experience. They are trying to decide whether to believe something, whether to build on it, whether to cite it, fund it, replicate it, or teach it. They need the claim, the evidence for it, and the reasons it might be wrong, and they need them in a form they can evaluate quickly. The notebook serves the researcher's memory and the integrity of the record. The paper serves the reader's judgement. What goes wrong, and why The failures of scientific papers are remarkably consistent across disciplines. A structural biologist, an ecologist and a clinical epidemiologist will write very different papers, yet the reviews those papers attract share a vocabulary. The contribution is unclear. The introduction does not establish why the question matters. The conclusions overreach the data. The methods are insufficient to reproduce the work. The discussion does not engage with competing explanations. The abstract promises something the results do not deliver. Almost every one of these complaints is a complaint about argument rather than about prose. Sentence-level polish matters, and later chapters address it, but a beautifully written paper with no clear claim is still unpublishable in a selective journal, while a plainly written paper with a sharp claim and clean evidence often sails through. This is why the booklet spends more time on structure and framing than on style. Style is how you make a clear argument easy to follow. It cannot supply an argument that is not there. There is a second, quieter reason papers go wrong. Scientists are trained intensively in doing research and almost never in writing about it. Writing is often learned by imitation, by reading papers in one's own field and absorbing their conventions without being told what the conventions are for. The result is that writers reproduce the surface of good papers, such as the section headings, the passive voice and the hedged conclusions, without the underlying logic that makes the best papers persuasive. Knowing why the Introduction narrows toward a gap, or why the Results section should be organised around claims rather than experiments, allows a writer to use the conventions deliberately instead of by habit. How this booklet is organised The chapters follow the life of a paper, from the moment a set of results begins to look like a finding to the moment an editor makes a final decision. The first chapter deals with the step most writers skip: deciding what the paper claims before writing any of it. It offers practical ways to extract a single, defensible central claim from a mass of results, and to decide what belongs in this paper and what belongs in another one or in none. The second chapter treats the standard structure of the research article, Introduction, Methods, Results and Discussion, as a logical architecture rather than a filing system. Each section answers a specific question a sceptical reader will ask, and understanding those questions makes the structure easier to fill. The third chapter is about the Introduction, and in particular about the move that makes an Introduction work: establishing a gap in knowledge that matters, and positioning the paper to fill it. The fourth deals with Methods and Results together, as the evidence chain on which the argument rests, including how to report statistics honestly and how to use the reporting guidelines that many journals now require. The fifth chapter addresses the Discussion, the section where most papers either earn their significance or overreach and lose it. The sixth turns to titles and abstracts, which are written last but read first and, for most readers, are the only parts of the paper they will ever see. The final three chapters move from writing to publishing. The seventh deals with choosing a journal, preparing a submission and writing a cover letter that gives an editor a reason to send the paper out. The eighth deals with peer review: how to read critical reports without either capitulating or becoming defensive, and how to write a response letter that makes a reviewer's job easy. The ninth explains editorial decisions, what the various verdicts actually mean, and how to decide among revising, appealing, transferring and walking away. What this booklet assumes, and what it leaves out The booklet is written for academic scientists in the natural, life and health sciences who have data and want to publish it well: doctoral students writing a first paper, postdoctoral researchers building a record, and principal investigators who supervise others and want a shared framework for doing so. Examples are drawn from several fields because the logic of argument is the same across them, though conventions differ in detail. Where a convention is specific to a field, such as the structured abstracts of clinical journals or the combined Results and Discussion sections common in chemistry, the text says so. It does not attempt to cover everything. It does not teach English grammar, although it discusses the handful of sentence-level habits that most affect clarity. It does not cover grant writing, review articles or theses, which have related but different logics. It says relatively little about research ethics and misconduct beyond what bears directly on honest reporting, because those subjects deserve full treatment of their own. And it does not offer a formula for getting into any particular journal, because no honest formula exists. What it offers instead is a way of thinking about a paper that makes every journal decision easier to understand and every revision easier to plan. A word on "high-impact" journals, since the subtitle mentions them. Selective journals are not a separate genre with secret rules. They apply the same standards as any good journal, with less tolerance for an unclear claim and a higher bar for how broadly the claim matters. The habits that get a paper into a leading general journal are the habits that make any paper better: a sharp claim, evidence arranged to support it, candour about its limits and a clear account of why it matters to people outside the immediate subfield. Pursuing those habits is worthwhile whether or not a given paper is aimed at the top of its field. And the booklet will be candid, where it matters, about the costs of chasing prestige for its own sake. The notebook remains the foundation of everything that follows. No amount of skilful framing can rescue results that are not there, and the argument of a paper is only as trustworthy as the record beneath it. The aim here is not to make weak work look strong. It is to make sure that strong work is seen for what it is. Chapter 1: From Notebook to Claim The most important sentence in a scientific paper is one that may never appear in it verbatim. It is the sentence that states what the paper establishes: the single claim that a reader should believe after reading it that they did not believe, or did not know, before. Writers who can state this sentence before they begin drafting tend to produce papers that are coherent and persuasive. Writers who cannot state it tend to produce papers that wander, and no amount of revision at the level of paragraphs will fix them, because the problem is upstream of the paragraphs. This chapter is about finding that sentence. It is the step most often skipped, partly because it feels like a delay before the "real" writing and partly because it is uncomfortable. Committing to a central claim means deciding what the work does not show, which results are peripheral, and which months of effort will be reduced to a line in the supplementary material or left out altogether. That discomfort is the price of a readable paper. Why the notebook order misleads Research proceeds by accident and revision. A project begins with one question, runs into a technical obstacle, pivots to a related question, discovers something unexpected along the way, and ends somewhere its originators did not foresee. The notebook records this path faithfully, and it is natural for the writer, who lived through it, to feel that the path is the story. It almost never is. Readers do not need the path; they need the destination and the evidence that the destination is real. Consider a cell biology group that set out to characterise a protein's role in membrane trafficking, spent a year establishing a knockout line, found no trafficking defect, and noticed along the way that the knockout cells divided abnormally. After another year of work on the division phenotype they have a strong result about mitosis. The chronological story begins with trafficking. The paper, if it is to be clear, begins with mitosis. The trafficking experiments may appear as a brief negative result, or as supplementary controls, or not at all. Beginning with the trafficking hypothesis forces the reader to hold a question in mind for pages only to learn it was the wrong question. The chronological instinct shows up in subtler ways, too. Results sections that begin "We first..." and continue "Next, we..." and "We then..." are often narrating the notebook. Sometimes the order of experiments happens to be the logical order, and then the narration is harmless. More often, the logical order differs: the experiment that most directly supports the claim was done late, after the tools were in place, while early experiments were exploratory and only make sense in its light. A reader should meet the evidence in the order that builds the argument, not the order in which the evidence was gathered. None of this licenses dishonesty about how the work was done. If a hypothesis was formed after seeing the data, the paper must not present it as having been specified in advance. This is the practice known as HARKing, hypothesising after the results are known, a term introduced by the psychologist Norbert Kerr in 1998, and it misrepresents the evidential weight of a finding. Reordering the presentation for logic is legitimate. Rewriting the history of what was predicted and what was discovered is not. The distinction is important enough to return to in the chapters on Results and Discussion. Stating the claim A central claim has a recognisable shape. It is a declarative statement about the world, not about the authors' activities. "We investigated the role of protein X in cell division" describes what the authors did. "Protein X is required for correct spindle orientation in dividing epithelial cells" states what they found. The first sentence could be written before any experiment was done; the second can only be written because the experiments succeeded. A strong claim has four properties. It is specific enough to be wrong: a reader can imagine evidence that would contradict it. It is supported by the data in hand, without needing results that are still pending. It is scoped correctly, neither shrunk to triviality nor inflated beyond what the evidence supports. And it matters to someone beyond the authors: it changes what a defined group of readers believes, does or can do. The scoping is where most of the judgement lies. A claim that protein X is required for spindle orientation "in dividing cells" generalises from the cell types actually studied to all cells. If the work used two epithelial lines and one organoid system, the claim should say "in epithelial cells", or should make the generalisation explicit as a hypothesis in the Discussion. Conversely, a claim that "knockdown of X in MDCK cells produced a 23% increase in spindle misorientation under our culture conditions" is so narrowly scoped that it no longer says anything a reader can use. The right claim lives between these, at the level of generality the evidence can bear. One practical test is to write the claim, then imagine the most knowledgeable sceptic in the field reading it. What is the first objection they would raise? If the objection is "that's obvious" or "who cares", the claim is too small or poorly connected to a larger question. If it is "you haven't shown that", the claim is too large. If it is "how do you know it's X and not Y?", the claim may be right but the paper needs to confront Y explicitly. The central claim should be the strongest statement that survives this imagined challenge. Several practical ways to find the claim Many writers know the claim instinctively once asked. Others, especially those who have been close to the data for a long time, find it genuinely hard to see. Several exercises help. The first is to explain the work aloud to a colleague outside the subfield in two minutes. Spoken explanation forces selection: there is no time to describe every experiment, so the speaker gravitates toward the finding and the key evidence. Recording or transcribing the explanation often yields a better first draft of the claim than hours at a keyboard. The second is to draft the title and a two-sentence abstract before anything else. The title should be a statement of the finding where the field's conventions allow it. The two sentences should say what was found and why it matters. These will be rewritten later, but writing them first exposes whether a clear claim exists. The third, recommended in various forms by many experienced writers, including the chemist George Whitesides in his widely circulated guidance to his research group, is to begin from the data displays rather than from prose. Whitesides advised his students to start a paper with an outline built around the key figures, tables and schemes, because they embody the evidence and their order embodies the argument. Arrange the candidate display items on a table, physically or on screen. Ask which single panel, if the reader saw nothing else, would come closest to proving the claim. That panel usually belongs early in the Results. Ask which displays are necessary to rule out the obvious alternative explanations. Those follow. Displays that do neither are candidates for the supplement. The fourth exercise is to ask what the paper would lose if a given result were removed. If the answer is "nothing about the central claim", the result may still be worth reporting, but it should not shape the structure. Papers that try to give every result equal prominence end up with no prominent result at all. One paper, one claim A recurring question is how much to put in a single paper. Researchers feel pressure in two directions. Career incentives that count publications encourage splitting a body of work into the smallest publishable units, a practice sometimes called salami slicing. Selective journals, on the other hand, often demand a complete story, and researchers aiming high may pile several findings into one manuscript in the hope that the accumulated weight will impress. Both tendencies can damage a paper. Excessive splitting produces papers whose claims are too thin to matter and forces readers to assemble the real finding from fragments scattered across several journals. It can also shade into redundant publication, which journals and bodies such as the Committee on Publication Ethics treat as misconduct when the same data are presented as new more than once. Excessive bundling produces papers with no clear centre, where reviewers cannot tell which of three findings is supposed to be the contribution and consequently judge each one as underdeveloped. The guiding principle is that a paper should contain one central claim and everything necessary to make it credible, and should not contain a second, independent claim that requires its own set of supporting evidence. Supporting claims are fine and usually necessary: a paper showing that a gene variant increases disease risk may need to show that the variant alters expression, that the altered expression affects a pathway, and that the effect replicates in an independent cohort. These are steps in one argument. A paper that additionally reports a new sequencing method developed along the way, validated in its own right, may contain two papers. Mensh and Kording, in their widely read 2017 article "Ten simple rules for structuring papers" in PLOS Computational Biology, put the principle bluntly: focus the paper on a single message. Their reasoning is that readers remember very little of what they read, and a paper that tries to communicate several messages will likely communicate none. A paper with one message, clearly repeated in title, abstract, Introduction, Results and Discussion, has a chance of being remembered and cited for it. Negative, null and inconvenient results Not every body of work leads to a positive claim, and the argument framework applies equally to null results, though the claim is different in kind. A well-designed study that found no effect can support a claim of the form "any effect of X on Y in this population is smaller than Z", provided the study had adequate precision to rule out effects of meaningful size. This is a real contribution, particularly when a positive result has been widely assumed or reported in smaller studies. The central claim in such a paper is about the bounds on an effect, and the evidence that supports it is primarily the confidence interval and the study's power, not the absence of statistical significance, which on its own tells a reader very little. Inconvenient results within an otherwise positive study pose a different problem. Suppose three of four experiments support the claim and the fourth does not. The honest options are to report all four and explain why the fourth may differ, to narrow the claim so that it covers only the conditions in which it holds, or to do further work to understand the discrepancy. Quietly omitting the fourth is not an option; it is a form of selective reporting that distorts the evidence and, when discovered, damages the authors' credibility far more than the anomaly would have. Well-handled anomalies often strengthen a paper, because they show readers that the authors looked for trouble and are reporting what they found. Knowing the audience before writing A claim does not exist in a vacuum; it matters to particular people. Deciding who those people are is part of deciding what the claim is, because the same finding can be framed differently for different readerships. Take a result showing that a common agricultural fungicide alters foraging behaviour in bumblebees at field-realistic doses. For a readership of insect neuroethologists, the interesting claim may be about the specific neural mechanism. For ecologists, it is about the consequences for pollination services. For environmental policy readers, it is about whether current regulatory testing, which may focus on lethal doses, misses sublethal behavioural effects. The data are the same; the central claim and its framing change with the audience. The choice should be made deliberately, early, and in coordination with the choice of journal, which Chapter 7 addresses. The practical upshot is that the writer should be able to finish the sentence "This paper will change the mind of..." with a specific group of people, and "...about..." with a specific belief. If the honest answer to the first half is "a handful of specialists in my subfield", that is fine, and the paper should be written for them and submitted to a journal they read. If the answer is broader, the paper must be written so that the broader audience can follow it, which usually means more explanation of why the question matters and less assumed background. Writing the claim down, and keeping it Once found, the central claim should be written at the top of the draft document, above the title, where it will be seen every time the file is opened. It is scaffolding and will be removed before submission, but while it is there it serves as a constant test: does this paragraph, this experiment, this sentence in the Discussion serve the claim? Material that does not is either supporting context that needs to be tied more explicitly to the argument, or it is material that belongs elsewhere. The claim will often change during writing, and it should be allowed to. Drafting is a form of thinking, and the act of arranging evidence frequently reveals that the claim is slightly different from what the authors believed. A group may discover, when writing the Results, that their strongest evidence actually supports a narrower claim than planned, or that an experiment they considered peripheral is in fact the linchpin. When this happens, the claim at the top of the document should be revised, and the rest of the paper realigned to match. What should not happen is the claim drifting silently, so that the Introduction sets up one argument and the Discussion concludes another. Reviewers notice this immediately, and it is one of the commonest reasons for a verdict that the paper "lacks focus". The claim and the co-authors Most research papers have several authors, and the central claim is something they must agree on. This is not always easy. A graduate student who spent two years on a technically demanding assay may feel that the assay is the contribution, while the principal investigator sees it as a means to a biological finding. A clinical collaborator may want the paper to emphasise patient outcomes while the laboratory partners want to emphasise mechanism. These disagreements are real and legitimate, and they are far cheaper to resolve before drafting begins than after a complete manuscript has been written around one person's view. A short meeting at the outset, with a one-page document stating the proposed claim, the intended audience, the target journal and a list of the key display items in order, saves a great deal of later conflict. The same meeting is the right time to settle authorship. The International Committee of Medical Journal Editors sets out widely used criteria requiring that each author has made substantial contributions to the work, drafted or critically revised the manuscript, approved the final version and agreed to be accountable for it. Many journals now also ask for contribution statements using the CRediT taxonomy, which lists fourteen roles such as conceptualisation, methodology, formal analysis and writing. Agreeing on who did what, and in what order names will appear, while the work is fresh prevents disputes that can otherwise sour collaborations for years. From claim to plan With a central claim agreed, a writer is ready to plan the paper. The plan should consist of the claim, a short list of supporting claims each tied to specific evidence, a list of the alternative explanations that a sceptic would raise and the evidence that addresses each, an honest list of the limitations that cannot be addressed, and a statement of why the claim matters and to whom. This plan maps almost directly onto the sections of a research paper, which is not a coincidence. The standard structure of the scientific article evolved as a way of presenting exactly this kind of argument, and the next chapter examines it in that light. The time spent here is repaid many times over. A paper built on a clear claim is easier to write, because every section has an obvious job. It is easier to review, because the reviewer can see what is being argued and evaluate the evidence for it. And it is easier to revise, because criticisms can be sorted into those that threaten the central claim, which must be addressed fully, and those that concern peripheral matters, which can be handled more lightly. The claim is the organising principle of everything that follows. Chapter 2: The Architecture of Argument The standard research article in the natural and health sciences is divided into an Introduction, a Methods section, a Results section and a Discussion. The format is so familiar, and so often imposed by journal instructions, that it can feel like bureaucracy: a set of boxes into which material must be sorted. Writers who treat it that way produce papers in which each box is filled but nothing connects them. The Introduction reviews the literature, the Methods list procedures, the Results report numbers and the Discussion comments on them, and the reader is left to assemble an argument the authors never made. The structure is better understood as the shape of an argument addressed to a sceptical reader. Each section answers a question that such a reader will ask, in the order in which they will ask it. Why should I care about this? What did you do? What did you find? What does it mean, and should I believe it? Seen this way, the format is not a constraint on argument but a tool for making one, and its conventions, such as the narrowing Introduction, the neutral Methods and the widening Discussion, all follow from the questions each section must answer. Where the structure came from The four-part structure, usually abbreviated IMRaD, is younger than many scientists assume. Early scientific papers, such as those in the seventeenth-century Philosophical Transactions of the Royal Society, were often letters, narrating observations and experiments in the first person and in chronological order. Over the nineteenth and twentieth centuries, as the volume of research grew and readers needed to find and evaluate information quickly, papers became more standardised. A survey of articles in four leading medical journals by Sollaci and Pereira, published in the Journal of the Medical Library Association in 2004, found that IMRaD was used by only a minority of papers in the 1940s, was adopted increasingly through the 1950s and 1960s, and had become the near-universal standard for original research in those journals by the 1980s. The reason it prevailed is instructive. IMRaD lets different readers use the same paper in different ways. A reader deciding whether a paper is relevant reads the title and abstract. A reader deciding whether to believe it reads the Methods closely. A reader wanting the finding goes to the Results. A reader wanting to place it in context reads the Introduction and Discussion. The structure is a kind of index to the argument, and its predictability is a service: a reader who knows where to find each component can evaluate the paper more efficiently. It follows that departing from the structure has costs, and should be done only for good reason. Many fields and journals have legitimate variants. Chemistry and materials science journals often combine Results and Discussion into one section, because each result is interpreted immediately before the next experiment is introduced. Several high-profile general journals place the Methods at the end, after the Discussion, or move most methodological detail online, on the reasoning that most readers want the findings first. Some fields in the physical sciences and mathematics use structures organised around theoretical development rather than experiment. The underlying questions remain the same in all of these, and a writer who understands the questions can adapt to any format a journal requires. The hourglass A useful image for the overall shape of a research article is the hourglass. The paper begins broad, narrows to a specific question, stays narrow through the Methods and Results, and widens again in the Discussion. The Introduction starts with a context that a wide readership recognises as important, for example a disease, an ecological process, a class of materials or a theoretical problem. It narrows through what is known, to what is not known, to the specific question the paper addresses. By the end of the Introduction, the reader should know exactly what question is being asked and, usually, what the answer turned out to be. The Methods and Results sections sit at the narrow waist. They concern this study, these samples, these measurements and these findings. They do not wander back into the general literature, except to cite the sources of methods or to compare a specific measurement with a specific published value. The Discussion widens again. It restates the principal finding, then considers what it means for the specific question, how it compares with other work, what its limitations are, and finally what it implies for the broad context that opened the paper. The last paragraph of the Discussion should, in a well-constructed paper, speak to roughly the same audience as the first paragraph of the Introduction. The hourglass explains several common faults. An Introduction that stays broad for too long, spending pages on general background before reaching the question, has an hourglass with a flat top. A Discussion that merely restates the results, never widening to their implications, has an hourglass with no bottom bulb. A Results section that keeps returning to the literature to justify each experiment has a waist that bulges. Recognising the shape helps a writer diagnose which part of the paper is failing. The question each section answers It helps to be explicit about what each section is for, what it typically sounds like, and how it most often goes wrong. Table 1 sets out these elements for the four core sections. Table 1. The four core sections of a research article and the reader's question each answers. Section Reader's question Main job Typical tense Common failure Introduction Why should I care, and what exactly are you asking? Establish importance, identify the gap, state the question Present for established knowledge; past for specific prior studies Literature review with no gap; question stated late or not at all Methods What did you do, and could I do it again? Make the work evaluable and reproducible Past Missing detail; methods cited to papers that cite other papers Results What did you find? Present evidence in the order that builds the claim Past for findings; present for what displays show Chronological narration; interpretation mixed with data; every result given equal weight Discussion What does it mean, and should I believe it? Interpret, compare, qualify, draw implications Present for claims and implications; past for this study's findings Restating results; overreaching; ignoring alternatives and limitations The tense conventions in the table are worth a brief explanation, because they carry meaning that careful readers pick up. In scientific English, the present tense is used for statements the authors regard as established and general: "Mitochondria generate most of the cell's ATP." The past tense is used for what was done and found in a specific study: "Smith and colleagues reported that..." or "We observed a twofold increase...". Stating one's own finding in the present tense, as in "Protein X is required for spindle orientation", signals that the authors believe it to be generally true; stating it in the past tense, as in "loss of protein X disrupted spindle orientation in our experiments", confines it to the study. Writers can use this deliberately. The Discussion is often where a finding moves from past to present tense, as the authors argue that what they observed reflects something general. Sections as moves in an argument Another way to see the structure is as a series of moves, each of which a reader must accept before the next can work. The Introduction must persuade the reader that the question is worth asking; if it fails, the reader stops. The Methods must persuade them that the approach can answer the question; if it fails, the results are not credible. The Results must show that the answer is what the authors say it is; if they fail, the claim collapses. The Discussion must persuade the reader that the answer means what the authors say it means, and that its limitations do not undermine it. Reviewers read papers this way, often without articulating it. A review that begins "The question is interesting but..." has accepted the first move and is objecting to a later one. A review that says "It is not clear why this question matters" has rejected the first move, and no amount of experimental rigour later in the paper will rescue it. Knowing which move a criticism targets tells the author where the repair must happen, which is one of the central skills of revision discussed in Chapter 8. The implication for drafting is that each section should be written with its move in mind. An Introduction paragraph that does not advance the case for the question's importance, or narrow toward the gap, is not doing Introduction work. A Results paragraph that interprets rather than presents may be doing Discussion work in the wrong place, which confuses the reader about what is observation and what is inference. Order of writing is not order of reading Papers are read from the front, but they are rarely best written that way. A common and effective drafting sequence is to begin with the display items, write the Results around them, then the Methods, then the Discussion, then the Introduction, and finally the abstract and title. The logic is that the Results are the least negotiable part of the paper. They are what the data show. Once they are written, the writer knows exactly what the paper can claim, which determines what the Discussion should interpret and what the Introduction must set up. Writing the Introduction first risks setting up an argument that the Results do not quite deliver, a mismatch that reviewers notice quickly. The Methods can be drafted early, often in parallel with the experiments themselves, because they are largely descriptive and because details are easiest to record accurately while they are fresh. The abstract and title are written last because they compress the whole argument, and the argument is not final until the other sections are written. They are then revised repeatedly, since they are the most read parts of the paper, as Chapter 6 explains. This sequence is a default, not a rule. Some writers think best by drafting the Introduction first, as a way of clarifying what the paper is for. That can work well, provided the Introduction is treated as provisional and rewritten once the Results are fixed. What matters is that the final paper reads as a single argument, with each section setting up the next. The paragraph and the sentence as units of argument The architecture of argument does not stop at the level of sections. Paragraphs and sentences have structures of their own, and these determine whether a reader can follow the argument at all. A paragraph in a research paper should make one point, and the point should usually be stated in its first sentence, sometimes called the topic sentence. The rest of the paragraph supports, qualifies or illustrates that point. A reader who reads only the first sentence of each paragraph should come away with a coherent summary of the section. This is a demanding test, and few drafts pass it on the first attempt, but it is one of the most effective revision tools available. Mensh and Kording recommend exactly this structure, and many experienced editors apply it instinctively when they skim a manuscript. Within sentences, the most influential guidance for scientific writers comes from George Gopen and Judith Swan, whose 1990 essay "The Science of Scientific Writing", published in American Scientist, argued that readers have structural expectations about where information will appear in a sentence. Readers expect the grammatical subject to tell them whose story the sentence is about, and expect it to be followed closely by its verb, which tells them what the subject is doing. They expect the beginning of a sentence, which Gopen and Swan call the topic position, to link back to what they already know, and the end, the stress position, to deliver the new and important information. Sentences that violate these expectations, for instance by separating subject and verb with a long intervening clause, or by putting the new information at the start and familiar context at the end, force readers to work harder and increase the chance that they will misread the point. Two short examples show the difference. Consider: "The observation that, in cells lacking the kinase, which had been generated using a CRISPR approach targeting the second exon, spindle orientation was randomised, suggests a requirement for the kinase in orientation." The subject, "the observation", is separated from its verb, "suggests", by thirty words. A revision puts the old information first and the new information last: "When we deleted the kinase, spindle orientation became random. The kinase is therefore required to orient the spindle." Each sentence now begins with something the reader knows and ends with something they learn. This principle, often summarised as "old before new", also governs how sentences link to one another. Each sentence's topic position should connect to something in the previous sentence, so that the reader moves through a paragraph along a chain of connected ideas rather than jumping between unrelated statements. When readers complain that a paper is "hard to follow", the problem is frequently at this level: each sentence is grammatically correct but the connections between them are missing or misplaced. Main text and supplement A feature of modern papers that complicates their architecture is the supplementary material. Many journals, especially selective ones with tight length limits, allow or encourage authors to move extensive methods, secondary analyses, control experiments and additional data into online supplements. These can run to many times the length of the main paper. Supplements serve a real purpose. They allow the main text to stay focused on the argument while making supporting evidence available to those who want it. But they also create a risk: that essential evidence gets buried where few readers will look, or that the main text makes claims whose support is only visible in the supplement. A reader should be able to evaluate the central claim from the main text alone. Controls that are necessary to rule out the obvious alternative explanation belong in the main text, even if space is tight. Controls that address less likely alternatives, extended methodological detail, and robustness checks can go to the supplement, with clear signposting from the main text. A useful discipline is to write the main text as though the supplement did not exist, then decide what can be moved out without weakening the argument. The reverse procedure, writing everything and then deciding what to keep in the main text, often produces a main text that refers constantly to supplementary items and cannot be read on its own. Structure as a courtesy The deepest reason to take structure seriously is that it respects the reader's time and attention. Researchers read under pressure, often skimming dozens of papers to find the few that matter to their work. A paper that follows the expected architecture, and uses it to make a clear argument, lets them find what they need and evaluate it quickly. A paper that ignores the architecture, or fills it without purpose, makes them work, and many will simply stop. The chapters that follow take each part of the structure in turn and examine how to make it do its job. The Introduction comes first, because it is where the reader decides whether to continue, and because the move it must make, establishing that a question matters and has not been answered, is one of the most misunderstood in scientific writing. Chapter 3: Introductions That Earn the Gap The Introduction is where most readers decide whether a paper deserves their attention, and where most editors decide whether it deserves a reviewer's. It is also where many otherwise capable scientists write their weakest prose. The characteristic failure is the Introduction as literature review: a sequence of paragraphs summarising what others have found, loosely ordered by topic or date, ending with a sentence that says "Here, we investigated..." The reader reaches the end with a great deal of background and no clear sense of why this study was needed. A good Introduction does something more pointed. It makes a case. It persuades the reader that there is a question worth answering, that the question has not yet been adequately answered, and that this paper answers it. Everything in the Introduction should serve that case. Background that does not contribute to it, however interesting or however much effort went into assembling it, belongs elsewhere or nowhere. The three moves The most influential analysis of how research introductions work comes from the applied linguist John Swales, whose 1990 book Genre Analysis described what he called the CARS model, for Create A Research Space. Swales studied large numbers of published research introductions across disciplines and found that successful ones typically made three rhetorical moves, in order. The first move establishes a territory. The writer shows that the general research area is important, interesting or problematic, and reviews what is known about it. This is where the broad context lives: the disease burden, the ecological process, the technological need, the theoretical puzzle. It is also where the key prior work is cited, not as an exhaustive survey but as the foundation of knowledge on which the paper builds. The second move establishes a niche. The writer identifies a gap, a problem or a limitation in existing knowledge. This can take several forms. Prior work may have left a question unanswered. It may have produced contradictory results. It may rest on an assumption that has not been tested, or on a method with a known weakness. It may have been done only in a limited setting, and whether its findings extend elsewhere is unknown. Whatever the form, this move creates the space into which the paper will fit. The third move occupies the niche. The writer states what this paper does to fill the gap: the question it asks, the approach it takes, and, in many fields, the principal finding. Some writers also outline the structure of the paper here, though in the short papers typical of the natural sciences this is usually unnecessary. Swales's model is descriptive rather than prescriptive; it describes what published introductions actually do rather than what writers ought to do. But it has become one of the most widely taught frameworks in academic writing instruction precisely because it captures something real. Introductions that follow the three moves read as purposeful. Introductions that omit the second move, reviewing literature and then announcing a study without ever saying what gap it fills, read as aimless, no matter how thorough the review. The gap is an argument, not an assertion The second move is where most Introductions succeed or fail, and it deserves close attention. The weakest version of the move is the bare assertion: "However, little is known about..." or "To date, no study has examined...". These phrases appear in thousands of papers and have become so conventional that experienced readers barely register them. More importantly, they are often simply claims of novelty, and novelty on its own is not a gap worth filling. There are infinitely many things no study has examined, most of them because nobody needs to know. A gap earns its place when the writer shows two things: that the missing knowledge matters, and that it is genuinely missing. Showing that it matters means connecting it to something the reader already cares about. The unknown mechanism is the step on which a proposed therapy depends; the untested assumption underlies a widely used model; the contradictory results leave practitioners unsure which approach to take. Showing that it is missing means demonstrating familiarity with the relevant literature and explaining why existing work does not answer the question. That existing work used a different population, or relied on correlational data that cannot establish causation, or measured a proxy rather than the outcome of interest. Compare two versions of the same niche-establishing move in a hypothetical ecology paper. The first reads: "Although many studies have examined the effects of warming on plant phenology, few have considered belowground responses." This asserts a gap but does not say why it matters. The second reads: "Warming advances the timing of leaf-out in most temperate trees, and models of forest carbon uptake assume that root growth shifts in step. If roots instead respond to soil rather than air temperature, which lags by several weeks, these models will overestimate early-season nutrient uptake and may misjudge carbon gains. Yet the timing of root growth under warming has been measured in only a handful of species, with conflicting results." The second version is longer, but it gives the reader a reason to care about the gap, identifies the assumption at stake, and explains why existing evidence is insufficient. By the time the reader reaches the study's aim, they want the answer. A test that helps here is to ask what would change if the gap were filled. If the answer is "we would know something we don't currently know", the gap is not yet well argued; every gap satisfies that. If the answer is "models of forest carbon uptake could be corrected", or "clinicians would know whether to prescribe drug A or drug B to this group", or "a theoretical debate between two positions could be resolved", the gap is doing its job. Citing to build the case The Introduction is where most of a paper's citations cluster, and how they are used says a great deal about the writer's command of the field. Citations in an Introduction are evidence for the case being built. Each one should support a specific claim the writer is making: that a problem is important, that a finding has been established, that a debate exists, that a method has a limitation. Two misuses are common. The first is the citation pile, where a general statement is followed by a string of ten or fifteen references. This signals that the writer has read widely, but it does not help the reader, who cannot tell which references are foundational and which are peripheral. Where a claim is well established, two or three of the most authoritative or most recent references, or a good review, usually suffice. Where a claim is contested, it is better to cite the key papers on each side and say what they found. The second misuse is the citation as decoration, where references are attached to statements they do not directly support, or where the writer cites a paper based on its abstract or on how others have described it rather than on reading it. This is surprisingly common and occasionally embarrassing, when a cited paper turns out to say something different from, or even the opposite of, what it is cited for. Reviewers who know the literature notice. The discipline of citing only what one has read, and citing it for what it actually shows, protects both the writer's credibility and the integrity of the literature. Citing fairly also matters. An Introduction that ignores a competing group's closely related work, or that describes it uncharitably, invites a hostile review, because editors often choose reviewers from among the authors of the most closely related papers. Beyond tactics, the scientific literature depends on accurate attribution. If another group has partially addressed the question, the Introduction should say so and explain what remains open. Doing so rarely weakens the case for the new paper; more often it strengthens it, by showing that the gap is real even after the best existing work is taken into account. How long, and how broad Introductions in the natural sciences are usually short, often four to six paragraphs in a journal article, sometimes fewer. The length should be set by what the case requires, not by a desire to demonstrate knowledge of the field. A reader who is a specialist does not need the basics explained; a reader who is not a specialist does not want a comprehensive history. Both need the territory established quickly, the gap argued convincingly, and the paper's contribution stated clearly. The breadth of the opening depends on the target audience. For a specialist journal, the Introduction can begin close to the specific question, assuming shared knowledge. For a general journal read by scientists across many fields, it must begin further out, with a context that a physicist or a psychologist would recognise as important, and must explain terms and concepts that specialists take for granted. This is one of the main differences between papers written for specialist and for general journals, and one reason a paper cannot simply be sent to a different journal without rewriting its Introduction. A common mistake in papers aimed at general journals is an opening that is broad but empty: "Cancer is a leading cause of death worldwide" or "Climate change is one of the greatest challenges facing humanity". These sentences are true, but they convey nothing the reader does not know, and they do not distinguish this paper from thousands of others. A better opening is broad in reach but specific in content: it states something about the general territory that is interesting, perhaps surprising, and leads directly toward the question. The first sentence should make a reader want to read the second. Stating the question and the answer The third move, occupying the niche, is usually the last paragraph of the Introduction, and it should be explicit. The reader should finish the Introduction knowing exactly what question the paper asks and how it approaches it. Fields differ on whether the Introduction should also state the answer. In much of biology, chemistry and physics, and in most general journals, the final paragraph of the Introduction summarises the principal findings: "Here we show that..." This is not a spoiler; it is a service to the reader, who can then read the Results knowing what each piece of evidence is supposed to demonstrate. In some fields, particularly in parts of medicine and the social sciences, the convention is to state only the aims or hypotheses and leave the findings for the Results. Writers should follow the conventions of their target journal, which can be learned quickly by reading a few recent papers in it. Where a study was designed to test specific hypotheses, the Introduction should state them as they were specified before data collection, and the paper should say whether they were pre-registered. Where the work was exploratory, the Introduction should say so, framing the aims as questions rather than hypotheses. The distinction matters because it governs how readers should interpret the statistical evidence in the Results. A confirmatory test of a pre-specified hypothesis carries more evidential weight than the same result found among many exploratory analyses. Presenting exploratory findings as though they had been predicted, the HARKing mentioned in Chapter 1, misleads readers about precisely this weight. A worked sequence It may help to see how the three moves fit together in a compact Introduction. Consider a hypothetical study of a hospital hand-hygiene intervention, sketched at the level of paragraph topics rather than full prose. The first paragraph establishes the territory. Healthcare-associated infections are a major cause of preventable harm; hand hygiene by staff is among the most effective preventive measures; compliance is nonetheless often low. Two or three key references support each point, including the World Health Organization's guidelines on hand hygiene in health care. The second paragraph continues the territory by describing what is known about interventions. Multimodal programmes combining education, reminders, audit and feedback improve compliance in many settings. But compliance is usually measured by direct observation, which is known to inflate rates because staff change their behaviour when watched, the Hawthorne effect. The third paragraph establishes the niche. Because most evaluations rely on direct observation, it is unclear how much of the reported improvement reflects real change in behaviour rather than a change in behaviour while being observed. Electronic monitoring systems that record dispenser use continuously offer an alternative measure, but few studies have compared the two measures before and after an intervention in the same wards, and those that have were small. If observed improvements do not reflect continuous behaviour, hospitals may be overestimating the effectiveness of their programmes and misallocating resources. The fourth paragraph occupies the niche. Here the authors report a study in which both measures were collected in twelve wards over two years spanning the introduction of a multimodal programme. They state the principal finding, perhaps that observed compliance rose substantially while electronically measured dispenser use rose much less, and its implication, that audit-based evaluations of hand-hygiene programmes should be interpreted with caution. Four paragraphs, each with a clear job, leading the reader from a problem everyone recognises to a specific question and answer. The sequence is simple, but it is remarkable how many published Introductions lack one or more of its elements. Revising the Introduction Because the Introduction is often drafted before the rest of the paper is complete, or drafted early and left alone, it frequently needs substantial revision once the Results and Discussion are finished. Several checks help. First, read the Introduction and then the Discussion in immediate succession. Does the Discussion answer the question the Introduction asks? Does it return to the broad context the Introduction opened with? If the Discussion has drifted to a different question, one of the two sections must change. Second, apply the topic-sentence test from Chapter 2. Read only the first sentence of each paragraph. Do they, in sequence, make the case: territory, gap, contribution? If a paragraph's first sentence does not advance the case, either the paragraph needs restructuring or it does not belong. Third, look at every citation and ask what claim it supports. Remove citations that support nothing specific and reduce citation piles to the most useful references. Fourth, count the paragraphs before the gap is stated. If the reader must work through four paragraphs of background before learning what is missing, the Introduction is probably front-loaded. Often the fix is to move some background into the Discussion, where it can be used to interpret the findings, and to reach the gap sooner. Finally, ask a colleague outside the immediate subfield to read the Introduction alone and then say, in their own words, what question the paper asks and why it matters. If they cannot, the Introduction has not yet done its job. This check takes a colleague ten minutes and is among the most valuable ten minutes of feedback a paper can receive. The Introduction makes a promise: here is an important question that has not been answered, and this paper will answer it. The rest of the paper must keep that promise. The next chapter turns to the sections where the promise is kept or broken: the Methods and Results, which together form the evidence chain on which every claim rests. Hashtags: #WritingTheScientificPaper #ScientificWriting #ResearchManuscripts #ScientificArgument #CentralClaim #IMRaDStructure #ResearchArticleStructure #LaboratoryNotebook #EvidenceChain #ResearchReporting #ScientificIntroductions #CARSModel #KnowledgeGap #MethodsReporting #ResultsWriting #DiscussionWriting #ScientificAbstracts #ResearchTitles #JournalSelection #HighImpactJournals #PeerReview #ReviewerResponse #PublicationStrategy #ResearchIntegrity #FutureOfScientificPublishing

Download the Book (PDF): Introduction Somewhere in a mobile examination trailer parked outside a county fairground, a technician draws blood from a volunteer, labels the tube, and logs the time. Somewhere in a Swedish tax office, a clerk updates an address. In a warehouse in Stockport, a robot retrieves one of several million frozen aliquots so that a machine in another building can count five thousand proteins in it. In 1950, an enumerator in rural Georgia wrote down the occupation of a man he would never see again, and in April 2022, seventy-two years later, that page became a row in a public file. None of these people was working for you. Every one of them made decisions that will shape the numbers you report. That is the central fact of secondary data analysis, and it is easy to forget once the files are on your screen. The UK Biobank, the National Health and Nutrition Examination Survey (NHANES), the Integrated Public Use Microdata Series (IPUMS), the Nordic population registries, and their many cousins are extraordinary public goods. They let a doctoral student test a hypothesis on half a million people for a few thousand pounds, let a historian follow individuals across four decades of censuses, and let a nutrition scientist estimate the prevalence of iron deficiency in American toddlers with a precision that no single laboratory could ever afford. They have also produced a large and growing literature of confident, precisely estimated, wrong answers. The two facts have the same cause. These datasets are cheap to analyse because someone else already bore the cost of designing and collecting them, and they are easy to misanalyse because the analyst did not make, and often does not understand, the design decisions baked into every row. The controlling idea This book argues one thing: a secondary dataset is the output of a data-generating process that someone else designed, and valid analysis begins by reconstructing that process and building your analysis to respect it. Sampling probabilities, oversampling of particular groups, the reasons people agreed or refused to take part, the way records were stitched together across institutions, the legal terms under which the data were released, and the variables that were never collected at all are not background details. They are part of the data. An analysis that ignores them is not a simplification of the correct analysis. It is a different analysis, answering a question nobody asked about a population that does not exist. Put that way, the idea sounds obvious. In practice it is violated constantly, for understandable reasons. The files arrive looking like any other rectangular dataset. Standard statistical software will happily fit an unweighted logistic regression to NHANES and return a tidy table of odds ratios and standard errors, without any warning that the standard errors are too small, the intercept describes nobody, and the sample over-represents the groups the survey designers deliberately oversampled. A Cox model fitted to the UK Biobank will return hazard ratios with confidence intervals so narrow they look like physical constants, and nothing in the output will mention that only about one in eighteen invited people chose to take part, or that the ones who did smoke less, drink less, and die less than the population they came from. Software reports what you asked it to compute. It does not report what you should have asked. Three families of data The book deals with three broad families of secondary data, and it is worth naming them at the start, because each carries a characteristic design problem. The first is the probability survey, of which NHANES is the leading health example and the American Community Survey and the Current Population Survey (both accessible through IPUMS) are the leading social examples. These are built from explicit sampling designs, with stratification, clustering and unequal selection probabilities, and they come with weights and design variables that encode that design. Their characteristic problem is that the analyst must carry the design into the analysis, and most introductory training does not teach how. The second is the register or census: the complete, or nearly complete, enumeration of a population for administrative or constitutional purposes. The Danish Civil Registration System, which has assigned a unique personal identification number to every resident since 1968, and the United States decennial censuses, now released in full-count form through 1950, are archetypes. Their characteristic problem is not sampling but meaning. The variables were recorded to administer benefits, collect taxes or apportion seats in a legislature, and they measure what those purposes needed, not what a researcher needs. Linking them across time and across institutions introduces a second problem: every link is a claim that two records belong to the same person, and some of those claims are false. The third is the volunteer cohort or biobank. The UK Biobank recruited around 500,000 people aged 40 to 69 between 2006 and 2010, measured them extensively, stored their biological samples, and has since linked them to hospital, cancer, death and, for a large subset, primary care records, while adding genotyping, whole-genome sequencing, imaging on a subsample and, from 2026, large-scale proteomics. Its characteristic problem is selection. Participants were invited, not sampled with known probabilities, and the few who accepted differ systematically from those who did not. That matters a great deal for some questions and very little for others, and telling which is which is one of the more demanding skills in modern epidemiology. Cutting across all three families is the problem that no amount of sample size can fix: confounding by variables the dataset never measured. Secondary data are, by definition, collected for someone else's purposes. The confounder you most need is often the one nobody thought to record. How the book is organised Chapter 1 establishes the habit of mind the rest of the book depends on: reading a dataset as the product of decisions, and reading its documentation before its data. It also compares the major resources side by side. Chapter 2 deals with access and governance, which is not paperwork to be delegated but a set of constraints that determine what analyses are possible, where they can be run, and what can be published. Chapters 3 and 4 cover complex survey data: first the mechanics of weights, strata, clusters and variance estimation, using NHANES as the running example, and then the harder questions of when and how to weight in regression models, with missing data and in pooled analyses. Chapter 5 turns to volunteer cohorts and the problem of selection, including the recent work on reweighting the UK Biobank. Chapter 6 covers record linkage, from deterministic rules through probabilistic matching to the historical census linking that has transformed economic history, and the biases that linkage error introduces. Chapters 7 and 8 address unmeasured confounding: first the study designs that can blunt it, and then the quantitative sensitivity analyses that tell you how much an unmeasured confounder would have to do to overturn your result. The Conclusion draws out what follows for practice. Who this is for, and what it assumes The intended reader is someone who has fitted a regression model and read an epidemiology or social science paper critically, and who now wants to work with large public datasets without falling into the traps that catch capable people. That includes graduate students, clinicians moving into research, analysts in government and industry, and senior researchers trained before these resources existed. The book assumes familiarity with ordinary regression and basic probability, but no survey sampling theory, no causal inference formalism, and no particular software. Where software matters, I name the relevant tools: the survey package in R, the svy prefix in Stata, PROC SURVEYREG and its relatives in SAS, and the cloud platforms on which some of these datasets now must be analysed. A note on currency. The access arrangements for these datasets change often, and several changed materially between 2023 and 2026. The UK Biobank no longer routinely allows bulk downloads; analysis happens on its cloud Research Analysis Platform. NHANES released its first post-pandemic cycle, covering August 2021 to August 2023, with a redesigned sample and much lower response rates, and the programme itself passed through a period of acute institutional uncertainty in late 2025. Europe adopted a regulation creating a European Health Data Space whose secondary-use provisions will begin to apply in 2029. The details given here were checked against the data custodians' own documentation in 2026. They will drift, and the book tries to teach the questions to ask rather than only the answers that are true today. A word on what "public" means The word "public" in public data is doing a lot of work, and it means different things for different resources. NHANES public-use files can be downloaded by anyone, anonymously, without an application. IPUMS extracts require free registration and agreement to terms of use, and the most detailed international and full-count files carry additional restrictions. The UK Biobank is open to any bona fide researcher for health-related research in the public interest, but requires an application, a fee, and analysis on a controlled platform. The Nordic registries are open, in the sense that researchers from anywhere can in principle obtain them, but only through national authorities, often only for analysis on secure servers inside the country, and usually only after months of review. These differences are not incidental. They reflect different judgments about the balance between the value of research and the risk to the people whose lives the data describe, and an analyst who understands those judgments will design better studies and have fewer of them rejected. The people in these datasets did not consent to your specific study. In most cases they consented, or were compelled by law, to a broad purpose, and trusted the custodian to police the boundaries. The obligations that follow from that trust are part of the method, not an ethical appendix to it. A good secondary analysis is one that the participants, if they could read it, would recognise as a fair use of what they gave. Chapter 1. Reading the Data Before the Data The most consequential hour in a secondary analysis is usually spent before any data are loaded. It is spent reading: the design documentation, the codebook, the analytic guidelines, the release notes, and the list of known errors. Analysts who skip this hour tend to spend the following six months discovering, one referee report at a time, what the documentation would have told them. This chapter is about how to read a secondary dataset as an artefact with a history, and about the handful of questions whose answers determine almost everything that follows. The data-generating process as the unit of understanding Every dataset is the endpoint of a chain of events. For a survey, the chain runs roughly as follows: a target population is defined; a sampling frame is chosen that approximates it; the frame is divided into strata and clusters; units are selected with known but unequal probabilities; some selected units cannot be found, some refuse, some complete only part of the protocol; answers and measurements are recorded, cleaned, edited, top-coded, imputed and suppressed; weights are computed to reverse the selection and adjust for nonresponse; and a public file is released with some identifying detail removed. For a registry, the chain runs through the administrative purpose that created the record, the rules that determined who must be recorded, the coding systems in force at each date, the incentives of the people doing the recording, and the processes by which records were corrected or linked. For a biobank, it runs through the invitation strategy, the decision to volunteer, the assessment protocol, the choice of which samples to assay and in what order, and the linkage of external records whose own chains are just as long. Each link in the chain can distort the relationship between what is in the file and what is true of the population. The analyst's first job is to know which links matter for the question at hand. A useful discipline is to write down, before looking at any results, the path by which a single individual in the target population ends up as a row in your analysis dataset, with a value for every variable you intend to use. Every step on that path at which the probability of reaching the next step could depend on the exposure, the outcome, or both, is a place where bias can enter. Some of those steps will be handled by the survey weights. Some will be handled by design choices you make. Some will be handled by nothing at all, and those are the ones to state honestly as limitations. Consider a concrete case. Suppose you want to estimate the association between serum vitamin D concentration and depressive symptoms among American adults, using NHANES. A person enters your analysis only if their county was selected as a primary sampling unit, their household was selected within the county, they were selected within the household, they completed the home interview, they attended the mobile examination centre, they consented to and completed the blood draw, their sample was successfully assayed for 25-hydroxyvitamin D, and they completed the Patient Health Questionnaire (PHQ-9) module in the private interview. The first three steps are controlled by the design, with known probabilities that the weights reverse. The fourth and fifth are affected by nonresponse, which the weights attempt to adjust for using demographic variables. The sixth, seventh and eighth are item-level losses that the standard weights may not address at all. If people with depression are less likely to attend the examination centre, or less likely to complete the blood draw, the weights will not know. That does not make the study impossible. It makes it a study whose limitations can be precisely described. What to read, and in what order The documentation for the major resources is extensive and uneven, and the order in which you read it matters. A sensible sequence is: the design overview, the analytic guidelines, the documentation for each specific file you will use, the release notes and errata, and finally a selection of well-cited papers that have used the same variables. For NHANES, the design overview and analytic guidelines are published for each cycle by the National Center for Health Statistics (NCHS). The data files are organised by component (demographics, dietary, examination, laboratory, questionnaire) and by cycle, with a letter suffix marking the cycle: files from 2017 to 2018 carry the suffix J, the combined pre-pandemic files covering 2017 to March 2020 carry the prefix P, and the August 2021 to August 2023 files carry the suffix L. Each file has its own documentation page listing the eligible sample, the protocol, the data processing and editing, and, crucially, which weight to use. That last point is not a formality. The laboratory components are often measured on a random subsample of examinees, with a subsample weight that must be used in place of the examination weight, and a fasting subsample carries its own weight again. For the UK Biobank, the central reference is the Data Showcase, which catalogues every data field by a numeric field identifier, with the number of participants who have data, the instance (the assessment visit, where 0 is baseline, 1 is the first repeat assessment in 2012 to 2013, 2 is the imaging visit that began in 2014, and 3 is the repeat imaging visit that began in 2019), and the array index where a question allowed multiple answers. The Showcase also hosts resource documents describing each assay, each linked dataset and each derived outcome. Many analysts go straight to the phenotype they want and never read the resource document explaining how it was derived. That is how a study ends up treating the absence of a hospital diagnosis code as evidence that a condition is absent, in participants whose primary care records were never linked. For IPUMS, the key documents are the variable descriptions and the comparability discussions attached to each harmonised variable. IPUMS exists precisely to harmonise: to recode occupation, industry, education, relationship to household head and hundreds of other variables so that they mean approximately the same thing across censuses a century apart or across countries with different statistical traditions. The word "approximately" is doing work. The comparability notes explain where the harmonisation is exact and where it is a best effort, and they record changes in question wording, universe (who was asked the question) and coding that can create artificial trends. The universe statement for each variable is particularly important: many census questions were asked only of people above a certain age, or only of household heads, or only in a sample of households, and a variable coded as "not in universe" is not the same as a zero. For national registries, the key documents are usually validation studies published by the custodians or by academic users. The Danish registries, for example, have an unusually well-developed literature assessing the positive predictive value of specific diagnosis codes against medical record review. Reading those studies tells you which codes can be trusted to identify a condition and which cannot, and whether a code's meaning changed when the country moved from the eighth to the tenth revision of the International Classification of Diseases in 1994. Comparing the major resources The resources this book returns to most often differ along several dimensions at once: how participants entered the data, how many there are, who can get access and how, and what kind of error most threatens a naive analysis. Table 1 sets out those differences for five representative resources. The characteristic hazard in the final column is not the only problem each resource has, but it is the one that most often invalidates published work. Table 1. Five major secondary data resources compared (sources: custodians' documentation, 2025–2026). Resource How people entered Scale Access route Characteristic hazard NHANES (US) Multistage probability sample, two-year cycles About 5,000 examined per year pre-2020; 8,860 examined in 2021–2023 Public-use download; restricted files via NCHS Research Data Center Ignoring weights, strata and clusters IPUMS USA and CPS Census and survey samples, plus full-count censuses 1850–1950 Samples of 1–5% of population; full counts of tens of millions Free registration and terms of use Harmonisation artefacts; "not in universe" coding UK Biobank Invited volunteers aged 40–69, recruited 2006–2010 About 500,000 Application, tiered fee, cloud platform analysis Healthy volunteer selection; linkage completeness Nordic registries Legal obligation to register all residents Whole national populations National authority approval; remote secure servers Administrative coding changes; unmeasured lifestyle confounders Linked administrative data (e.g., hospital episodes) Contact with a service Everyone who used the service Custodian approval, often via trusted research environments Selection into care; linkage error Two features of the table deserve comment. First, scale and representativeness run in opposite directions for the health resources. The survey with the explicit probability design, NHANES, is the smallest. The biobank with the largest sample, the UK Biobank, has the least representative recruitment. The registries escape the trade-off by covering everyone, but pay for it in the thinness of what they record: a Danish registry knows your prescriptions and hospital diagnoses in detail but does not know whether you smoke. Second, the access routes are becoming more restrictive over time at the top end and more open at the bottom. Public-use files remain freely downloadable, but the richest data increasingly live in trusted research environments where the analyst brings code to the data rather than data to the code. Chapter 2 examines what that shift means in practice. Matching the question to the data Before choosing variables and methods, it is worth being explicit about what kind of question you are asking, because the three main kinds of question make different demands on the data-generating process. A descriptive question asks what is true of a defined population: the prevalence of diagnosed and undiagnosed diabetes among American adults in 2021 to 2023, the share of married women in the labour force in 1950, the age-standardised incidence of hip fracture in Denmark. Descriptive questions demand representativeness. The estimate is only meaningful for the population the sample represents, and the whole apparatus of survey weights exists to make that link. For descriptive questions, a volunteer cohort such as the UK Biobank is a poor instrument, and its own investigators have always said so. The prevalence of smoking in the UK Biobank tells you about UK Biobank participants. A predictive question asks how well some information forecasts some outcome: whether a polygenic score improves the prediction of coronary disease beyond conventional risk factors, whether a set of blood proteins predicts ten-year mortality. Prediction does not require causal interpretation of the coefficients, but it does require that the relationship between predictors and outcome in the development data resembles the relationship in the population where the model will be used. Selection matters here in a subtler way. A risk model developed in a population healthier than the target will usually be miscalibrated, predicting too few events, even if its discrimination is fine. A causal question asks what would happen to an outcome if an exposure were changed: whether statins reduce dementia risk, whether a minimum-wage rise reduces employment, whether moving out of a high-poverty neighbourhood improves children's adult earnings. Causal questions demand that the comparison between exposed and unexposed be free of confounding, selection bias and measurement error sufficient to overturn the conclusion. Representativeness is not required for internal validity, though it matters for transporting the result to other populations. Chapters 5, 7 and 8 are mostly about causal questions, because that is where secondary data are most often used and most often misused. Many published papers slide between these three kinds of question without noticing. A paper reports an association in the UK Biobank, interprets it causally in the discussion, and then recommends a population-wide policy on the strength of it, implicitly claiming descriptive validity for the whole country. Each of those three moves requires a separate justification. Naming which question you are asking, in a single sentence at the top of your analysis plan, prevents a surprising amount of confusion later. A pre-analysis protocol With the documentation read and the question named, the final step before analysis is to write a short protocol. For secondary data, the protocol has an additional job beyond its usual role in preventing selective reporting: it forces you to make design decisions explicitly, before you see which ones produce the most attractive results. The protocol for a secondary analysis should specify at minimum the following. The target population and how it maps onto the dataset's sampled or recruited population. The eligibility criteria, applied in order, with the expected number excluded at each step, so that the flow of participants can be reported and checked. The exposure, outcome and covariate definitions, down to the specific field identifiers, variable names or diagnosis codes. The time structure: when eligibility is assessed, when exposure is measured, when follow-up begins and ends, and what happens at the boundaries. The weights, strata and cluster variables to be used, and the variance estimation method. The handling of missing data. The primary analysis and the sensitivity analyses, including those addressing unmeasured confounding. Several registries and repositories now accept protocols for observational studies. The Open Science Framework hosts many; the UK Biobank requires a description of the research as part of the access application, though that description is not a protocol in the full sense. Registration matters more for secondary data than for primary data, not less. When the data already exist, the only barrier between an analyst and a hundred different specifications is self-discipline. With 500,000 participants and thousands of variables, some specification will produce a small p-value for almost any hypothesis. The protocol is the evidence that you chose yours first. A final habit is worth building at this stage. Before running the analysis you care about, reproduce a published number from the documentation or from a well-known paper using the same data. NCHS publishes prevalence estimates with standard errors in its Data Briefs; if your code cannot reproduce the published prevalence of obesity for the cycle you are using, to the decimal place and with the same standard error, something is wrong with your weights, your design variables, or your understanding of the eligible population. IPUMS publishes population totals that a correctly weighted extract should match. For the UK Biobank, the cohort profile papers report baseline characteristics that a correctly specified extract should reproduce. The replication step takes an hour and catches errors that would otherwise surface only after publication, if at all. Data that move under your feet A last feature of secondary data that the documentation reveals, and that newcomers rarely anticipate, is that the data change. A published NHANES file is occasionally re-released with corrections, and the documentation page records the date and nature of each revision. A laboratory variable may be recalibrated when an assay method changes, and NCHS may publish a crosswalk equation so that values from different cycles can be compared; an analyst who pools cycles without applying the crosswalk will find a spurious trend. IPUMS releases new versions of its databases, with new samples, corrected codes and revised harmonisation, and asks users to cite the specific version. An extract downloaded in one year may not match an extract of the same variables downloaded two years later. The UK Biobank changes in a further way. Participants may withdraw at any time, and when they do, the resource periodically notifies approved researchers, who are required to remove the withdrawn participants' data from their analyses. Hospital, death and cancer registry linkages are refreshed at intervals, extending follow-up and sometimes revising earlier records as late-registered events arrive. The censoring date for each linked source differs by country of the United Kingdom, because the records come from separate English, Scottish and Welsh systems with separate update schedules. A study that sets a single administrative censoring date for all participants, later than the date to which one nation's records are complete, will undercount events in that nation and bias any comparison that correlates with where people live. The practical consequences are simple to state. Record the date and version of every extract. Keep the raw extract untouched and do all derivation in scripts, so that a refreshed extract can be pushed through the same pipeline. Check the release notes before each new analysis. And when you report results, state which release, which version and which follow-up end dates you used, so that someone trying to reproduce your numbers two years later knows what to ask for. These are not bureaucratic niceties. In a field where the same dataset is analysed by thousands of groups, they are what make disagreements between groups resolvable, rather than a matter of whose extract happened to be older. Chapter 2. Access, Agreements and the Rules of Use Researchers new to secondary data often treat access as an administrative hurdle to be cleared as quickly as possible so that the real work can begin. That view is expensive. The terms under which a dataset is released determine which variables you can see, at what level of geographic and temporal detail, whether you can link them to anything else, where your computation must run, what you are allowed to publish, and how long you may keep what you derived. Those are design constraints. A study planned without them in view will often be redesigned, sometimes after months of work, when the access terms turn out to forbid the linkage it depends on or the output rules suppress the subgroup it was built to study. Why the rules exist The people in secondary datasets are real, and most of them are identifiable in principle. That is true even of files stripped of names and addresses. A combination of a few ordinary attributes, such as exact date of birth, sex and a small geographic area, is unique for most of the population of a country, and sensitive attributes in the same row, such as a psychiatric diagnosis, an HIV test result or an income, then become attached to a person. Genomic data are identifying by nature. Custodians therefore face a trade-off between the scientific value of detail and the risk that detail enables re-identification, and different custodians strike it differently. The most widely used framework for thinking about that trade-off is the Five Safes, developed at the UK Office for National Statistics and set out by Tanvi Desai, Felix Ritchie and Richard Welpton. It asks whether the project is appropriate (safe projects), whether the researchers can be trusted to use the data properly (safe people), whether the environment prevents unauthorised use (safe settings), whether the data themselves carry a disclosure risk that is appropriate to the other safeguards (safe data), and whether the statistical results leaving the environment are non-disclosive (safe outputs). The value of the framework is that it treats the dimensions as substitutes. A dataset with high intrinsic disclosure risk can still be made available if the setting is locked down and the outputs are checked. A dataset released for anyone to download must be heavily de-identified, because none of the other safeguards applies. Almost every access regime a researcher will meet can be read as a particular setting of the five dials. NHANES public-use files turn "safe data" up to maximum, by removing geography below the national level, top-coding extreme values, and masking the design variables, and turn the other four down to nearly nothing: anyone may download them, anywhere, for any purpose consistent with a short use statement. The NCHS Research Data Center turns the dials the other way: the restricted variables, including county identifiers and exact dates, are available only to approved projects, analysed in a secure environment, with every output reviewed before release. The principal regimes in 2026 Table 2 summarises the arrangements for the resources this book discusses, as documented by the custodians in 2025 and 2026. The detail changes frequently, and the custodians' own pages are the authority, but the structure of each regime has been stable enough to plan around. Table 2. Access arrangements for major secondary data resources (sources: custodian websites, checked 2026). Resource Who may apply Cost to researcher Where analysis happens Notable conditions NHANES public-use Anyone None Anywhere No attempt to identify participants NCHS restricted data Approved projects Fees for Research Data Center use Secure physical or virtual enclave Output review before release IPUMS USA, CPS, International Registered users; International requires application None Anywhere No redistribution; citation required UK Biobank Bona fide researchers, health-related public-interest research Tiered access fee plus cloud compute and storage Research Analysis Platform (cloud) Downloads exceptional; return of derived data Nordic registries Researchers affiliated with approved institutions Agency processing fees National remote-access servers Ethics and data-protection approval; aggregate outputs only The UK Biobank deserves the fullest treatment, because its regime changed substantially in the early 2020s and the change affects how studies are designed. Access is open to researchers at academic, charitable, government and commercial institutions, for health-related research in the public interest, and applications are reviewed against that criterion rather than for scientific merit. The fee is set by data tier. According to the resource's published fee schedule in 2026, three years of access to Tier 1 (questionnaire, physical measures, health outcomes and linked records) costs £3,000; Tier 2, which adds biochemistry, genotypes, proteomics and other assays, costs £6,000; and Tier 3, which adds imaging, whole-genome and exome sequencing and other large-scale data, costs £9,000, with lower annual fees for extensions and an extra charge for each additional collaborating institution. Students and researchers in low- and middle-income countries pay a much reduced fee of £500 for three years. The larger change is where the analysis happens. Since the introduction of the UK Biobank Research Analysis Platform, built on the DNAnexus cloud, approved researchers analyse the data inside the platform rather than downloading it, and the resource considers applications to download data only in exceptional circumstances. The platform charges for computation, for data storage and for data egress (moving results out), with an initial credit for new users and, for a period, larger credits for new projects. The effect on study design is concrete. An analysis that would have been free on a university server now has a marginal cost for every hour of computation. Genome-wide analyses of whole-genome sequence data on hundreds of thousands of participants can cost thousands of pounds in compute alone if written carelessly and far less if written well. Budgeting for computation belongs in the grant application, and efficient code, such as running association tests on pre-filtered variant sets and using the platform's optimised tools rather than exporting data to generic software, has become a matter of money rather than taste. The UK Biobank also requires approved researchers to return the results of their research, including derived variables, to the resource, so that later researchers can use them. This makes the resource cumulatively richer, and it means that when you derive a complex phenotype, you are writing documentation for someone else, whether you intend to or not. NHANES remains the most open major health survey in the world, but its restricted data are more tightly controlled. Researchers who need geographic identifiers, exact examination dates, or linked data beyond what the public-use files include apply through the federal Standard Application Process, and analyse the data in an NCHS or Federal Statistical Research Data Center, physically or through a virtual desktop. The linked mortality files are an important middle case. NCHS links NHANES participants to the National Death Index and releases public-use versions, most recently with follow-up through 31 December 2019, with certain details perturbed to protect confidentiality; the restricted versions contain the unperturbed data. For many mortality analyses the public-use files are adequate, and NCHS has published comparisons showing close agreement with the restricted versions for common analyses. IPUMS is free, but not unconditional. Users register and agree to terms that, for IPUMS USA, prohibit redistribution of the data except for subsets needed to meet journal replication requirements, prohibit redistribution of the full-count census files entirely, and require citation of the data in a specified form that includes the version. IPUMS International, which distributes census microdata from national statistical offices around the world under agreements with each, adds an application and restricts use to the approved research and classroom purposes described in it. These are the terms on which national statistical offices were persuaded to release their data, and a breach by one user endangers access for everyone. The Nordic countries illustrate a model in which whole-population data are available in principle to researchers from anywhere, but only through a regulated process. In Denmark, researchers work on de-identified registry data through the remote-access servers of Statistics Denmark or the Danish Health Data Authority, with only aggregate results leaving the environment. In Finland, the Finnish Social and Health Data Permit Authority, Findata, created under a 2019 act on the secondary use of health and social data, issues permits for combining data from multiple custodians and provides a secure operating environment. Sweden requires ethical approval from the Swedish Ethical Review Authority before the registry holders, such as the National Board of Health and Welfare and Statistics Sweden, will link and release data. Processing times of six months to more than a year are common across the region and should be built into project timelines. The "safe people" dial has its own machinery. In the United Kingdom, access to de-identified government microdata under the research provisions of the Digital Economy Act 2017 requires researchers to be accredited by the UK Statistics Authority, which in practice means completing a short training course on disclosure control and the legal framework and passing an assessment. NCHS and the Federal Statistical Research Data Centers require background checks and sworn confidentiality obligations. These requirements feel like friction to an individual researcher, but they are what allow custodians to release more detailed data than they otherwise could, and a research group that keeps several accredited analysts on its staff can move much faster than one that must train someone for every new project. What a data use agreement actually says When access requires a formal agreement, the agreement usually contains a predictable set of clauses, and each one has design implications. A purpose limitation clause restricts use to the project described in the application. A change of research question may require an amendment. That is a reason to describe the project broadly enough to accommodate sensible sensitivity analyses and follow-on questions, but not so broadly that the reviewers cannot tell what you intend. A no re-identification and no linkage clause forbids attempts to identify individuals and forbids linking the data to other sources without permission. The second part catches many researchers by surprise. Merging a restricted dataset with area-level deprivation indices, air pollution estimates or hospital characteristics at a geographic level finer than the release allows can breach the agreement, even though no individual is identified, because it increases disclosure risk. A security clause specifies where data may be stored, who may see them, and how they must be protected. In trusted research environments, the environment enforces this. In agreements that allow data to be held locally, the clause usually requires encryption, restricted access and a named data custodian at the researcher's institution. An output control clause specifies the rules that results must meet before they leave a secure environment. The rules usually include minimum cell sizes, prohibitions on reporting minimum and maximum values, and restrictions on releasing residuals or other individual-level quantities. The minimum cell size varies: the US Centers for Medicare and Medicaid Services, for example, prohibits publishing any cell of fewer than eleven beneficiaries, and many European trusted research environments apply thresholds of five or ten with rounding. Output rules affect design most for rare exposures, rare outcomes and small subgroups. A study of a rare cancer in a specific age and ethnic group may be analysable inside the environment and yet impossible to report in the detail readers need. Checking the output rules before designing the tables avoids that outcome. A publication clause may require notification of publications, submission of manuscripts for review before submission, or specific acknowledgment text. Review clauses are usually for disclosure checking, not scientific censorship, but they add time. A retention and destruction clause specifies what happens to the data at the end of the agreement. Researchers who want to preserve their analytic datasets for replication must negotiate this explicitly, because the default is usually destruction. Ethics review and the law Secondary analysis of de-identified public data is often outside the scope of research ethics review, or exempt from it, but the determination belongs to the ethics committee or institutional review board, not to the researcher. In the United States, analyses of publicly available, de-identified data such as NHANES public-use files generally do not meet the regulatory definition of human subjects research under the revised Common Rule, and most institutions will issue a brief determination to that effect. Restricted data usually require review, and data covered by the HIPAA Privacy Rule may be released as a limited data set, with dates and some geographic detail retained, only under a data use agreement meeting the rule's requirements. In the United Kingdom and the European Union, the General Data Protection Regulation (and its UK counterpart) treats pseudonymised data as personal data, because the custodian can re-identify them. Research processing of health and genetic data requires both a lawful basis and a condition for processing special category data; for research, the relevant condition permits processing for scientific research purposes subject to safeguards, including data minimisation and, where possible, pseudonymisation. In practice, data custodians such as the UK Biobank, which relies on participants' broad consent together with these provisions, handle most of this analysis, but the researcher's institution remains responsible for its own processing and may need its own data protection impact assessment. The European Health Data Space regulation, which entered into force on 26 March 2025, will reshape access to health data across the European Union. Its secondary-use provisions, which create national health data access bodies empowered to issue data permits for research, policy and innovation and to make data available in secure processing environments, are scheduled to apply from March 2029, with some categories, including genetic data, following later. The model is close to the Finnish one: a single permit authority, secure processing environments, and outputs restricted to anonymous statistics. Researchers planning multinational European studies for the end of the decade should expect the permit process to become the standard route. Designing with the constraints The practical conclusion is that access terms should be read at the protocol stage, not after approval, and the analysis designed around them. If outputs must be aggregated, decide in advance which aggregates answer the question, and check that they will pass the cell-size rules. If the analysis must run in a cloud environment with metered computation, write and test the code on a small synthetic or public dataset first, and bring only debugged code into the paid environment. If linkage to external data is essential, name the external data in the application and obtain permission for the linkage explicitly. If the data will need to be retained for replication, negotiate retention at the start. It is also worth thinking about what a secure environment does to reproducibility. Code written inside a trusted research environment can usually be exported, but data cannot, and some environments restrict the software that can be installed. Openly publishing the code, together with the precise specification of the extract and the software versions, is the best available substitute for publishing the data. The OpenSAFELY platform, developed during the COVID-19 pandemic to analyse primary care records for tens of millions of patients in England, made that principle a requirement: all code executed against the real data is published, and every execution is logged publicly. It demonstrated that full transparency of method is compatible with complete confidentiality of data, and it is a model that other environments are gradually adopting. Finally, access regimes change, and the direction of change matters for how long a study takes. The broad trend since about 2015 has been away from releasing copies of sensitive data and towards bringing analysts into controlled environments. That trend is unlikely to reverse. It is safer for participants, it gives custodians visibility of how their data are used, and it lets them host datasets, such as whole-genome sequences on half a million people, that are too large to move. The cost falls on analysts, who must learn to work in environments that are less flexible than their own machines, and on the pace of research, which slows at every step of output review. Planning for that pace is part of the design. Chapter 3. Weights, Strata and Clusters: The Mechanics of Survey Data A national probability survey is a deliberately distorted picture of a population, together with the instructions for undoing the distortion. The distortion is intentional. Survey designers oversample small groups so that estimates for them are precise, cluster interviews geographically so that fieldwork is affordable, and stratify so that important subdivisions of the population are guaranteed representation. The instructions come in the form of weights, stratum identifiers and cluster identifiers. An analyst who uses the data without the instructions gets the distorted picture. This chapter explains what each part of the design does, how to carry it into an analysis, and where the common mistakes lie, using NHANES as the running example because its design is complex, well documented and representative of the major health examination surveys. Why the sample does not look like the population The NHANES sample is drawn in four stages: primary sampling units, which are counties or groups of contiguous counties; segments within counties, typically groups of census blocks; households within segments; and individuals within households. At each stage, units are selected with probabilities that the design controls, and those probabilities are deliberately unequal. For most of the continuous NHANES era, from 1999 until data collection was suspended in March 2020, the survey oversampled specific groups. In different periods these included Mexican American and later all Hispanic persons, non-Hispanic Black persons, non-Hispanic Asian persons from 2011, low-income non-Hispanic White and other persons, and older adults. Oversampling a group means selecting its members with higher probability than their share of the population. The result is that an unweighted NHANES sample contains far more Hispanic, Black and Asian participants, relative to their population share, than a simple random sample would. That is the point: it allows reliable estimates for those groups. It also means that any unweighted national estimate, such as the prevalence of hypertension among all American adults, is pulled toward the values in the oversampled groups. The August 2021 to August 2023 cycle changed the design in response to the pandemic. To reduce the amount of in-person contact needed during household screening, NCHS eliminated oversampling by race, Hispanic origin and income. Instead, within each selected household, all members aged 0 to 19 and all members aged 60 and over were selected, along with one or two randomly chosen adults aged 20 to 59, depending on household size. The sample remains unequal-probability, now by age and household composition rather than by race and income, and the weights still matter. The practical consequence NCHS draws in its analytic guidance is that estimates for some race and Hispanic origin subgroups in 2021 to 2023 are less precise than in earlier cycles, because those groups are no longer boosted in the sample. A small worked example shows how large the effect of oversampling can be. Suppose a population contains 90,000 people in group A and 10,000 in group B, and a condition affects 10 percent of group A and 30 percent of group B. The true population prevalence is (9,000 + 3,000) / 100,000, or 12 percent. A survey samples 900 people from group A (1 in 100) and 900 from group B (9 in 100). Unweighted, the sample contains 90 cases in A and 270 in B, for an apparent prevalence of 360 / 1,800, or 20 percent. Each respondent in A represents 100 people and each in B represents about 11.1. Weighting the cases by those factors gives 9,000 + 3,000 cases in a weighted total of 100,000 people, recovering the 12 percent. Real surveys have hundreds of distinct weight values rather than two, but the logic is identical. How weights are built A final survey weight is typically the product of three components, and knowing them helps in understanding what the weight can and cannot fix. The base weight is the inverse of the probability of selection. A person selected with probability 1 in 20,000 receives a base weight of 20,000, meaning that the person stands in for 20,000 people in the population. Base weights undo the deliberate distortions of the design, and nothing else. The nonresponse adjustment inflates the weights of respondents to compensate for selected people who did not take part. It is usually done within adjustment cells or by modelling response propensity from variables known for both respondents and nonrespondents, such as age, sex, race and Hispanic origin, and characteristics of the area. The adjustment corrects nonresponse bias only to the extent that, within cells defined by those variables, respondents resemble nonrespondents on the outcome being studied. If people with undiagnosed diabetes are less likely to attend the examination, and that tendency is not explained by the adjustment variables, the weights will not fix it. The calibration or post-stratification adjustment scales the weights so that weighted totals match independent population counts, usually from the Census Bureau's population estimates, for categories of age, sex, race and Hispanic origin. After calibration, a weighted NHANES sample reproduces the Census Bureau's count of, say, non-Hispanic Black women aged 40 to 59 exactly. NHANES provides separate weights for each stage of participation, because each stage has its own nonresponse. The interview weight applies to everyone who completed the household interview; the examination weight, which is always zero for interviewed-only participants, applies to everyone who also attended the mobile examination centre; and further weights apply to subsamples selected for particular laboratory tests, to the fasting subsample, and to participants who completed one or two days of dietary recall. For August 2021 to August 2023, NCHS also created phlebotomy weights for analyses of blood analytes, to account for examined participants who did not complete the blood draw. These last weights are an example of the survey adjusting for a stage of item-level loss that earlier cycles left to the analyst. The nonresponse problem is not academic. NCHS reports that in August 2021 to August 2023 the interview response rate was 34.6 percent and the examination response rate 25.7 percent, compared with 51.0 and 46.9 percent in the 2017 to March 2020 pre-pandemic period, and far higher rates in the early 2000s. NCHS's nonresponse bias assessment found no evidence that the sampled counties failed to represent the national population and concluded that the weighting adequately addressed the nonresponse it could detect. That is reassuring, and it is also exactly what a nonresponse analysis can conclude: it can check characteristics that are known for nonrespondents, not the health outcomes that were never measured in them. Analysts using the most recent cycle should read the assessment and consider, for their own outcome, whether nonresponse is likely to be related to it beyond what the weighting variables capture. Choosing the right weight The rule for choosing among NHANES weights is simple to state: use the weight associated with the smallest subsample that contributes a variable to the analysis. If an analysis uses interview variables and examination variables, the examination weight is correct, because only examined participants have all the variables. If it adds a laboratory variable measured on a fasting subsample, the fasting subsample weight is correct. If it adds dietary intake from two days of recall, the two-day dietary weight is correct, unless it also uses a fasting laboratory variable, in which case there is no official weight for the intersection, and NCHS's tutorials advise using the weight for the smaller subsample and accepting a modest approximation. Table 3 sets out the choice for common combinations. Table 3. Choosing an NHANES weight for common analyses (source: NCHS NHANES weighting tutorial). Variables in the analysis Weight to use Why Household interview only Interview weight All interviewed participants have the data Interview plus examination Examination weight Only examined participants have both Plus a laboratory test on a subsample Subsample weight for that test Only the subsample has the test Plus fasting laboratory values Fasting subsample weight Fasting subsample is smaller still Plus two days of dietary recall Two-day dietary weight Only two-day completers have the data Choosing the wrong weight is one of the most common errors in published NHANES analyses. Using the interview weight for an analysis of examination data gives weights that do not account for examination nonresponse and do not sum to the population total over the examined sample. Using the examination weight for a fasting glucose analysis ignores the fact that the fasting subsample was selected at random from examinees, with its own weight to reflect that selection. The error rarely changes a point estimate dramatically, but it is avoidable, and reviewers who know the survey will catch it. Combining cycles requires a further adjustment. The weights in each two-year cycle sum to the population total for that period. When two cycles are pooled, the combined weights would sum to twice the population, so each two-year weight is divided by the number of cycles pooled. NCHS provides special four-year weights for 1999 to 2002, because those two cycles were weighted to different census bases, and pre-pandemic weights for the combined 2017 to March 2020 file, because the partial 2019 to 2020 cycle cannot stand alone. Pooling 2021 to 2023 with earlier cycles is possible arithmetically but NCHS advises caution, because the interruption in data collection from April 2020 to July 2021 coincided with large changes in health care use, employment and schooling, and because the design changed. A pooled estimate spanning the pandemic gap does not describe any real period. Clusters, strata and the variance problem Weights fix the point estimate. They do not fix the standard error, and on their own they can make it worse. The standard error of a survey estimate depends on the design, and two features of the design matter most. Clustering inflates variance. People in the same county, the same neighbourhood and the same household resemble one another in health, diet, income and exposure. A sample of 5,000 people drawn from 15 counties contains less independent information than a simple random sample of 5,000 from the whole country, because much of the variation between the sampled people is shared within clusters. Leslie Kish's classic approximation expresses this through the design effect: for clusters of average size m and an intraclass correlation ρ, the variance is multiplied by roughly 1 + (m − 1)ρ. Even a small intraclass correlation produces a large design effect when clusters are large. With 300 examinees per county and ρ of 0.02, the design effect is about 7, and the effective sample size is a seventh of the nominal one. Unequal weighting also inflates variance, independently of clustering. Kish's approximation for this component is 1 + CV², where CV is the coefficient of variation of the weights. Weights that vary widely, as they do when some groups are heavily oversampled, reduce the precision of estimates for the whole population even as they increase precision for the oversampled groups. Stratification, by contrast, usually reduces variance slightly, because it guarantees that each stratum is represented in its expected proportion and removes between-stratum variation from the sampling error. To let analysts compute correct standard errors without revealing the actual counties sampled, NCHS releases masked variance units: a pseudo-stratum variable, SDMVSTRA, and a pseudo-PSU variable, SDMVPSU, constructed so that the variance structure is preserved but the geography is not identifiable. The design is represented with two pseudo-PSUs per pseudo-stratum. The number of degrees of freedom available for variance estimation is approximately the number of PSUs minus the number of strata, which in a single cycle is around 15, not the thousands implied by the number of participants. That low figure has consequences: confidence intervals should use t rather than normal critical values, and models with many parameters can exhaust the degrees of freedom available for testing them jointly. Estimating variance correctly Three families of method produce design-based standard errors, and the major statistical packages implement all of them. Taylor series linearisation approximates a nonlinear statistic, such as a ratio or a regression coefficient, by a linear function of weighted totals, and computes the variance of that linear function from the between-PSU variation within strata. It is the default in most software and the method NCHS generally recommends for NHANES. In R, the survey package, written by Thomas Lumley, specifies the design with a call along the lines of svydesign(ids = ~SDMVPSU, strata = ~SDMVSTRA, weights = ~WTMEC2YR, nest = TRUE, data = nhanes), after which functions such as svymean and svyglm produce linearised standard errors. In Stata, the equivalent declaration is svyset with the PSU, the weight and the stratum, followed by commands prefixed with svy. In SAS, the SURVEYMEANS, SURVEYFREQ, SURVEYREG and SURVEYLOGISTIC procedures take STRATA, CLUSTER and WEIGHT statements. Replication methods, including balanced repeated replication, the jackknife and the bootstrap, estimate variance by recomputing the statistic many times with modified weights and measuring how much it varies. Many surveys distribute replicate weights instead of design variables, because replicate weights can encode the design without revealing the structure. The American Community Survey public-use files, available through IPUMS, include 80 replicate weights computed by the successive difference replication method; the variance of an estimate is four eightieths of the sum of the squared differences between each replicate estimate and the full-sample estimate. The Current Population Survey's Annual Social and Economic Supplement supplies 160 replicate weights. An analyst using these files who ignores the replicate weights and computes naive standard errors will usually understate uncertainty, sometimes severely for small geographic areas. Generalised variance functions are simpler approximations, sometimes published by survey agencies, that relate the standard error of an estimate to its size. They are useful for quick checks but should not be used for formal analysis when design variables or replicate weights are available. Subpopulations: the most common error The single most frequent technical error in published survey analyses is to handle a subpopulation by deleting everyone outside it before declaring the design. An analyst studying women aged 20 to 44 drops all other participants from the file, specifies the survey design on what remains, and computes estimates. The point estimates are correct. The standard errors are not, because the number of sampled people in the subpopulation within each PSU is itself a random quantity that the variance calculation must account for, and because deleting records can remove entire PSUs from some strata, which breaks the variance computation or silently changes it. The correct approach is to declare the design on the full file and then restrict the analysis. In R's survey package this is done with the subset function applied to the design object, which retains the full design information. In Stata it is the subpop option to svy. In SAS it is the DOMAIN statement. The difference in standard errors between the correct and incorrect approaches is often small, but it is occasionally large, particularly for small subpopulations concentrated in a few PSUs, and there is no reason to accept the risk. A related issue is reliability. For small subgroups, an estimate may have so few observations or degrees of freedom that it should not be reported. NCHS publishes data presentation standards for proportions, which combine thresholds on effective sample size, degrees of freedom and the width of the confidence interval to decide whether an estimate is reliable enough to present. Adopting them, or a similar explicit rule, prevents the temptation to report every subgroup the software can produce. Reproducing the official figures The chapter began by describing a survey as a distorted picture with instructions for undoing the distortion. The best evidence that you have followed the instructions correctly is that you can reproduce an estimate NCHS has published. The NCHS Data Brief released with the August 2021 to August 2023 data reported the prevalence of obesity among adults, with a standard error. Reproducing that figure requires the correct file, the correct age restriction declared as a subpopulation, the correct examination weight, the correct design variables and, if the published figure is age-adjusted, the correct standard population. If your figure matches, your pipeline is sound. If it does not, the discrepancy will usually point directly at the mistake. Few checks in applied statistics offer so much protection for so little effort. Hashtags: #SecondaryDataAnalysis #PublicData #AdministrativeData #PopulationRegistries #CensusMicrodata #LongitudinalBiobanks #NHANES #IPUMS #UKBiobank #NordicRegistries #ComplexSurveyDesign #SurveyWeights #StratifiedSampling #ClusterSampling #VarianceEstimation #VolunteerSelectionBias #HealthyVolunteerBias #RecordLinkage #ProbabilisticLinkage #DataGovernance #TrustedResearchEnvironments #UnmeasuredConfounding #SensitivityAnalysis #SecondaryDataReproducibility #FutureOfSecondaryDataAnalysis

Download the Book (PDF): Introduction A scientific figure is a measuring instrument pointed at a reader. That claim sounds odd at first, because we are used to thinking of instruments as things that point at the world. A spectrometer measures light; a sequencer measures nucleotides; a thermocouple measures temperature. But the last step of any measurement chain is a person forming a belief, and the graphic is the part of the apparatus that does that conversion. It takes numbers, which the human nervous system cannot apprehend directly in any quantity, and converts them into stimuli that a visual system with well-documented properties will process in well-documented ways. The output of the instrument is not ink. The output is an estimate in someone's head — of a difference, a trend, a cluster, an outlier, an effect size — together with a level of confidence in that estimate. Instruments can be calibrated or uncalibrated, precise or noisy, linear or distorting. So can graphics. A bar chart whose baseline has been cut at 90 rather than zero is not a neutral display with a caveat attached; it is an instrument with a systematic gain error, and it will produce biased readings in most people who use it, including people who know about the error. A rainbow colormap laid over a continuous field is not a stylistic preference; it is an instrument that inserts false boundaries at the cyan and yellow transitions and flattens real structure in the green, and it will make readers see edges that the data does not contain. A two-dimensional embedding of a fifty-dimensional dataset is not a picture of the data; it is a lossy transform with characteristic artifacts, and a reader who does not know which properties it preserves will confidently read off quantities it never encoded. This booklet is about designing that instrument deliberately. It rests on a single controlling idea, which is worth stating plainly at the outset because everything else follows from it: Every graphical choice is a bet about what a particular perceptual and cognitive system will do with a particular stimulus, and graphical excellence consists in making those bets knowingly, in favour of the signal that is actually in the data rather than the one the designer hopes is there. Two consequences run through the chapters that follow. The first is that a large part of figure design is not a matter of taste. There is a substantial experimental literature — running from Cleveland and McGill's psychophysical studies in the 1980s through crowdsourced replications and modern work on colour difference, correlation perception and uncertainty displays — that tells us, with reasonable confidence, how accurately people extract quantities from different visual encodings. Position along a common scale beats length, which beats angle, which beats area, which beats colour saturation. That ordering is not a style guide. It is an empirical result about error rates, and it has the same status in figure design that a detector's quantum efficiency has in instrument design. Disregarding it is allowed, but it costs accuracy, and the cost should be paid on purpose. The second consequence is less comfortable. If a figure is an instrument, then distorting it is not a design flaw but a measurement error, and the distinction between an honest mistake and a manipulation collapses at the point of use. The reader of a truncated axis is misled whether or not the author intended it. This is why the middle chapters of this book spend so much time on arithmetic — on what happens to perceived ratios when you map a quantity to the radius of a circle rather than its area, on what an aspect ratio does to a perceived slope, on what a bar chart of group means conceals about the distributions underneath it. These are not ethical questions in the first instance. They are questions about transfer functions. The ethics arrive only when you know the transfer function and publish anyway. Who this is for, and what it assumes The intended reader is someone who makes figures for a technical audience: a researcher preparing a paper, an analyst building a dashboard that other people will make decisions from, a data scientist reporting on a model, an editor deciding whether a submitted graphic supports the claim in the abstract. It assumes you can already produce plots in some tool — R with ggplot2, Python with matplotlib or one of its descendants, Julia, MATLAB, D3, Vega-Lite, or a commercial dashboard platform — and that the obstacle is not syntax. It does not assume any background in vision science, and it does not require one. The perceptual results that matter for this work are few in number, robust, and statable in ordinary language. Nor does it assume a particular discipline. The examples run across genomics, climate science, epidemiology, machine learning, physics and the social sciences, because the perceptual constraints are identical everywhere and only the conventions differ. One assumption is worth flagging because it shapes the whole treatment: this book is about figures that are supposed to be read quantitatively, by people who will act on what they read. That excludes a good deal of what gets called data visualisation. Graphics designed primarily to attract attention, to be memorable, or to be shared operate under different constraints and are optimised against different criteria; the evidence on their design is real but it is a different evidence base. A figure in a methods paper and an infographic in a magazine are both visualisations in the way that a caliper and a sculpture are both metal. What is deliberately left out Short books are made good by what they refuse. This one refuses three things. It does not teach any particular software. Tools change every few years; the constraints of the human visual system have not changed in the period covered by the written record, and will not change during the working life of anyone reading this. A book organised around a plotting library ages into a manual for a version nobody runs. The occasional code fragment appears where a concrete idiom is clearer than a paragraph, but the argument is tool-agnostic and should transfer to whatever you use. It does not attempt a survey of visualisation techniques. There are excellent taxonomies that catalogue treemaps, chord diagrams, hive plots, Sankey layouts and the rest; consulting one when you have an unusual data structure is sensible. But a catalogue teaches recognition, not judgement, and the failure mode in scientific publishing is almost never that the author did not know about parallel sets. It is that a perfectly ordinary scatterplot was drawn in a way that hid the thing it was supposed to show. It does not contain a single picture. That is a constraint worth naming in a book about graphics, and it is an instructive one. Everything here is carried by prose, with a handful of tables where the material is genuinely tabular. If the argument survives that treatment it is because the underlying claims are propositional — about what the eye does, about what an encoding costs, about what a transform preserves — and propositional claims can be written down. The discipline of explaining a design in sentences also happens to be an excellent test of whether you understand it. An author who cannot describe, step by step and without gesturing, why a particular panel arrangement makes a comparison easy, usually has not worked out why it does. The shape of the argument The book moves from the eye outward. The first chapter establishes what a scientific graphic is for, historically and functionally, and why the answer has consequences. The second describes the perceptual machinery the graphic is aimed at: what is extracted instantly and in parallel, what requires effortful serial attention, what is compressed logarithmically, and what is simply not encoded at all. The third turns those constraints into an ordering of visual encodings by measured accuracy, and shows how to choose among them when accuracy is not the only criterion. Chapters four and five treat the two areas where scientific figures most often go quietly wrong: colour, which is misused in more published figures than any other channel, and scales, where a handful of arithmetic decisions — baseline, transform, aspect ratio, binning, area mapping — silently determine what the reader concludes. Chapter six takes on distributions, and argues that the most common summary display in the life sciences is also among the least defensible. Chapters seven through nine take on the harder cases named in the title. Chapter seven is about high dimensionality: what scatterplot matrices, parallel coordinates, small multiples and tours can do, what dimension reduction methods such as PCA, t-SNE and UMAP preserve and destroy, and how to read an embedding without over-reading it. Chapter eight is about the craft of the publication-grade figure — panel layout, typography, direct labelling, captions, file formats, reproducible pipelines, accessibility — the unglamorous work that separates a figure that survives peer review from one that survives use. Chapter nine is about interaction and dashboards, where the designer's power over the reader is greatest, the feedback is weakest, and the discipline required is correspondingly higher. The conclusion is not a summary. It argues something the preceding chapters make possible but do not state: that the responsibility for a figure's accuracy cannot be discharged by disclosure, that the reviewer's role in this is badly underdeveloped, and that the most useful habit an individual researcher can adopt is to treat every figure as a claim about what the reader will conclude, and to check that claim on an actual reader before publishing. There is no shortage of advice about visualisation. What there is a shortage of is the connective tissue between the psychophysics and the plotting call — the reasoning that gets you from a result about angular judgement to a decision about whether this particular comparison should be a grouped bar chart or a slope graph. That connective tissue is what this book tries to supply. It will not make every figure obvious. It should make every figure deliberate. Chapter 1: The Figure as Instrument In September 1854, John Snow marked the deaths from cholera in the Soho district of London on a street map, one short black bar for each fatal case, stacked outward from the address where the person had lived. The result is often described as the birth of epidemiological mapping, and it is usually reproduced as an image of triumphant clarity: a dense black thicket around the Broad Street pump, thinning with distance, with the Marlborough Street workhouse and the Lion Brewery sitting as conspicuously empty islands in the middle of the cluster. What makes the map an argument rather than a decoration is not the marks. It is the correspondence between a spatial question and a spatial encoding. Snow's hypothesis was about proximity to a water source. Proximity is a property of position, and the map encodes position as position. No translation is required of the reader, no mental arithmetic, no holding of one number in memory while extracting a second. The pattern that would confirm or refute the hypothesis is the pattern the eye is best at seeing: a spatially contiguous concentration of marks. The brewery mattered because its workers drank beer from the company's own well and ale allowance rather than the Broad Street pump, so its emptiness was exactly the anomaly the hypothesis predicted. A table of the same deaths by address would have contained identical information and been effectively unreadable. That is the whole business, compressed into one historical example. A scientific figure works when the structure the analyst cares about is mapped onto a visual property the reader's perceptual system extracts accurately and without effort, and when nothing else in the display competes for that extraction. It fails in one of three ways: the structure is mapped onto a property the eye handles badly, the mapping is distorting, or the display is so cluttered that the relevant property cannot be isolated. Almost every specific piece of advice in the chapters that follow is an instance of one of those three failures and how to avoid it. Three jobs, three different figures It helps to be explicit about what a given graphic is for, because the same data supports very different displays depending on the job, and much bad figure design comes from building for one job and using for another. Exploration. Here the analyst is the reader, the audience is one person, the figure is disposable, and the goal is to find out what is in the data — including things nobody suspected. Speed and coverage matter more than polish. The right instinct is to make many plots quickly, to look at raw values rather than summaries, and to accept overplotting, ugly defaults and unlabelled axes as the price of iteration. John Tukey, whose Exploratory Data Analysis is still the best single book on this mode, described the purpose as forcing yourself to notice what you did not expect. The failure mode of exploratory graphics is premature tidiness: the analyst makes one beautiful plot of the thing they already believed and stops. Confirmation and diagnosis. A narrower job: checking whether an assumption holds. Residuals against fitted values, a quantile-quantile plot against a theoretical distribution, a trace plot from a sampler, a calibration curve. These figures exist to make a specific class of deviation visible, and they are effective in proportion to how boring they look when nothing is wrong. A good diagnostic plot has a null appearance — a structureless band, a straight line, a fuzzy caterpillar — so that departure is detected as a violation of an expected pattern rather than as a judgement about degree. Communication. The published figure, the one in the paper or the report or the dashboard. Now the reader is someone else, usually in a hurry, often reading only the abstract and the figures, and frequently less invested in the result than the author. The figure has to be self-contained, correctly labelled, robust to being viewed at reduced size on a screen, and honest under the least charitable reading. This is the mode this book is mainly concerned with, and the mode where the perceptual results bite hardest, because the author cannot stand next to the reader and explain. The distinction matters because the habits of one mode are actively harmful in another. Exploratory instincts — throw everything on, use whatever colours, trust the viewer to interrogate — produce dense, unreadable published figures. Publication instincts — summarise, smooth, aggregate, polish — produce exploratory analyses that never discover anything, because everything surprising was averaged away before it could be seen. Andrew Gelman and Antony Unwin made a related point about the gulf between statistical graphics and information visualisation: the two communities optimise for different things, and a display that wins design awards may be inert as an analytical instrument, while a display that reveals structure to a statistician may be impenetrable to everyone else. Neither is wrong. They are answers to different questions. What "excellence" meant, and what it means now The modern conversation about graphical quality begins with two figures from the nineteenth century and one book from the twentieth. Charles Joseph Minard's 1869 chart of Napoleon's Russian campaign encodes six variables — the army's position in two dimensions, its direction of travel, its dwindling size, the date, and the temperature during the retreat — in a single flowing band whose width shrinks from a broad ribbon at the Niemen to a thread at the return. Florence Nightingale's polar area diagrams of mortality in the Crimea, published in 1858, were built to make one comparison unmissable to a government audience: that deaths from preventable disease vastly exceeded deaths from wounds. Both are cited endlessly, and both are worth examining not as objects of admiration but as engineering. Minard's band works because quantity is mapped to width, a length-like judgement, along a path the reader traverses in one direction. Nightingale's works less well on strictly perceptual grounds — area judgements are poor, and her wedges vary in radius so that area grows as the square of the encoded quantity — but it succeeded rhetorically, which was the job it was built for. Edward Tufte's The Visual Display of Quantitative Information, published in 1983, turned scattered craft knowledge into a set of principles: maximise the proportion of ink devoted to data, remove redundancy, avoid decoration that carries no information, and never let the graphic's size effects exceed the data's. His "lie factor" — the ratio of the size of the effect shown in the graphic to the size of the effect in the data — remains the single most useful diagnostic in the field, precisely because it is a number you can compute rather than a matter of taste. Tufte's prescriptions have been challenged in detail. Experiments on so-called chartjunk have found that embellished charts are sometimes remembered better than minimal ones, and that memorability and accuracy are not the same target; work on the memorability of visualisations has found that unusual, pictorial and dense displays stick in memory even when they are not the most efficient to read. Maximal data-ink ratios can produce sparse, hard-to-parse graphics. But the challenges are refinements of a framework that was broadly right, and the core of it — that a graphic's job is to represent quantity faithfully and that ornament competes with that job — has survived four decades of testing better than most normative advice in any field. What has changed is the evidentiary basis. Tufte argued from example and from a strong aesthetic. The work that followed, beginning with William Cleveland and Robert McGill's experiments in the mid-1980s, argued from measured error rates in controlled tasks. That shift is the reason this book is possible. We are no longer restricted to saying that a pie chart is inelegant; we can say that angular judgements produce larger errors than length judgements by a measurable margin, and that the penalty grows as the number of slices grows. A 2021 review by Steven Franconeri and colleagues in Psychological Science in the Public Interest collected what this literature now supports with reasonable confidence, and it is a good indication of how much of figure design has moved from opinion to evidence. The reader is not a general-purpose decoder The most common unstated assumption in bad figures is that the reader will do arithmetic. They will not. A stacked bar chart with four categories asks the reader to compare the second segment across bars. Only the bottom segment sits on a common baseline; every segment above it starts at a different offset, so comparing them requires either mentally subtracting two endpoints or comparing lengths floating in space. Readers do neither reliably. A dual-axis chart asks the reader to hold two different scales in mind and to treat crossings of the two lines as meaningless, which is precisely what nobody does — the crossing looks like an event, because visually it is one. A log-scaled axis asks the reader to interpret equal vertical distances as equal ratios, which trained readers can do with effort and untrained readers routinely do not do at all. Every one of these designs is defensible in some circumstance. The point is that each imposes a cognitive tax, and that the tax is paid in accuracy and in errors that the reader does not know they are making. There is a well-documented illusion, sometimes called the bar-tip limit error, in which readers treat the region under the top of a bar as more likely than the region above it even when the bar's top represents a mean with symmetric uncertainty — the bounded shape of the bar leaks into the inference. Readers do not interpret marks; they perceive them, and then rationalise. The practical consequence is a design rule that will recur throughout this book: put the comparison you want the reader to make into a single, direct perceptual judgement, and make everything else subordinate. If the claim is that treatment A exceeds treatment B, the figure should be built so that the A–B difference is the most salient length or position difference on the page. If the claim is that a trend reversed in 2014, the reversal should be visible as a change in slope, not inferred by comparing two numbers from a table-like grid. A figure that supports six comparisons equally well supports none of them well. Figures as evidence, and what that implies In most empirical fields the figure is not an illustration of the evidence. It is the evidence, in the sense that it is the form in which almost all readers encounter the result. Reviewers examine figures before they examine methods. Readers who skim read abstract, figures and captions. Meta-analysts extract effect sizes from plots when tables are unavailable. In some fields the figure is the only place the raw data appears at all. That status carries obligations that are easy to state and often ignored. A published figure should show the data, not only a model of it. When sample sizes are small — and in much of experimental biology they are very small — the individual observations should appear, because the summary cannot be trusted to represent them and because the reader has a right to see how many points there are and how they are spread. When sample sizes are large, showing everything becomes counterproductive and a principled summary is required, but then the summary should be one whose failure modes are known. A published figure should make its uncertainty visible, and should say in the caption exactly what the interval represents. An error bar with no definition is uninterpretable: standard deviation, standard error and confidence interval differ by factors that depend on sample size, and readers who are not told which one they are looking at will assume whichever supports the conclusion they are inclined toward. A published figure should be readable by the population that will read it. Roughly eight per cent of men of northern European ancestry have some form of red–green colour vision deficiency, which means that in any large readership a substantial number of people cannot distinguish the two most commonly paired categorical colours. A figure that encodes its key contrast in red versus green is not merely suboptimal; for those readers it does not work at all. And a published figure should be reproducible from data by a script. This is partly a reliability argument — hand-edited figures acquire errors that nobody can trace — and partly an argument about revision. A figure that takes twenty minutes to regenerate will be regenerated when a reviewer asks; one that took a day of manual work in a drawing program will be defended instead. A failure with a known cost On the evening of 27 January 1986, engineers at Morton Thiokol argued with NASA managers about whether to launch the Space Shuttle Challenger the following morning, in forecast temperatures far below any previous launch. The engineers believed that cold degraded the resilience of the rubber O-rings sealing the joints of the solid rocket boosters. They had data: post-flight inspections from twenty-four previous shuttle flights, recording erosion and blow-by of the seals, along with the ambient temperature at each launch. The material faxed to Kennedy Space Center that night did not put those two variables against each other. It presented damage histories organised by flight and by joint, with temperatures mentioned in text, and it concentrated on the flights that had shown damage. Flights that had flown without incident — most of them warm — were largely absent from the presentation. The consequence is easy to state in the vocabulary of this chapter: the analysis was about a relationship between two quantitative variables, and the display encoded neither of them as position on a common scale. There was no plot of damage against temperature. Without the undamaged flights, there was not even a complete sample to plot. A scatterplot of an O-ring distress index against launch temperature, including every flight, shows a pattern that later statistical reanalysis confirmed: the incidence and severity of damage rise as temperature falls, and the forecast launch temperature lay far outside the range of prior experience. Whether such a plot would have changed the decision that night is not knowable, and it would be glib to claim it. Organisational pressure, a reversed burden of proof and a compressed timeline all contributed, and historians of the episode have rightly resisted the tidy moral that better graphics would have saved the crew. But the narrower claim survives the caveats, and it is the claim this book cares about. The engineers held a conviction that the data supported, and the form in which they presented it made that support invisible — invisible to the managers, and arguably to the engineers themselves, who could not point to the pattern because they had never drawn it. The general lesson is not that graphics are decisive. It is that the display determines which patterns are available for anyone in the room to reason about. A relationship that is never encoded as a relationship cannot be seen, argued over, or defended, and the people who own the data are not immune to that limitation merely because they know the numbers. That episode also illustrates something about selection. Restricting a display to cases where the outcome occurred is a sampling decision dressed as a formatting decision, and it is astonishingly common: the plot of responders only, the trajectory panel showing the cells that survived, the map of locations where the species was found with no marks where it was looked for and not found. The graphic inherits every bias in the subset it draws, and adds the authority of having been drawn. The chain from number to belief It is worth laying out the full chain the instrument metaphor implies, because each link is a place where the signal can be corrupted, and the rest of the book works through them in order. A quantity in the data is mapped to a visual variable — position, length, angle, area, colour, texture, motion. That mapping has a transfer function, which may be linear, compressive, or ambiguous. The visual variable is rendered into a stimulus, subject to the constraints of the medium: resolution, contrast, size, the printed page or the calibrated monitor or the projector in a bright room. The stimulus falls on a retina with extremely non-uniform spatial sampling, and is processed by early visual mechanisms that extract some properties in parallel across the whole field and others only within the narrow window of attention. The extracted properties are combined with prior expectations, with the caption, with the conventions of the discipline, and with whatever the reader wanted to be true. Out comes a belief. Corruption can enter at any link. A distorting mapping — quantity to radius rather than area — corrupts at the first. A rainbow colormap corrupts at the third and fourth, by making the perceptual distance between adjacent values wildly non-uniform. Overplotting corrupts at the fourth, by making density illegible where it matters most. An unlabelled interval corrupts at the fifth. The instrument fails as a whole if any one link fails, and the author is responsible for all of them, because the reader has no way to inspect the chain. This is why the next chapter starts with the retina rather than with plot types. You cannot design an instrument without knowing the characteristics of the detector at the far end, and in scientific visualisation the detector is a visual system whose properties are known well enough to design against. Chapter 2: What the Eye Actually Does The visual system is often described as though it were a camera feeding a screen inside the head. Nothing about it works that way, and almost every useful design rule in visualisation follows from the ways it does not. Four properties matter most for figure design. The retina samples space with extreme non-uniformity. Certain visual features are extracted across the entire field at once, while others require attention to be aimed at one location at a time. Magnitude estimation is systematically non-linear, and the non-linearity differs by feature. And visual memory across glances is far poorer than introspection suggests. Each of these has direct consequences for how a graphic should be built, and the consequences are often the opposite of what a designer's intuition proposes. Acuity is a narrow spotlight Detailed vision is confined to the fovea, a region of retina subtending roughly two degrees of visual angle — about the width of a thumbnail at arm's length, or a coin held at a metre. Cone density falls off sharply outside it, and spatial resolution falls with it. At ten degrees of eccentricity, acuity is a small fraction of what it is at the centre. Crowding compounds the loss: in peripheral vision, nearby objects interfere with one another's identification even when each is individually resolvable, so a dense scatter of small marks off to the side is not merely blurry but structurally unreadable. What this means in practice is that reading a figure is a sequence of fixations. The eye lands, extracts detail from a small region, and jumps — three or four times a second, with vision largely suppressed during the jump. A complex multi-panel figure is not apprehended; it is scanned, in an order determined partly by layout and partly by what the periphery flags as worth looking at. Two design consequences follow. First, anything that must be read precisely — a tick label, a small annotation, a distinction between two similar symbols — has to be at the point of fixation, which means the figure's design should control where fixations go. Direct labelling of a line at its end works because the label is where the eye already is after tracing the line; a legend in a corner works less well because each lookup costs a saccade, a fixation, a return, and a memory operation in between. Second, peripheral vision is not useless — it is what guides the next fixation — but it only carries coarse information: large-scale luminance structure, strong colour blobs, motion, orientation at low spatial frequency. A figure whose important structure is visible only at high spatial frequency will never attract the eye to it. This is the honest argument for a degree of visual salience in scientific figures. Not decoration, but the deliberate use of size, contrast and position to make the eye land where the evidence is. If the key comparison is between two of twelve lines, those two should be heavier and darker and the others grey, because otherwise the reader's fixation sequence is determined by accident. Preattentive features: what is free Some visual properties are computed in parallel across the whole visual field, early and fast, before attention is deployed. The classic demonstration is visual search: a red dot among blue dots is found in roughly the same time whether there are ten distractors or a hundred, because the colour difference is available everywhere at once. Searching for a red square among red circles and blue squares — a conjunction of two features — takes time proportional to the number of items, because conjunctions require attention to bind features at each location in turn. Anne Treisman and Garry Gelade's feature integration theory, published in 1980, formalised this distinction, and while the theory has been revised repeatedly since, the empirical pattern is solid and directly useful. The features that support parallel search include hue, luminance, size, orientation, curvature, motion, flicker, spatial position, length, width, and a handful of others including stereoscopic depth and certain texture statistics. A rough working list for figure design would be: colour, brightness, size, orientation, shape at a coarse level, position, and motion. Three practical rules come out of this. One channel per question. If you want an outlier class to pop out, encode it on a single preattentive dimension that nothing else uses. If group membership is encoded by colour and the same colour is also doing duty for measurement condition, neither pops. Conjunctions are expensive. A figure that requires the reader to find "the filled triangles that are also blue" imposes a serial search. Sometimes that is unavoidable — a scatterplot with two categorical variables has to encode both — but it should be a conscious cost, and the more important variable should get the stronger channel. Pop-out is graded, not binary. The speed advantage depends on how different the target is from the distractors relative to the variability among the distractors. A category encoded in a slightly darker blue among other blues does not pop out at all. Designers routinely underestimate how large a difference has to be to work. There is a corresponding negative rule. Because pop-out is automatic, anything that is highly salient will be attended whether or not it is important. Gridlines at full contrast, heavy plot borders, saturated background panels, drop shadows, and bold reference lines all compete for the same early visual resources as the data. This is the defensible core of the data-ink argument: not that ink is inherently wasteful, but that salience is a fixed budget and non-data elements spend it. Magnitude perception is compressive and channel-specific Ask people to judge how much bigger one stimulus is than another and their answers are not proportional to the physical difference. For many continua the relationship follows a power law — Stevens's law — with an exponent characteristic of the dimension. Brightness and area are strongly compressive: doubling the physical quantity produces a perceived increase considerably less than double. Line length is nearly veridical, with an exponent close to one. This is why a bar chart works and a bubble chart does not, quite apart from the geometry. The geometry compounds the psychophysics in one particularly common error. If a quantity is mapped to the radius of a circle, the area grows as the square of the quantity, so a value three times larger occupies nine times the ink. Mapping to area instead removes that error, but leaves the perceptual compression: people underestimate area ratios, typically reading a circle of nine times the area as something like four or five times the value. The cartographic literature has grappled with this since the 1970s, when psychophysical scaling of proportional symbols led to proposals for deliberately exaggerated area scales to compensate. Those corrections are contested and rarely used in scientific plotting, which leaves the practical advice simple: use area encodings only for rough orders of magnitude, never for comparisons the argument depends on, and if you must use them, scale by area and label the extremes. A related result concerns the just-noticeable difference. Ronald Rensink and Gideon Baldridge showed that the precision with which people can discriminate correlation in a scatterplot follows a regular law, with discrimination becoming markedly finer as correlation approaches one; Lane Harrison and colleagues extended this across nine different chart types and found the same Weber-law form, with different sensitivity constants per chart type. The practical implication is under-appreciated: the difference between r = 0.2 and r = 0.3 is nearly invisible in a scatterplot, while the difference between r = 0.9 and r = 0.95 is easy to see. If your argument turns on a small difference in weak correlation, the plot will not carry it and you should report the number. Colour is three channels, and they are not equal Colour is not a single continuum. Human colour vision is trichromatic at the receptor level, but the signals are immediately recoded into one achromatic channel (light–dark) and two chromatic opponent channels (roughly red–green and blue–yellow). These channels have very different properties. The luminance channel carries high spatial resolution. Edges, texture, shape from shading and fine detail are all conveyed by luminance. The chromatic channels are low-pass: they carry much coarser spatial information, which is why image compression schemes throw away colour resolution first and why thin coloured lines on a background of similar lightness are hard to see even when the hue difference is large. This single fact explains a large fraction of the colour failures in published figures. A heatmap that encodes its quantity purely in hue, with roughly constant lightness, presents fine spatial structure on the channel least able to resolve it. A categorical palette in which all the colours have similar lightness produces lines that are perfectly distinguishable at large size and mushy at print size. A continuous colormap whose lightness is non-monotonic — rainbow being the archetype — creates apparent edges wherever lightness happens to reverse. Ordered data therefore needs ordered lightness. This is the one colour rule that resolves most disputes: if the variable has a magnitude, the colormap must change monotonically in perceived lightness across its range, with hue and saturation doing secondary work to widen the discriminable range. Chapter four develops this at length, but it is fundamentally a perceptual constraint rather than an aesthetic preference. Colour vision deficiency adds a second constraint. Anomalies of the red–green opponent channel affect around eight per cent of men and under one per cent of women of northern European descent, with somewhat different rates in other populations. For these readers, red and green of matched lightness are close to indistinguishable. The robust solutions are to vary lightness along with hue, to use blue–orange rather than red–green pairings, and to add a redundant non-colour channel — shape, line style, direct labels — for anything essential. Visual working memory is tiny Between one fixation and the next, very little is retained. The phenomenon of change blindness — a large change to a scene going unnoticed when it occurs during a saccade, a blink, or a brief blank — demonstrates that we do not hold a detailed internal image. Estimates of visual working memory capacity converge on something like three to four simple objects, and fewer when the objects are complex. Three implications for figures, all of them significant. Comparisons that require memory are unreliable. Any design that forces the reader to look at one panel, memorise a value, and look at another has inserted a low-capacity buffer into the measurement chain. This is why animation is frequently worse than small multiples for comparing states: animation puts the comparison in memory, while small multiples put it side by side where it becomes a simultaneous perceptual judgement. Experiments comparing animated and static displays of trend data have generally found small multiples more accurate, with animation sometimes preferred and enjoyed more even while producing more errors. Legends are memory operations. Every legend lookup is an encode–saccade–match cycle, and with more than about five or six categories the reader begins losing track. Direct labelling removes the cycle entirely, and is the single highest-yield change available in most line charts. Ordering is a memory aid. Sorting categories by value rather than alphabetically, arranging small multiples so that adjacent panels differ in one variable at a time, and keeping axis ranges identical across panels all reduce what the reader must hold in mind. Inconsistent axis ranges across panels of a multi-panel figure are the most common way to make a comparison impossible while appearing to enable it. Absolute judgement is severely limited There is a difference between telling two stimuli apart when they sit side by side and identifying one in isolation. The first is relative discrimination and it can be remarkably fine; the second is absolute judgement and it is poor. George Miller's 1956 review of the experimental literature found that across a wide range of unidimensional continua — tone pitch, loudness, line length, position of a marker on a line, saltiness of a solution — people could reliably assign stimuli to only about seven distinct categories, and often fewer. The number rises somewhat when several dimensions are combined, but never dramatically. This is why a legend that maps twelve shades of blue to twelve values is not a quantitative encoding. A reader can tell two adjacent patches apart when they touch; asked to look at a patch in the middle of a map and say which legend entry it matches, they cannot do it beyond a handful of levels. The same limit applies to line thickness, symbol size, texture density and hue. For colour, careful work on colour difference perception in visualisation — including models that account for the size of the mark, since small marks are much harder to discriminate than large ones — puts the number of reliably identifiable categorical colours at somewhere between eight and twelve under good conditions, and the number for small scattered points considerably lower. Two design rules follow. When a channel with low absolute-judgement capacity carries a quantitative variable, treat the encoding as ordinal — the reader will be able to say "more" or "less", not "how much" — and provide the actual numbers wherever the argument needs them. And when a categorical variable has more levels than the channel can carry, do not add colours; change the design. Facet into small multiples, group the minor categories into an "other" class, label the important series directly and grey out the rest, or accept that the figure is a texture showing overall shape rather than a lookup table. The corollary is that redundant encoding is not waste. Encoding the same variable twice — colour and position, colour and shape, colour and direct label — converts an absolute judgement into a relative one, and costs nothing but ink. In a scatterplot with four groups, using both colour and marker shape is often derided as redundant; in practice it is what makes the figure work when it is printed in greyscale, viewed by a reader with a colour vision deficiency, or reduced to a column width. Gestalt grouping: the eye organises before you do Early vision imposes organisation on marks whether or not it corresponds to structure in the data. Marks that are close together are grouped; marks that are similar in colour or shape are grouped; marks that fall along a smooth continuation are read as one object; a closed contour is read as a region with an inside; elements that move or change together are read as a unit. These principles are tools when exploited and traps when ignored. Grouping by proximity is why whitespace is the most effective panel separator and why a grouped bar chart with equal spacing between all bars fails to communicate its grouping. Continuity is why connecting points with a line asserts that the intervening values exist — a line between two categorical levels is a factual claim about interpolation, and usually a false one. Closure is why a filled area under a curve reads as a quantity, so that filling under a line whose baseline is not zero quietly implies a magnitude that is not there. Common fate is why brushing in an interactive display works so well: making the selected points change together binds them into an object across panels. The most common gestalt error in scientific figures is unintended grouping by colour. Using one palette for experimental conditions in panel A and re-using the same palette for a different variable in panel B makes the reader group across panels, because colour identity is a stronger grouping cue than panel membership. Within a single figure, a colour should mean one thing. Putting the constraints together Taken together, these properties describe a detector with an unusual specification: extremely high resolution in a tiny window, coarse but parallel processing everywhere else, accurate for position and length, compressive for area and brightness, split into one sharp achromatic channel and two blurry chromatic ones, with almost no memory between glances, and a strong built-in tendency to organise marks into groups and continuations. A figure designed for that detector looks like this. The critical quantity is encoded as position or length on a common, unbroken scale. The critical comparison is adjacent, so that no memory is required. Categories are few, and distinguished on more than one channel. Colour with a magnitude varies in lightness monotonically. Salience is spent on the data rather than on frames, grids and backgrounds. Labels sit next to the things they label. Panels share scales, and differ in one variable at a time. Nothing is asked of the reader that requires arithmetic, and nothing important is asked of peripheral vision. That specification is abstract. The next chapter makes it concrete by ranking the available encodings against measured human performance, which turns "encode the critical quantity well" into a decision procedure. Chapter 3: The Ranking of Encodings In 1984 William Cleveland and Robert McGill published a paper in the Journal of the American Statistical Association that did something the field had not done before: it treated the choice of graphical encoding as an empirical question with a measurable answer. Their method was to decompose graph reading into what they called elementary perceptual tasks. When a reader extracts a quantity from a graphic, they are performing one of a small number of operations — judging position along a common scale, judging position along identical but non-aligned scales, judging length, judging direction or angle, judging area, judging volume or curvature, judging shading or colour saturation. Different chart types demand different tasks. A dot plot requires position along a common scale. A stacked bar chart requires length. A pie chart requires angle. A bubble chart requires area. A choropleth map requires colour. Cleveland and McGill then ran experiments in which subjects estimated ratios between marked values in displays that isolated each task, and measured the error. The result was an ordering, from most accurate to least, which Table 1 sets out alongside the chart forms that depend on each task. Table 1. Elementary perceptual tasks ranked by measured accuracy, after Cleveland and McGill (1984), with the chart forms that rely on each. Rank Perceptual task Typical chart forms Use for 1 Position along a common scale Dot plot, scatterplot, line chart, aligned bars Quantities the argument depends on 2 Position on identical, non-aligned scales Small multiples with shared axes Quantities compared across panels 3 Length Stacked bars, floating bars, Gantt spans Secondary quantities 4 Angle and slope Pie charts, slope graphs, line angles Trend direction, not magnitude 5 Area Bubble charts, treemaps, cartograms Rough orders of magnitude 6 Volume, curvature Three-dimensional symbols Avoid 7 Shading, colour saturation Choropleths, heatmaps Ordinal patterns, spatial context The ordering has held up. Jeffrey Heer and Michael Bostock replicated the core experiments in 2010 using crowdsourced participants, obtaining results consistent with the original ranking, and extended the method to questions Cleveland and McGill had not addressed — including how the accuracy of rectangular-area judgements varies with the aspect ratio of the rectangles, and how small a chart can be made before accuracy degrades. Later work has filled in the constants for particular comparisons. The qualitative ordering is now about as settled as anything in the field. What the ranking is and is not It is a statement about accuracy in ratio estimation, and nothing else. It does not say that pie charts are forbidden, that heatmaps are bad, or that every figure should be a dot plot. It says that if a reader must extract a quantitative comparison, the encodings near the top of the list will produce smaller errors than the ones near the bottom, and it quantifies roughly by how much. Three qualifications keep it from being applied mechanically. Accuracy is not the only objective. A choropleth map encodes quantity in colour, the worst channel on the list. It is nevertheless the right display for many geographic questions, because the spatial arrangement is itself the point and no other encoding preserves it. A treemap uses area, ranked fifth, but it can show a thousand nested items in a space where a ranked dot plot would need thirty pages. The rule is not to choose the top encoding always; it is to know what you are paying when you go down the list, and to spend it on something. The task matters more than the chart type. A pie chart is poor for comparing two similar slices and perfectly adequate for the judgement "this is about half". A stacked bar is poor for comparing middle segments and fine for showing that the total is roughly constant. Naming a chart type good or bad without naming the task is a category error, and it is the reason chart-type arguments on the internet never resolve. Ratio estimation is not the only elementary task. Later work by Robert Amar, James Eagan and John Stasko catalogued the low-level operations analysts actually perform: retrieve a value, filter, compute a derived value, find extrema, sort, determine range, characterise distribution, find anomalies, cluster, correlate. Several of these are not ratio judgements at all. Finding anomalies benefits from preattentive pop-out; characterising a distribution benefits from a display that shows shape; correlating benefits from a scatterplot, which is not near the top of the ranking for reading individual values but is unrivalled for reading joint structure. Match the encoding to the operation, not to a list. From ranking to a design procedure The ranking becomes useful when combined with a second idea, due to Jock Mackinlay in a 1986 paper on automating graphic design. He proposed two criteria that any mapping from data to graphic must satisfy. Expressiveness: the graphic should express all the facts in the data, and only the facts in the data. A mapping that leaves out information fails; so does one that adds information the data does not contain. Plotting unordered categories along a positional axis violates the second half, because position implies an ordering that does not exist — which is why alphabetical category ordering is so pernicious, since it asserts a sequence that means nothing while suppressing the meaningful one. Connecting categorical values with a line asserts interpolation. Using a continuous colour ramp for nominal categories asserts that some categories are between others. Effectiveness: among expressive mappings, choose the one the reader decodes most accurately, which is where the Cleveland–McGill ranking enters, adjusted for the data type. The effective channels differ by type. Quantitative data is best on position, then length, then angle, then area, then density, then saturation, then hue. Ordinal data is best on position, then density, then saturation, then hue, then texture, then connection. Nominal data is best on position, then hue, then texture, then connection, then containment, then density, then shape. Together these give a procedure that will produce a defensible first draft for almost any dataset: 1. List the variables and their types: quantitative, ordinal, nominal, temporal, spatial. 1. Identify the one or two comparisons the figure exists to support. Write them as sentences. 2. Assign the variables in those comparisons to the highest-ranked channels available for their type. 3. Assign the remaining variables to the channels left over, or move them out of the figure into faceting, or drop them. 4. Check expressiveness: does the encoding imply any ordering, continuity, or magnitude that the data does not have? Step 2 is the one people skip, and it is the one that does the work. A figure whose purpose is "show the data" has no basis for choosing among encodings and generally ends up with whatever the plotting library does by default. A worked case: six treatments, four assays, three replicates Abstract rules are easy to agree with and hard to apply. Take a concrete and thoroughly ordinary dataset: a screening experiment with six compounds, each measured on four assays, with three biological replicates per combination. Seventy-two numbers. The question the paper asks is whether any compound shows activity across multiple assays rather than in one alone. The default output of most analysis pipelines is a heatmap: compounds on rows, assays on columns, mean activity in colour. It fits on a postage stamp and it is what everyone in the field draws. Run it through the procedure. The comparison the figure exists to support is: within a compound, is activity elevated on more than one assay, and is that elevation large relative to replicate noise? That comparison involves a quantitative variable (activity), two nominal variables (compound, assay), and a measure of dispersion. The heatmap puts the quantitative variable on colour — rank seven — and discards the replicates entirely. It is expressive about the two nominal variables and mute about the one thing the question turns on. A better mapping falls out almost mechanically. Activity is quantitative and carries the argument, so it goes on position along a common scale. Assay is nominal with four levels, and the comparison is within compound and across assay, so assay goes on the other positional axis — as a categorical axis, ordered not alphabetically but by something meaningful, such as mechanistic proximity or overall signal. Compound has six levels and the comparison across compounds is secondary, so it becomes the faceting variable: six small panels, shared axes, arranged in a single row or a two-by-three block. Replicates are plotted as individual points, because with n of three no summary is credible and three dots cost nothing. If a summary is wanted, a short horizontal line at the mean over the points serves, with no bar and no error bar pretending to be an interval. The result is six panels of twelve points each. Position along a common scale carries the quantity; panels on identical non-aligned scales carry the compound comparison, which is rank two; the reader sees replicate spread directly rather than inferring it. Nothing is encoded in colour at all, which means the figure survives greyscale printing and can spend colour later if a highlight is needed. Now the honest accounting. The heatmap has two real advantages that the faceted version gives up. It is far more compact, so if the screen had contained sixty compounds rather than six the faceted display would be unusable and the heatmap would be the only viable option. And a heatmap sorted by clustering reveals block structure — groups of compounds behaving alike — that separated panels do not, because similarity of colour pattern is a texture judgement made across the whole matrix at once. If the paper's claim were about clusters of compounds, the heatmap would be the right instrument and the faceted plot the wrong one. That is what applying the ranking actually looks like. Not "heatmaps are bad", but: name the comparison, find the channel that serves it, notice what the alternative buys, and choose with the cost in view. The same six compounds might appear as a heatmap in a supplementary screen-wide panel and as a faceted dot plot in the main figure that supports the central claim, and both would be correct. One further move is worth noting because it recurs. Suppose the question changes slightly: not "which compounds are active" but "did activity go up or down between a control and treated condition for each compound". Now the data is paired, and pairing is structure the display should preserve. A grouped bar chart of control and treated means destroys it — the reader sees two population summaries and cannot recover which control value belongs with which treated value. A slope graph, in which each compound is a line segment joining its control value on the left axis to its treated value on the right, encodes the paired change as slope and direction. Slope sits at rank four for magnitude, but the judgement here is not magnitude; it is sign and consistency, which slope and crossing patterns deliver preattentively. A dozen lines all tilting the same way is a result visible in a quarter of a second, and no bar chart of means conveys it at all. The general lesson is that the ranking answers the question "how accurately will the reader read a quantity from this channel", and many scientific claims are not about quantities read one at a time. They are about consistency, ordering, pairing, sign, dispersion, or the presence of an anomaly. For those, ask which display makes the relevant pattern a single perceptual object, and use the ranking to settle the residual choices. Position, and why it wins so decisively Position along a common scale wins for several converging reasons. It is preattentive, so gross patterns are available without search. It is nearly veridical — the psychophysical exponent for length and position judgements is close to one, so perceived differences track real differences. It supports both relative judgements and, with axis labels, approximate absolute ones. It permits many marks in a small space without the marks interfering. And it is the only channel that comfortably supports the judgement of two variables at once, which is what makes the scatterplot the most information-dense elementary display we have. The practical consequence is a strong default: when in doubt, put the quantity on an axis. An enormous fraction of figure improvement in practice consists of converting something else into a positional encoding. A pie chart of shares becomes a sorted dot plot. A stacked bar of multiple series becomes a set of small multiples of aligned bars, or a line chart if the categories are ordered. A bubble chart of three variables becomes a scatterplot of the two that matter with the third as a facet. A heatmap whose rows are being compared becomes a set of profile lines. A table of numbers whose point is a ranking becomes a dot plot, which Cleveland introduced for exactly this purpose and which remains badly underused. Bars deserve a note of their own. A bar chart encodes quantity in both position (the top of the bar) and length (the bar itself), which is why it reads so easily. The doubling is also why the baseline cannot be moved: truncating the axis breaks the length encoding while leaving the position encoding intact, and the reader perceives both. A line chart has no length encoding, which is precisely why its axis may legitimately be truncated when the data warrants it. This is the single most useful thing to know about the truncation debate, and it is developed further in chapter five. When to go down the list on purpose There are good reasons to use a lower-ranked encoding, and it is worth naming them so that the choice can be made rather than drifted into. Space and cardinality. With ten thousand items, position-based displays run out of room. Colour and area scale to densities that position does not. A genome-wide heatmap, a treemap of a filesystem, a hexbin density plot — these accept a weaker channel to gain capacity. Preserving a spatial or topological structure. Maps, brain images, microscopy, network layouts, and anything where the spatial arrangement of the data has physical meaning. Here position is already spoken for, and the measured variable must go somewhere else. Colour is the standard answer and the reason careful colormap design matters so much in these fields. Gist over precision. If the message is "most of this is one category" or "the pattern is roughly uniform", a lower-ranked encoding may deliver the gist faster. A pie chart genuinely does communicate part-of-whole membership quickly, which is why it survives. Convention. Fields have reading habits. Manhattan plots, Kaplan–Meier curves, volcano plots, forest plots, Bland–Altman plots, receiver operating characteristic curves: each is a compact convention that a trained reader parses far faster than a from-scratch alternative, even where the encoding is not theoretically optimal. Deviating from a strong convention imposes its own cost, and it should be done only when the conventional form actively misleads. What is not a good reason is novelty, or the desire to fit more variables into one display than the reader can extract. The commonest expression of the second is the three-dimensional bar chart, which converts an accurate position judgement into an inaccurate volume judgement, adds occlusion, and introduces perspective distortion that makes bars at the back systematically smaller. It has no defensible use in quantitative work. Redundancy, and the grammar underneath The encoding ranking presumes that each variable is mapped to one channel. In practice good figures often map one variable to two channels at once. A series encoded by both colour and line style survives greyscale printing. Points encoded by both colour and shape survive colour vision deficiency and small size. A sorted bar chart encodes value in both length and vertical position. Redundancy costs nothing except the channels it uses up, and it converts absolute judgements into easier ones. The only caution is that a redundantly encoded variable consumes channels that another variable might have needed, and that redundancy between different variables — where colour means condition in one place and timepoint in another — is the opposite of helpful. Underneath all of this sits a formal structure worth naming, because it changes how you think about plots. Leland Wilkinson's The Grammar of Graphics, published in 1999 and implemented in Hadley Wickham's ggplot2 and in Vega-Lite, treats a statistical graphic not as a chart type chosen from a menu but as a composition: data, a statistical transformation, a mapping from variables to aesthetic channels, geometric objects that realise those channels, a coordinate system, and scales with guides. Under that grammar, a pie chart is a stacked bar in polar coordinates, and a bar chart and a histogram differ only in the statistical transformation applied before drawing. Chart types stop being primitives. The practical value of the grammar is that it makes the encoding decision explicit and forces it into the code. When you write a mapping of a variable to the y position and another to colour, you have stated your design in the same vocabulary the perceptual literature uses, and the question "is this the right channel for this variable?" becomes a question you can actually answer. Hashtags: #ScientificVisualization #DataVisualization #HighDimensionalData #GraphicalPerception #VisualEncoding #PerceptualAccuracy #ClevelandMcGillRanking #PositionEncoding #ColourPerception #ScientificColormaps #DataIntegrity #UncertaintyVisualization #DistributionVisualization #SmallMultiples #DimensionalityReduction #PrincipalComponentAnalysis #TSNE #UMAP #ScatterplotMatrices #ParallelCoordinates #PublicationGraphics #ReproducibleFigures #VisualizationAccessibility #GrammarOfGraphics #FutureOfScientificVisualization

Download the Book (PDF): Introduction Most researchers learn to communicate in a single, highly specialised genre. The journal article has a fixed architecture: introduction, methods, results, discussion. It assumes a reader who already cares, who shares a vocabulary, who will forgive a slow opening because the payoff is a contribution to a literature they are tracking. It rewards caution, qualification and completeness. It withholds the conclusion until the evidence has been laid out, and it treats the author's personality as noise to be filtered out. These are virtues inside the walls of a discipline. Outside them, almost every one of those habits becomes a liability. A parent deciding whether to vaccinate a child, a city councillor weighing a flood defence scheme, a journalist with ninety minutes before deadline, a video editor asking for a thirty-second version of a four-year project, a stranger on social media who has decided your paper is part of a conspiracy: none of these people is the reader the journal article was designed for. They arrive with different questions, different stakes, different amounts of time and different reasons to trust or distrust you. Communicating with them is not a matter of saying the same thing more slowly or with fewer long words. It is a different task, with its own structure, its own failure modes and, increasingly, its own body of evidence. That last point matters. For a long time, advice on public communication came in two flavours. The first was inspirational: be passionate, tell stories, get out of the lab. The second was defensive: stay away from journalists, never speculate, let the work speak for itself. Neither was grounded in much more than anecdote. Over the past four decades, however, psychologists, risk researchers, political scientists and communication scholars have built a serious literature on how people actually take in, interpret and act upon scientific information. Baruch Fischhoff and his colleagues at Carnegie Mellon developed methods for finding out what people already believe before trying to change it. Michael Dahlstrom has reviewed what is known about how narrative works on non-expert audiences and where it becomes ethically fraught. Matthew Nisbet and others have shown how the frame around a scientific finding shapes which audiences listen and what they conclude. Roger Pielke Jr. has given researchers a vocabulary for the different roles they can play in policy debates. Randy Olson, a marine biologist turned filmmaker, distilled narrative structure into a template that working scientists can apply in an afternoon. Studies of press releases have traced where exaggeration in health news actually comes from, and the answer has made university press offices uncomfortable. Research on misinformation has overturned some confident claims about how corrections backfire. This booklet draws on that literature to give researchers a practical, evidence-informed approach to communicating with the public, with policymakers and with the media. Its controlling argument is simple to state and demanding to practise: communication beyond the academy succeeds when researchers start from what a particular audience needs to understand or decide, build a story that the evidence can honestly carry, and choose deliberately what role they are playing, so that the goal becomes earning trust rather than winning an argument. Each part of that sentence carries weight. Start from the audience. The dominant instinct among scientists is to begin with the finding and work outward. The evidence consistently points the other way. People interpret new information through mental models they already hold, through the values and identities of the groups they belong to, and through the decisions they are facing. A message designed without knowing any of this will be misunderstood in predictable ways. The first chapter examines why the old "deficit model", in which public scepticism is treated as a shortage of facts to be topped up, fails so persistently, and what better models ask communicators to do instead. Build a story the evidence can carry. Narrative is not decoration. It is how human beings organise cause, consequence and meaning, and it is remarkably effective at holding attention and aiding recall. It is also a powerful instrument that can outrun the data it describes. The second chapter lays out the structure of scientific narrative, from Olson's "And, But, Therefore" template to the larger arcs that suit longer formats, while taking seriously the ethical questions Dahlstrom and others have raised. The third chapter turns to framing: the choice of which aspect of a finding to put in the foreground, and why that choice so often determines who listens. The fourth deals with language and numbers, the level at which most communication actually breaks down, and offers concrete techniques for removing jargon and presenting risk in forms people can use. Choose your role. A researcher who speaks to a minister, a reporter or a public meeting is doing something different from one who publishes a paper, and the difference is not only one of style. It raises questions about authority, advocacy and responsibility. The fifth chapter uses Pielke's typology of roles, together with research on communicating uncertainty, to help researchers decide in advance how they will engage with policy. The sixth covers the practical realities of working with journalists, from writing an accurate press release to preparing for a live interview. The final two chapters address formats and hazards that the older advice barely anticipated. Video abstracts, short explainers and visual summaries are now routine requests from journals, funders and institutions, and they reward a distinct set of skills in scripting, pacing and design. Social media, meanwhile, has made researchers directly visible to very large audiences, which brings reach and conversation but also coordinated hostility, misinformation and, in the worst cases, threats. The seventh chapter treats the video abstract as a genre with its own rules. The eighth offers a way of distinguishing legitimate critique from bad-faith attack, and of responding to each without being either defensive or naive. A few words on what this booklet is not. It is not a guide to becoming a celebrity scientist, and it does not assume that every researcher should be doing public engagement all the time. Some people will never want to appear on camera; some fields rarely touch public controversy; some career stages leave little room for anything but the core work. The aim is to make the engagement you do choose to undertake more effective and less risky, and to help you recognise when a request is one you should decline. Nor is it a manual for persuasion in the marketing sense. Several of the techniques described here, narrative and framing above all, can be used to push audiences towards conclusions the evidence does not support. The booklet is explicit about where those lines fall, because a researcher's credibility is a long-term asset that a single overreaching message can spend. The examples throughout are drawn from real, documented cases wherever possible: the L'Aquila earthquake trial, the "arsenic life" controversy, the conflicts over the MMR vaccine, the harassment of researchers during the COVID-19 pandemic, and others. Where an example is invented to illustrate a technique, it is labelled as hypothetical. The research cited is real and listed in the Notes and Further Reading at the end, so that readers who want to go deeper can check the sources and draw their own conclusions. One final point about disposition. Researchers are trained to be sceptical of rhetoric and to trust data, and many find the language of "storytelling" and "messaging" faintly distasteful. That instinct is healthy and worth keeping. But the choice is never between communicating with a story and communicating without one. Every press release, every briefing, every conference talk already has a structure, a frame and a set of word choices; the only question is whether those were chosen deliberately or by default. A default structure, inherited from the journal article, will serve a lay audience poorly and will often be misread. Deliberate choices, made with an understanding of how audiences work and a firm commitment to the integrity of the evidence, are what this booklet sets out to teach. Chapter 1: Beyond the Deficit: What Audiences Actually Do with Science In 1985 the Royal Society published a report, chaired by the geneticist Walter Bodmer, called The Public Understanding of Science. It argued that a scientifically literate public was essential to a modern democracy and that scientists had a duty to communicate with it. The report was influential and well-intentioned, and it helped launch a generation of public lectures, science festivals and popular books. It also crystallised a set of assumptions that have proved remarkably durable: that the public's relationship with science is chiefly a matter of knowledge, that scepticism or opposition reflects a shortage of that knowledge, and that the remedy is for experts to supply more of it, more clearly. Scholars later gave this cluster of assumptions a name: the deficit model. Fifteen years after Bodmer, the House of Lords Select Committee on Science and Technology published a report, Science and Society (2000), written in the aftermath of the BSE crisis, in which British officials had spent years reassuring the public that beef was safe before the government announced in 1996 a probable link between BSE and a new variant of Creutzfeldt-Jakob disease in humans. The Lords described a crisis of confidence and called for a shift from one-way communication towards dialogue. The phrase "from deficit to dialogue" became a slogan of the field. And yet, as any researcher who has sat through a media-training session or read a university's public engagement strategy can confirm, the deficit model has never gone away. Understanding why it fails, and why it persists anyway, is the necessary starting point for everything else in this booklet. Why more facts rarely settle the matter The deficit model makes a testable prediction: people who know more science should hold attitudes closer to the scientific consensus. The evidence does not support it in any strong form. In a widely cited 2004 meta-analysis of survey data from Europe and the United States, Patrick Sturgis and Nick Allum found that scientific knowledge is associated with attitudes towards science, but modestly, and that the relationship varies considerably by topic and is shaped by political and cultural factors. Knowledge matters; it is simply one input among several, and often not the decisive one. The more striking result came from work on what Dan Kahan and colleagues call cultural cognition. In a 2012 paper in Nature Climate Change, Kahan's group reported that among Americans, higher scores on measures of science literacy and numeracy were not associated with greater concern about climate change overall. Instead, they were associated with greater polarisation: the most scientifically literate people on each side of the cultural divide were the furthest apart. The interpretation that has attracted the most attention is that people use their reasoning skills to defend positions that signal membership of groups they care about. If accepting a finding would put you at odds with your community, then being cleverer mainly makes you better at finding reasons to reject it. This literature is sometimes over-read. It does not show that facts are useless or that everyone is hopelessly tribal on every topic. Most scientific findings never become identity markers, and on those topics clear information does shift understanding. Later work has also debated how general the polarisation effect is. But on the minority of issues that have become entangled with political or religious identity, such as climate change, evolution, vaccination in some communities, nuclear power and genetically modified crops, the lesson is sobering. A communicator who treats resistance as ignorance will misdiagnose the problem, and a message built on that diagnosis may make things worse by implying that the audience is stupid. Why, then, does the deficit model survive? A 2016 paper by Molly Simis, Haley Madden, Michael Cacciatore and Sara Yeo in Public Understanding of Science examined this question directly. They suggested several reasons: scientists' training emphasises rational, evidence-based argument and may lead them to assume others reason the same way; the model is simple and flatters the expert's role; and institutional structures, including how communication is funded and evaluated, reward counting outputs such as lectures delivered and articles written rather than changes in understanding or relationships. In other words, the deficit model persists less because anyone defends it on the evidence than because it is the path of least resistance. The alternatives that have grown up since the Lords report are often grouped into three broad families, summarised in Table 1. The categories overlap in practice, and most real engagement mixes them, but it is useful to know which assumptions you are operating under. Table 1. Three broad models of public engagement with science. Model Assumption about the public Role of the researcher Typical formats Main limitation Deficit Lacks knowledge; scepticism signals ignorance Expert who transmits facts Lectures, press releases, explainers Ignores values, identity and trust Dialogue Has relevant knowledge, concerns and values Participant in two-way exchange Q&A events, consultations, cafés Can become consultation without influence Participation Has a legitimate stake in shaping research and its uses Co-producer and partner Citizen juries, co-design, citizen science Costly, slow, hard to scale The dialogue and participation models are not simply more virtuous versions of the deficit model. They answer different questions. If a research council wants to know how the public weighs the benefits and risks of a new technology before it is widely deployed, an approach that James Wilsdon and Rebecca Willis described in their 2004 Demos pamphlet See-through Science as "upstream engagement", a lecture explaining the technology will not produce that knowledge. If a hospital wants patients to understand how to take a new medication safely, a deliberative citizens' jury is overkill. The right model depends on the purpose, and the first question a communicator should ask is what the engagement is actually for. Dialogue has its own failure modes, and they are worth knowing because they can damage trust as badly as a patronising lecture. The United Kingdom's 2003 national public debate on genetically modified crops, run under the title GM Nation?, was an ambitious attempt to put dialogue into practice at scale, with regional meetings, local events and published findings. A detailed independent evaluation by Tom Horlick-Jones and colleagues, published in 2007 as The GM Debate: Risk, Politics and Public Engagement, found real strengths but also problems: the open meetings tended to attract people who already held strong views, the timetable was compressed, and it was unclear to participants how their input would affect decisions. The general lesson has been repeated in many settings since. Participants who suspect that a consultation is a ritual, with the decision already made, conclude that they were being managed rather than heard. If you invite people into dialogue, be clear at the outset about what is genuinely open to influence and what is not, and report back afterwards on what happened to their contributions. There is also a quieter form of deficit thinking that survives inside dialogue formats. A "café scientifique" or a public question-and-answer session can look like two-way exchange while functioning as a lecture with a longer queue at the end. The test is whether anything the audience says can change what the researcher thinks, does or says next time. If the honest answer is no, the event is still worth running, but it should be designed and described as what it is. Finding out what people already believe If the facts do not simply pour into empty vessels, what does happen to them? The most practically useful answer comes from risk communication research, and in particular from the "mental models" approach developed at Carnegie Mellon by M. Granger Morgan, Baruch Fischhoff, Ann Bostrom and Cynthia Atman and set out in their 2002 book Risk Communication: A Mental Models Approach. The premise is that people do not lack beliefs about technical subjects; they have beliefs, often coherent and reasonable on their own terms, that differ from expert understanding in specific and discoverable ways. Communication fails when it ignores those beliefs, because new information is interpreted through them. The method has a clear sequence: Build an expert model of the problem: a structured representation, often an influence diagram, of the processes and factors that matter for the decisions people face. Conduct open-ended interviews with members of the intended audience, beginning with very general prompts ("Tell me about radon") and only gradually narrowing, so that their own framings emerge rather than the interviewer's. Compare the lay and expert models to identify gaps, misconceptions and, importantly, correct beliefs that the communication can build upon. Use a structured questionnaire with a larger sample to estimate how common each belief is. Draft a communication that targets the consequential gaps, then test it with members of the audience and revise. In their early work on household radon, the Carnegie Mellon group found that many people conflated radon with other forms of radiation and contamination and believed, for example, that a house with a radon problem would be permanently contaminated. That belief matters for behaviour: if contamination is permanent, testing seems pointless and frightening, whereas in reality radon levels can be reduced by relatively simple ventilation measures. A generic leaflet explaining what radon is and where it comes from would have left that misconception untouched. A leaflet designed after the interviews could address it directly. Fischhoff later summarised the broader programme in a 2013 paper in the Proceedings of the National Academy of Sciences, "The sciences of science communication". He described four tasks: identify the science most relevant to the decisions people face; determine what people already know; design communications to fill the critical gaps; and evaluate whether the communication is adequate, then repeat. The emphasis on decisions is essential. Scientists tend to communicate what they find most interesting or most novel. Audiences need what bears on the choices in front of them. Those are often not the same thing, and the difference explains a great deal of wasted effort. Most researchers will not run a formal mental models study before every talk, but the logic scales down. Before preparing any significant piece of public communication, it is worth answering five questions in writing: • Who, specifically, is the audience? "The general public" is not an answer; "parents of primary-school children in a district considering a new school air-quality policy" is. • What decision, if any, do they face, and on what timescale? • What do they probably already believe about the topic, including things that are correct, partly correct and mistaken? How could you find out cheaply, for instance by reading comment threads, talking to a teacher, or asking five people? • What do they value, and which of those values does the topic touch? • What reasons might they have to trust or distrust you, your institution or your field? Suppose, as a hypothetical example, that a team of atmospheric chemists has found that levels of fine particulate matter outside several schools peak during drop-off and pick-up times, largely because of idling vehicles. Working through the five questions, they might realise that their primary audience is not "the public" but parents and head teachers; that the decision in question is whether to support a car-free zone around the school gates; that many parents probably believe outdoor air is healthier than indoor air and that their own car contributes little; that they value their children's health, convenience and not being judged; and that a university team telling them to change their behaviour may be received as condescending. Each answer reshapes the communication. The team might lead with the timing finding rather than with average pollution levels, explain why a single car contributes more at close range than parents expect, and involve a local head teacher as a co-presenter rather than arriving as outside experts. Trust is the medium, not the message The third lesson of the post-deficit literature is that the source matters as much as the content. People rarely evaluate scientific claims directly; they cannot check the data themselves and would not have time if they could. Instead, they make judgements about whether the person or institution offering the claim is trustworthy, and they accept or reject the claim largely on that basis. This is not irrational. It is how everyone, including scientists outside their own specialisms, handles the division of cognitive labour in a complex society. Research on what makes an expert seem trustworthy has converged on a small set of dimensions. Friederike Hendriks, Dorothe Kienhues and Rainer Bromme developed an instrument, published in 2015 in PLOS ONE, that measures perceived expertise, integrity and benevolence. The distinction is useful. Expertise is whether you know what you are talking about. Integrity is whether you are honest and follow the rules of your profession. Benevolence is whether you have the audience's interests at heart. Susan Fiske and Cydney Dupree, writing in PNAS in 2014, reported survey evidence that Americans tend to see scientists as highly competent but only moderately warm, and suggested that communicators could earn trust by showing that they share the audience's goals rather than by displaying more credentials. The practical implications are substantial. Researchers typically work hard to establish expertise, which audiences largely grant already, and neglect integrity and benevolence, which are often where the doubt lies. Integrity is signalled by acknowledging uncertainty, disclosing funding and conflicts of interest, admitting when earlier views turned out to be wrong, and refusing to overstate. Benevolence is signalled by showing that you understand why the question matters to the people asking it, by listening before explaining, and by not treating disagreement as stupidity. A simple way to apply this is to audit a draft talk, article or briefing against the three dimensions before delivering it. Take the hypothetical air-quality team again. Their draft opens with the team's credentials and funding, spends most of its length on measurement methods and ends with a recommendation. An audit would show that expertise is covered several times over, integrity is barely addressed (the draft never mentions what the measurements cannot show, such as whether the peaks are large enough to affect health over a school year), and benevolence is absent (nothing acknowledges that the school run is hard, that many parents have no realistic alternative to driving, or that the team is not there to assign blame). Adding two honest sentences about limitations and one about the practical difficulty of the school run costs almost nothing in time and changes how the whole message is received. There is a further point about trust that researchers often miss: it is not only earned or lost by individuals. When a university press office hypes a modest finding, when a funder's campaign oversells a field's promise, when one scientist makes a confident prediction that fails, the cost is borne partly by everyone else in the field. Public trust in science is a commons. The BSE episode that prompted the Lords report was damaging not because officials lacked facts but because they had projected a confidence the evidence did not warrant, and when that confidence was shown to be misplaced, the damage spread to institutions well beyond the ministry involved. Each chapter that follows returns to this theme in a different register, because every technique in the communicator's kit, from narrative to framing to plain language, can either build this commons or deplete it. The shift from deficit thinking is therefore not a matter of abandoning facts. Facts remain the substance of what scientists have to offer. It is a matter of recognising that facts are received by particular people, with particular beliefs, facing particular decisions, who extend or withhold trust for particular reasons. A communicator who takes that seriously will spend more time listening and less time broadcasting, will choose different starting points, and will measure success by whether the audience is better able to understand and decide, not by whether it agrees. The next step is to give those facts a shape that human minds are built to follow. Chapter 2: The Shape of a Story: Narrative Structure for Research Ask a researcher to describe a project to a friend and you will often hear something like this: "We're looking at soil microbes in grassland plots, and we sampled at four depths, and we sequenced the communities, and we also measured nitrogen, and we found some interesting differences between the grazed and ungrazed plots, and we're writing it up now." Every clause is true. None of it is memorable, and the listener has no idea why any of it matters. The problem is not vocabulary or length. It is structure. The description is a list, and human beings are poor at remembering lists and good at remembering stories. Randy Olson, a marine biologist who left a tenured position to become a filmmaker, has spent much of the past two decades trying to persuade scientists of this. His first book on the subject, Don't Be Such a Scientist (2009), was a polemic about the cultural gap between science and mass communication. His second, Houston, We Have a Narrative (2015), was more practical: an attempt to give scientists a simple tool for building narrative structure into their communication. That tool, the ABT template, is the most useful single device in this booklet, and this chapter begins with it before turning to the research on why narrative works and where it becomes dangerous. And, But, Therefore Olson credits the core idea to an unlikely source: the creators of the animated series South Park, Trey Parker and Matt Stone, who described in a widely circulated classroom appearance how they revise scripts by replacing the word "and" between story beats with "but" or "therefore". A sequence of events joined by "and then" is dull; a sequence joined by "but" and "therefore" creates tension and consequence. Olson generalised this into a three-part template: We know that [setup] AND [further setup], BUT [problem, contradiction or gap], THEREFORE [what we did or what follows]. The "and" section establishes the context and agreement: the things the audience accepts as true. The "but" introduces a problem, a contradiction, a surprise or a gap in knowledge. That word is where the story begins, because it creates a question in the listener's mind. The "therefore" resolves the tension, at least partially, by describing what was done, what was found or what needs to happen next. Apply it to the soil-microbe description: Grasslands store a large share of the world's soil carbon, and grazing by livestock is the most common use of grassland. But we know surprisingly little about how grazing changes the microbial communities that control whether that carbon stays in the ground. Therefore we compared microbes in grazed and ungrazed plots at four soil depths, and found that the biggest differences were not at the surface, where most studies sample, but deeper down. Every fact in the original is still present or implied. What has changed is that the listener now knows why the work exists and what question it answers, and the finding lands as a resolution rather than as one more item on a list. The template also exposes weaknesses: if you cannot write a convincing "but", you may not yet know what your project's question really is. Olson makes this point forcefully. The ABT is not only a presentation device; it is a diagnostic for clarity of thought. Olson places the ABT in the middle of what he calls a narrative spectrum. At one end is "AAA", "and, and, and": pure accumulation of facts with no tension. This is the default mode of much scientific writing and almost all poorly prepared talks. At the other end is what he calls "DHY", "despite, however, yet": a structure with so many reversals and qualifications that the listener loses the thread. Many discussion sections live here. The ABT sits in between, with one clear problem and one clear response. For a lay audience, one "but" is usually enough. Two or three can work in a long-form piece, provided each is resolved before the next is introduced. Olson also proposes a scalable exercise he calls the Word-Sentence-Paragraph method. You first try to reduce your project to a single word that captures its central theme (for the grassland study, perhaps "depth"). Then a single sentence, often in the "nothing in X makes sense except in the light of Y" form borrowed from Theodosius Dobzhansky's famous 1973 essay title "Nothing in Biology Makes Sense Except in the Light of Evolution". Then a paragraph in ABT form. The point of the progression is to force prioritisation. Researchers usually know far more about their project than they can communicate; the discipline lies in choosing the one thing that everything else serves. The ABT has limitations, and Olson himself is clear about them. It is a structure for the core of a message, not a complete communication plan. It does not tell you which audience to address, which frame to choose or which words to use. It can be applied mechanically, producing a string of fake tensions ("Cells are important, but we don't know everything about them"). A good "but" is specific and genuine: it names an actual gap, contradiction or surprise that the audience can feel. Compare two versions for a hypothetical study of sleep in teenagers. Weak: "Sleep is important for health, but there is still much we don't know about it." Strong: "Teenagers need more sleep than adults, and most secondary schools start earlier than primary schools, but no one had measured what happens to exam performance when a school moves its start time later." The first "but" could be attached to any topic in biology; the second creates a specific question that the audience immediately wants answered, and it sets up a "therefore" that can report the actual study design and result. A useful test is to read your "but" aloud to someone outside your field and ask whether they can guess what the "therefore" will be about. If they can, the tension is real. And the "therefore" must be honest. If the study did not resolve the problem, the "therefore" should say what it did achieve and what remains open. There is also some evidence that narrative structure matters even inside the academy. In a 2016 study in PLOS ONE, Ann Hillier, Ryan Kelly and Terrie Klinger analysed the abstracts of several hundred climate-change papers and rated them for narrative features such as a sense of setting, a clear conflict or problem, and a resolution. Papers whose abstracts had more narrative characteristics tended to be cited more often, even after accounting for other factors they could measure. The study is correlational and cannot show that narrative writing causes more citations, but it undermines the assumption that narrative is only for outsiders. Why narrative works Michael Dahlstrom's 2014 review in PNAS, "Using narratives and storytelling to communicate science with nonexpert audiences", is the best short introduction to the research on why stories are so effective. He draws on a distinction made by the psychologist Jerome Bruner in the 1980s between two modes of thought. The paradigmatic or logical-scientific mode seeks general truths through argument, evidence and abstraction. The narrative mode deals in particular experiences, arranged in time, with characters whose actions have causes and consequences. Science is built in the first mode. Most people, most of the time, think in the second. Dahlstrom summarises several consistent findings. Narratives tend to be easier to understand than expository text for non-expert readers, and they are recalled better. They generate more engagement and interest. And they can be more persuasive, partly because of a phenomenon that Melanie Green and Timothy Brock called "transportation" in a 2000 paper in the Journal of Personality and Social Psychology. When people are absorbed into a story, they become less likely to generate counterarguments and more likely to adopt beliefs consistent with the story, even when they know it is fiction. That last property explains both the power and the danger of narrative. On the positive side, a well-built story can carry an audience through material they would never read as an argument. On the negative side, a story can persuade independently of its evidential weight. A single vivid case can outweigh statistical evidence covering thousands. A striking illustration comes from the psychology of vaccine decisions. In a 2011 experiment published in Medical Decision Making, Cornelia Betsch and colleagues exposed participants to a simulated online forum about vaccination in which the base-rate statistics on adverse events were held constant but the proportion of personal stories describing harm was varied. Participants who read a higher proportion of such stories perceived vaccination as riskier and reported lower intention to vaccinate, even though the statistical information they saw was the same. The stories did work that the numbers did not, and in this case the work was misleading. The related "identifiable victim effect" is well documented. In a 2007 study, Deborah Small, George Loewenstein and Paul Slovic found that people donated more to help a single identified child than in response to statistics about millions in need, and that adding the statistics to the individual story actually reduced donations. Scientists often complain that the public responds to anecdotes rather than data. This research suggests that the complaint describes a feature of human cognition rather than a failing of particular audiences, and that communicators who refuse to use stories are ceding the ground to those who will. The ethics of telling science as a story If narrative is this powerful, a researcher who uses it takes on responsibilities. Dahlstrom and Shirley Ho addressed this directly in a 2012 paper in Science Communication, "Ethical considerations of using narrative to communicate science". They framed the issue around three questions that every communicator should answer before building a story. The first is the purpose of the communication. Is the aim to help people understand something, or to persuade them to believe or do something? Both can be legitimate. A public health agency trying to increase vaccination rates is engaged in persuasion; a researcher explaining how a vaccine was tested is engaged in explanation. But the ethical standards differ. Persuasion by narrative, which works in part by reducing counterarguing, deserves particular scrutiny, and it is more easily justified when the evidence is strong and the behaviour in question benefits the audience than when the evidence is contested or the benefit accrues to someone else. The second is accuracy, and here narrative raises specific problems. A story has a representativeness problem: the individual case at its centre may or may not be typical. A patient whose tumour vanished on a new drug makes a compelling story, but if the response rate in the trial was modest, the story conveys a false impression unless it is explicitly placed in context. Stories also impose causal clarity on events that may not have been so clear. Narrative's natural grammar is "this happened, therefore that happened", and it will tend to make correlations feel like causes. The communicator's job is to make sure that the "therefore" in the story is one the evidence supports. The third is whether narrative should be used at all. Dahlstrom and Ho note that some scientists regard narrative as inherently at odds with scientific norms, because it privileges the particular over the general. Their answer, and the one this booklet adopts, is that avoiding narrative does not produce neutral communication. It simply produces communication that audiences are less likely to understand or remember, leaving the field to others who may have fewer scruples. The better response is to use narrative with discipline. In practice, that discipline comes down to a short set of rules: • Make the story typical, or say that it is not. If you use an individual case, choose one that represents the central tendency of the evidence, or explicitly state how it differs. • Pair the story with the numbers. The story gives the audience a reason to care; the numbers tell them how much to care. Research on the identifiable victim effect suggests the combination may reduce emotional impact, but in science communication, accuracy is the constraint and impact is the objective, not the other way round. • Do not invent. Composite characters and reconstructed scenes are common in journalism and documentary, but they must be labelled. A researcher who presents a composite patient as a real one, or a hypothetical scenario as a real event, has crossed a line that damages trust when discovered. • Let the "therefore" match the evidence. If the study showed an association, the story's resolution should not imply that one thing caused the other. If the finding needs replication, the story should end with an open question, not a triumph. • Watch the villain. Stories need conflict, and it is tempting to supply it with an antagonist: an industry, a regulator, a rival theory. Sometimes that is accurate. Often it is not, and it can turn an explanatory message into a partisan one, with consequences taken up in the next chapter. Stories at different scales The ABT works at the level of a paragraph or a thirty-second pitch. Longer formats need more architecture. Three structures are useful to know. The first is the quest, which follows a researcher or team pursuing a question through obstacles to an answer. It suits profiles, documentaries and long-form features, and it is the structure most people imagine when they think of "science stories". Its strength is that it makes the process of science visible, including dead ends and uncertainty. Its weakness is that it foregrounds the scientist, which can feel self-promotional and can distort the collective nature of most research. The second is the mystery, which begins with a puzzling observation and works towards an explanation. This is often the most natural fit for scientific material, because much research really does begin with something that does not make sense. A talk that opens, in a hypothetical example, "Three summers ago, fishermen along one stretch of coast started pulling up species that should not have been there" creates immediate engagement, and the rest of the talk can unfold as the solution of the puzzle. The mystery structure has the virtue of mirroring how science actually works: observation, hypothesis, test, revision. The third is the problem and response, which begins with something that matters to the audience, explains its causes, and ends with what can be done. This is the natural structure for policy communication and public health messages, and it is essentially an extended ABT. Its risk is that the "response" may carry more certainty than the evidence warrants, especially if the communicator is also an advocate for a particular solution. Within any of these structures, a few narrative elements consistently repay attention. Characters give the audience someone to follow, and they need not be human: an organism, a molecule, a glacier or a dataset can serve if it is given a trajectory. Specific details, a place, a date, an object, anchor abstractions. Stakes tell the audience why the outcome matters. And a clear ending, even if it is an open question, gives the audience something to carry away. Consider how these elements might transform a hypothetical five-minute talk on antibiotic resistance for a school audience. The expository version would define antibiotics, explain the mechanism of resistance, present surveillance data and conclude with recommendations. A narrative version might follow a single bacterium through a course of antibiotics that the patient stops taking early, showing how the survivors are the ones that happen to carry resistance genes, and how they pass those genes on. The mechanism is the same; the audience now has a character to follow and a moment of consequence to remember. The surveillance data can then appear as the answer to a question the story has already raised: how often does this happen? Narrative, in short, is not an alternative to evidence. It is the vehicle that carries evidence into minds that would otherwise let it slide past. The communicator's craft lies in building a vehicle that holds attention without distorting the load. The next question is which part of the load to put at the front, and that is the problem of framing. Chapter 3: Framing: Choosing the Lens Without Distorting the Picture In 1981 Amos Tversky and Daniel Kahneman published a short paper in Science that has become one of the most cited in the social sciences. They asked participants to imagine that the United States was preparing for the outbreak of an unusual disease expected to kill 600 people, and to choose between two programmes. In one version of the question, participants were told that Programme A would save 200 people, while Programme B had a one-third probability of saving all 600 and a two-thirds probability of saving no one. Most chose A. In another version, the same options were described in terms of deaths: under Programme C, 400 people would die; under Programme D, there was a one-third probability that nobody would die and a two-thirds probability that 600 would die. Most chose D. The options were mathematically identical. Only the description had changed, from lives saved to lives lost, and preferences reversed. This is framing in its narrowest sense, sometimes called an equivalence frame: logically equivalent descriptions that produce different judgements. Scientists who communicate about risk encounter it constantly. A treatment that gives a 90 per cent survival rate sounds better than one with a 10 per cent mortality rate, although they are the same treatment. Health communicators have spent decades trying to turn the equivalence effect into practical guidance. In an influential 1997 paper in Psychological Bulletin, Alexander Rothman and Peter Salovey proposed that gain-framed messages ("exercising keeps your heart strong") should work better for encouraging prevention behaviours, while loss-framed messages ("failing to get screened can mean a cancer is found late") should work better for detection behaviours such as screening, because detection involves accepting the risk of an unwelcome finding. The idea was elegant, and it shaped a great deal of public health messaging. Subsequent meta-analyses by Daniel O'Keefe and Jakob Jensen, however, found that the differences in persuasiveness between gain and loss frames were generally small and not consistent across behaviours. The honest lesson for researchers is modest: the way a number is described matters, audiences should ideally be given both forms (survival and mortality, benefit and harm), and no single wording trick should be relied on to do the work of a clear explanation. The ethical point is sharper than the persuasive one. Because equivalence frames can shift choices without changing any facts, a communicator who presents only the favourable form of a statistic, survival without mortality or relative risk reduction without absolute risk, is steering the audience rather than informing it. Chapter 4 returns to this in detail. But most of what communication researchers mean by framing is broader, and more consequential, than this. What a frame does The political scientist Robert Entman offered a definition in 1993 that is still widely used: "To frame is to select some aspects of a perceived reality and make them more salient in a communicating text, in such a way as to promote a particular problem definition, causal interpretation, moral evaluation, and/or treatment recommendation." These are sometimes called emphasis frames. They do not describe the same thing in equivalent ways; they choose which of many true things about an issue to put in the foreground. Every scientific finding of public interest has multiple aspects. Consider research on gene drives, a genetic technique that can spread a trait through a wild population faster than normal inheritance would allow. It can be described as a potential tool for eliminating malaria-carrying mosquitoes, as an ecological intervention with uncertain downstream effects, as a question of who gets to decide whether a technology is released in a particular country, as a frontier of scientific competition, or as an example of humans altering nature in a new way. All of these descriptions are accurate. Each emphasises a different aspect, invites a different interpretation of what the issue is about, and appeals to different values. The communicator cannot avoid choosing among them, because no message can say everything. Even a deliberately neutral description, with no evaluative language, embeds a frame through what it mentions first and what it leaves out. The idea that scientists should think consciously about framing entered wide debate through a 2007 Science article by Matthew Nisbet and Chris Mooney, "Framing Science". They argued that scientists often assume the facts speak for themselves, and that this assumption leaves the framing of scientific issues to others, especially to those with strong interests in the outcome. Their article provoked a lively response. Some scientists wrote that framing sounded like spin, and that scientists should simply present the evidence accurately and let the public decide. Nisbet and Mooney's reply, and the position that most communication scholars now take, is that this objection misunderstands the choice available. Presenting "just the facts" involves selecting which facts, in what order, with what emphasis. The question is not whether to frame but whether to frame deliberately and honestly. Drawing on earlier work by William Gamson and Andre Modigliani and by John Durant and colleagues, Nisbet identified a set of frames that recur across science-related policy debates in many countries and topics. Table 2 presents a version of this typology, with a hypothetical illustration for each frame from the gene-drive example. Table 2. Recurring frames in public debates about science, with gene drives as an illustration. Frame Defines the issue as Hypothetical gene-drive illustration Social progress Improving quality of life or solving problems A tool that could help eliminate malaria Economic development Investment, competitiveness, jobs A field where national research capacity is at stake Morality and ethics Right and wrong, limits we should respect Whether humans should permanently alter wild species Scientific uncertainty What is known and not known Unknown ecological effects of removing a species Pandora's box Technology out of control, irreversible harm A release that cannot be recalled Public accountability Who decides and in whose interest Whether affected communities consented to field trials Middle way Compromise between polarised options Phased, contained trials with community oversight Conflict and strategy A game among elites; who is winning Rival research groups and funders competing Several observations follow from the table. First, the frames are not mutually exclusive, and serious communication usually combines several. Second, the frames carry different implications for what should be done: a social progress frame implies urgency, a Pandora's box frame implies caution, a public accountability frame implies consultation. Third, and most important for researchers, the frames resonate differently with different audiences. A community in a malaria-endemic region, a European environmental group, a national funding agency and an ethics committee will each find some frames more relevant than others. Evidence that frames change who listens The strongest evidence for the value of deliberate framing comes from studies showing that a change of frame can shift how particular audiences respond to the same underlying science. Climate change has been the main testing ground. In a 2010 study published in BMC Public Health, Edward Maibach, Matthew Nisbet and colleagues tested how different segments of the American public responded to a description of climate change framed as a public health issue, compared with environmental and national security framings. They found that the public health frame tended to elicit more hopeful and engaged reactions across segments, including among some audiences that were dismissive of climate change as an environmental problem. The study was exploratory, and later research has produced mixed results about how large and durable such effects are, but it helped establish public health as one of the standard frames in climate communication. A different line of research has examined how frames interact with moral values. Matthew Feinberg and Robb Willer, in a 2013 paper in Psychological Science, drew on moral foundations theory, which proposes that liberals and conservatives in the United States tend to weight different moral intuitions. Environmental messages typically emphasise harm and care, foundations that liberals prioritise. Feinberg and Willer found that reframing pro-environmental messages in terms of purity and sanctity, emphasising pollution as contamination of something pure, increased pro-environmental attitudes among conservatives and substantially narrowed the gap between the two groups. The finding suggests that part of what looks like resistance to science is a mismatch between the values the message invokes and the values the audience holds. Dan Kahan and colleagues reported a related effect in a 2015 study in The Annals of the American Academy of Political and Social Science. They found that reading about geoengineering, the deliberate large-scale intervention in the climate system, as part of a story about climate science made some participants who were otherwise sceptical more open to the evidence on climate change itself. Their interpretation was that emphasising technological responses, which are compatible with the values of people who favour markets and innovation, reduced the identity threat that climate evidence ordinarily carried for them. Whatever one thinks of geoengineering as policy, the study illustrates the principle: the frame around a fact can determine whether an audience engages with the fact at all. The messenger is part of the frame. A frame that emphasises religious stewardship of creation will be received differently if it comes from a researcher who shares the audience's faith than from one who does not. The climate scientist Katharine Hayhoe, an evangelical Christian, has for many years spoken to faith communities about climate change in terms of shared values, and her work is often cited as an example of the importance of messenger-audience fit. For most researchers, the practical lesson is not to adopt identities they do not hold, which would be dishonest and quickly detected, but to recognise when a trusted intermediary, a local doctor, a farmer, a faith leader or a teacher, would be a more credible messenger than a university scientist, and to support those intermediaries rather than competing with them. Where framing becomes distortion If framing is unavoidable and can make communication more effective, where are its limits? This is not an abstract concern. Framing techniques are used every day by lobbyists, campaigners and political strategists whose goal is to win, not to inform, and the same techniques can be turned towards conclusions the evidence does not support. A workable line can be drawn with three tests. The first is accuracy of emphasis. An emphasis frame selects among true aspects of an issue. It becomes distortion when it implies something false about the overall balance of evidence. Framing gene drives primarily around malaria elimination is legitimate if the communicator is honest about how far the technology is from deployment and what uncertainties remain. It becomes misleading if it implies that elimination is imminent or assured. The frame may highlight; it may not misrepresent. The second is consistency across audiences. It is legitimate to emphasise the health aspects of air pollution to a health audience and the economic aspects to a business audience. It is not legitimate to tell one audience that a risk is serious and another that it is negligible. A useful check is to ask: if these two audiences compared notes, would they feel deceived? If the answer is yes, the framing has crossed a line. The third is respect for the audience's decision. Framing to help people understand how an issue connects to their own values is a service. Framing to exploit a cognitive bias so that people reach a conclusion they would reject on reflection is manipulation. The difference is often visible in intent: is the communicator trying to make the evidence meaningful, or trying to prevent the audience from thinking about it carefully? Researchers face one framing temptation more than any other: the social progress frame applied to their own work. Grant applications, press releases and institutional communications all reward describing research as a step towards a cure, a solution or a breakthrough. Each individual instance may be defensible; the cumulative effect is a public that hears about cures that never arrive and begins to discount everything scientists say. Chapter 6 examines research on how exaggeration in press releases passes into news coverage. The framing lesson is that the social progress frame is the one most likely to generate cynicism when overused, and researchers should reach for it with particular care. There is also a risk in adopting the conflict frame, which journalists find attractive because conflict is newsworthy. A researcher who is drawn into describing a debate as a battle between their side and another, whether a rival scientific camp, an industry or a political party, may gain attention in the short term but will tend to push the issue towards polarisation, where, as Chapter 1 noted, facts do less work. When a journalist asks "Who is winning this debate?", a better response usually redirects to what the evidence shows and what remains uncertain. Choosing a frame in practice Suppose, as a hypothetical example, that a research team has completed a study showing that a common agricultural fungicide reduces the foraging efficiency of wild bumblebees at concentrations found in field margins. The team is invited to present the findings at three events in the same month: a regional farmers' association meeting, a public lecture at a natural history museum, and a briefing for officials at the national agricultural ministry. A deficit-model approach would prepare one talk and deliver it three times. A framing-aware approach begins with the audience analysis described in Chapter 1 and then asks which true aspects of the finding are most relevant to each. For the farmers, the natural frame is economic development combined with middle way. Many crops depend on pollination, and farmers have direct economic interests in healthy pollinator populations. The team might lead with the dependence of certain local crops on wild bumblebees, present the finding as a risk to a service farmers rely on, and discuss practical options, such as timing of application or buffer strips, that reduce exposure without abandoning the fungicide. Leading with a Pandora's box frame ("this chemical is harming nature") would be accurate in part but would likely trigger defensiveness and lose the room. For the museum audience, social progress in the sense of understanding, and scientific uncertainty, may be more engaging: how the team measured foraging, what surprised them, what they still do not know about how the effect scales to whole populations. This audience wants a story of discovery, and the mystery structure from the previous chapter fits well. For the ministry, public accountability and scientific uncertainty are central. Officials need to know how strong the evidence is, how it relates to the existing regulatory assessment, what further evidence would change the picture, and what the range of policy options looks like. As Chapter 5 will discuss, this is where the researcher also has to decide what role to play. How would the team know whether its framing choices were right? Large organisations test frames with surveys and focus groups; a research team rarely has that luxury, but it can do a scaled-down version. A draft of the farmers' talk could be shown to one or two agronomists or extension officers who work with farmers daily, with a simple request: what would make this audience stop listening? The museum version could be tried on a few friends outside science. The ministry briefing could be read by a former civil servant or a colleague who has sat on an advisory committee. None of this is rigorous research, but it catches the most common failure, which is a frame that seems natural to the researcher and alienating to the audience. It also often reveals frames the team had not considered: an agronomist might point out, for instance, that local farmers are already worried about declining yields in a particular crop and would be more receptive if the talk connected the finding to that concern. In all three settings, the core findings are identical, the limitations are stated, and none of the audiences would feel deceived if they compared notes. What differs is the entry point and the emphasis. That is framing done responsibly: not changing the evidence to suit the audience, but finding the door through which each audience can walk into it. Hashtags: #ScientificStorytelling #PublicEngagement #ScienceCommunication #PublicUnderstandingOfScience #DeficitModel #DialogueModel #ParticipatoryEngagement #AudienceAnalysis #MentalModels #TrustInScience #RiskCommunication #NarrativeCommunication #ABTStorytelling #ScienceFraming #EvidenceBasedCommunication #CommunicatingUncertainty #SciencePolicyCommunication #MediaEngagement #PressReleases #VideoAbstracts #SocialMediaScience #MisinformationResponse #PublicTrust #ResearchCommunication #FutureOfScienceEngagement

Download the Book (PDF): Introduction A graduate student in ecology wants to know whether a population of salamanders in a single mountain stream carries a fungal pathogen that has devastated amphibians elsewhere. The question is sharp, the fieldwork would take a month, and the laboratory work would cost a few thousand dollars in swabs, reagents and sequencing. No federal agency will fund it. The study is too small to be worth a panel's time, too preliminary to have the pilot data a panel expects, and too uncertain to guarantee a publishable result. Her adviser's grant covers other work. Her department's travel fund covers conferences, not field seasons. The question will go unasked unless she finds the money somewhere else. Over the past fifteen years, a growing number of researchers in that position have found it in the same place: among strangers, friends, former teachers, amateur naturalists, patients, retirees and the idly curious, who give ten, fifty or five hundred dollars through a crowdfunding page because they want the question answered. Dedicated platforms now exist for exactly this purpose. The largest of them, Experiment, reports that roughly half the campaigns it hosts reach their targets, and the typical target is a few thousand dollars. Larger efforts have raised far more. The American Gut Project, a citizen-science study of the human microbiome run from the University of California, San Diego, drew in more than a million dollars from participants who paid to have their own samples sequenced and contributed to a shared, open dataset. This book is about how to do that well, and about why doing it well is more demanding than it first appears. The argument of this book The usual way of describing scientific crowdfunding treats it as a small grant from an unusual source. You write a short proposal, record a video, set a target, and if enough people give you money, you have a grant. The difference from an agency grant, on this view, is mainly one of scale and reviewer: a crowd of laypeople instead of a panel of experts. That description is wrong in a way that causes real harm. A crowdfunded research project is not a grant. It is a public promise made to a community that the researcher has, in large part, assembled. The money arrives attached to people, and the people arrive with expectations: that they will be told what happens, that the work will actually be done, that the results will be shared whether or not they are exciting, and that the researcher is the kind of person they believed when they clicked the button. None of those expectations is written into a contract. All of them are real. A researcher who treats the campaign as a funding transaction and then disappears into the laboratory will, sooner or later, discover that the community she built can turn against her, and that the damage extends to every scientist who tries the model after her. The controlling idea of this book follows from that. Crowdfunding works for science, and remains honest, when the whole project is designed around the relationship with backers from the beginning: the choice of question, the budget, the pitch, the timeline and the eventual publication all have to be built for an audience that is paying to watch. Researchers who design this way find that the discipline improves their science. They scope questions more tightly, budget more carefully, state their risks more plainly and publish their results more openly than they might under a conventional grant. Researchers who do not design this way tend to fail twice: first at raising the money, and then, if they succeed, at keeping faith with the people who gave it. What crowdfunding is good for It is worth being clear from the start about the kind of research this model serves. Crowdfunding is well suited to small, well-bounded, high-risk exploratory questions: a pilot study, a field season, a sequencing run, a prototype instrument, a survey, a first attempt at a method nobody has tried. It suits early-career researchers without track records, independent researchers without institutions, researchers in unfashionable fields, and researchers whose questions cross disciplinary lines that grant panels police. It suits work with a story that ordinary people can follow and care about. It is poorly suited to large, capital-intensive projects, to anything that requires years of guaranteed salary, to clinical trials of treatments, and to work whose value can be explained only to specialists. It is dangerous when it is used to promise outcomes that research cannot promise, such as cures, products or guaranteed discoveries. Much of what follows is about recognizing which side of that line a given idea sits on, and reshaping ideas so that they land on the right side. How the book is organized The chapters follow the life of a campaign, from the decision to try it to the moment the last backer receives the final report. Chapter 1 explains why crowdfunding for research exists at all: the structural reasons conventional funding systems struggle to support small exploratory work, the short history of research-specific platforms, and what the published evidence says about which campaigns succeed. Chapter 2 is about choosing and shaping the project itself, which is where most campaigns are won or lost long before launch. Chapter 3 surveys the platforms, funding models and rules, including the institutional and legal questions that a researcher inside a university and an independent researcher will answer differently. Chapter 4 is about money: building a campaign budget that accounts for fees, rewards, taxes and contingency, and setting a target that is both honest and achievable. Chapter 5 is about the audience, which the evidence suggests matters more than anything else, and about the weeks of preparation that precede a good launch. Chapter 6 covers the pitch video and the campaign page, the two objects most prospective backers will actually see. Chapter 7 turns to what happens after the money arrives: updates, delays, failures, negative results and the management of expectations over a project that may take longer than anyone planned. Chapter 8 addresses the ethical duties specific to research funded by the public, including the oversight of work with human participants or animals, the handling of data, the honest reporting of results and the obligations of researchers who work outside institutions. The conclusion draws out what all of this implies for researchers, for institutions and for the wider relationship between science and the public that pays for it. Throughout, the book draws on the small but real body of research on scientific crowdfunding, on well-documented public cases, and on the practical experience that has accumulated since the first research campaigns in the early 2010s. Where the evidence is thin, it says so. Where a practice is a matter of judgment rather than data, it says that too. A word on scale Nothing in this book assumes that crowdfunding will replace public research funding, and the argument does not depend on it doing so. The sums involved are small. A successful campaign typically covers a few thousand dollars of direct costs, not salaries, not overheads, not a laboratory. The point is not that the crowd can fund science in general. The point is that there is a specific, valuable kind of research that the established system funds badly, and a specific, well-run kind of campaign that can fund it well, while opening the work to people who would otherwise never see how research is done. That second benefit is not a side effect. For many researchers who have run campaigns, the most lasting result was not the money but the audience: a few hundred people who now follow the work, ask questions, forward papers to their friends and, sometimes, give again. Treated with care, that audience is one of the most valuable things a researcher can build. Treated carelessly, it becomes a liability. The chapters that follow are about the difference. Chapter 1: Why the Crowd Funds What Panels Will Not Every funding system has a shape, and the shape determines which questions get asked. Before deciding whether to crowdfund a piece of research, it helps to understand why the established system leaves certain kinds of work unfunded, because the gaps it leaves are precisely where crowdfunding does its best work. A researcher who understands the gap can design a campaign to fill it. A researcher who does not will often try to use crowdfunding as a cheaper, easier substitute for a grant, and will be disappointed on both counts. The shape of conventional research funding Most research in wealthy countries is funded through competitive grants awarded by government agencies, foundations and, increasingly, philanthropic organizations. The core mechanism is peer review: a proposal is read by a small number of experts, scored, discussed by a panel, and ranked against other proposals. Money flows to the highest-ranked proposals until the budget runs out. This system has real strengths. It concentrates judgment in people who understand the field, it provides a check against obvious error, and it spreads money across many institutions rather than letting it pool around the politically connected. But it also has predictable biases, and several of them work against small exploratory research. The first is conservatism. Reviewers are asked to judge whether a project will succeed, and the most reliable evidence of future success is past success: preliminary data, a track record in the method, a publication history in the area. A genuinely new idea, by definition, lacks these things. The economist Paula Stephan, in her book How Economics Shapes Science, describes at length how the incentives of grant competition push both applicants and reviewers toward projects that are safe enough to promise results. When funding rates are low, the pressure intensifies. At the United States National Institutes of Health, success rates for the main investigator-initiated research grants have hovered around one application in five in recent years, and reviewers faced with that arithmetic have little room to back a long shot. The second bias is scale. The fixed costs of writing, reviewing and administering a grant are substantial, and they do not shrink much when the grant does. It is uneconomic for an agency to run a full review process for a request of three thousand dollars, and so most agencies do not offer awards that small, or offer them only through narrow schemes. The researcher who needs a few thousand dollars for a field season or a sequencing run falls below the floor of the system. The third bias is eligibility. Most major funders restrict applications to people with particular appointments at particular kinds of institutions. Graduate students usually cannot apply as principal investigators. Independent researchers, community scientists and people between positions usually cannot apply at all. Researchers at small teaching institutions can apply but compete against research universities with dedicated grant offices. The fourth is time. From idea to money, a conventional grant commonly takes the better part of a year: writing, submission deadlines, review cycles, council decisions, award paperwork. For a question tied to a season, an event, an outbreak or a short-lived opportunity, that delay can be fatal. None of this makes grant funding bad. It makes it a system optimized for a particular kind of project: medium to large, built on established foundations, led by credentialed investigators, and able to wait. The kind of project this book is about is the complement of that: small, new, sometimes led by people outside the usual credentials, and often in a hurry. Where crowdfunding fits Crowdfunding inverts several features of the grant system. The reviewers are not experts but interested members of the public, and they are not asked to rank proposals against one another but to decide, individually, whether a particular project is worth some of their own money. There is no formal eligibility requirement beyond what the platform imposes. Campaigns can be prepared in weeks and run for a month. And because each backer gives a small amount, the total can be as small as the project requires. These features make crowdfunding naturally suited to the gaps just described. A graduate student can run a campaign in her own name. An independent researcher can raise money without an institutional affiliation. A project that needs three thousand dollars can ask for three thousand dollars without apology. A question tied to a season can be funded before the season arrives. But the inversion also creates new constraints. Backers are not experts, and they cannot evaluate methodology the way a panel can. What they can evaluate is whether the question interests them, whether the researcher seems competent and honest, and whether the plan seems concrete enough to be carried out. This means that a crowdfunded project must be legible: the question must be explainable to a curious non-specialist, and the connection between the money and the answer must be visible. Research that can only be justified in the technical language of a field, however important, is hard to crowdfund. The absence of expert review also shifts responsibility. When a panel funds a project, the panel shares in the judgment that the project is sound. When a crowd funds a project, the judgment rests almost entirely on the researcher's own representations. This is one reason the ethical demands on crowdfunding researchers are, in some respects, higher than on grant recipients, a theme Chapter 8 develops at length. A short history of research crowdfunding Crowdfunding as a general practice predates the internet: subscription publishing, community barn-raisings and public appeals for monuments all pooled small contributions for a shared purpose. The modern online form took off around 2009, when Kickstarter launched, joining earlier platforms such as Indiegogo. These general-purpose platforms were designed for creative projects and products, and their model was built around rewards: backers gave money and, in return, received the album, the gadget, the book or some token of thanks. Scientists noticed quickly. Some early research campaigns ran on the general platforms, and a handful attracted wide attention. In 2012, the pharmacologist Ethan Perlstein, then a postdoctoral researcher at Princeton, raised roughly twenty-five thousand dollars on the platform RocketHub for a study of how methamphetamine distributes inside neurons, a project that drew media coverage partly because of its provocative subject and partly because it was among the first to show that a basic-science question could attract public money. Perlstein went on to found an independent research company and remained a prominent advocate of alternative funding for science. At around the same time, researchers began organizing collective experiments in science crowdfunding. The #SciFund Challenge, started in 2011 by the ecologists Jarrett Byrnes and Jai Ranganathan, ran a series of coordinated rounds in which groups of scientists launched campaigns at the same time, shared advice and compared results. The organizers treated the rounds as a natural experiment and later published an analysis of what distinguished the campaigns that raised more money from those that raised less. Their central finding, discussed in detail in Chapter 5, was that the size of a researcher's existing audience mattered more than almost anything about the campaign itself. Dedicated research platforms followed. Petridish, launched in 2012, hosted science projects for a few years before closing. Microryza, founded in 2012 by Cindy Wu and Denny Luan, rebranded as Experiment and became the largest platform specifically for research. Other platforms emerged for particular communities: academist in Japan, for example, and Consano, which focused on medical research. Universities began running their own crowdfunding sites, often through their development offices, to let researchers raise money from alumni and local supporters. The same period produced a few larger and more complicated cases. The American Gut Project, launched in 2012 by the microbiologist Rob Knight and colleagues, invited members of the public to contribute money in exchange for having their own microbiome sequenced, and assembled what became one of the largest open human microbiome datasets; its organizers published an overview of the project in the journal mSystems in 2018. Around the same time, a start-up called uBiome raised money on Indiegogo with a similar citizen-science pitch; it later grew into a venture-backed company whose founders were charged by federal authorities in 2021 with securities and health-care fraud over its later billing of insurers and its dealings with investors. The contrast between the two is instructive and is revisited in Chapter 8. And in 2013 the Glowing Plant project raised more than four hundred and eighty thousand dollars on Kickstarter by promising backers seeds of a genetically engineered plant that would glow; the plants never arrived, and the episode led Kickstarter to prohibit genetically modified organisms as rewards. Chapter 7 examines what went wrong. What the evidence says about success Research on scientific crowdfunding is still modest in volume, but a few careful studies give a reasonable picture of what a typical campaign looks like and what distinguishes successful ones. The most comprehensive is an analysis by Henry Sauermann, Chiara Franzoni and Kourosh Shafi, published in PLOS ONE in 2019, of 725 campaigns on Experiment. About 48 percent of campaigns reached their targets. The median target was around three thousand five hundred dollars, and the median amount raised by successful campaigns was a little over three thousand. Around four-fifths of campaign creators were affiliated with educational institutions, and many were students: undergraduates, master's students and doctoral students together made up more than half. Most projects were straightforward research, concentrated in biology, ecology, medicine and engineering. Several features were associated with success. Campaigns with smaller targets were more likely to be funded, as one would expect. Campaigns that posted lab notes, the platform's name for progress updates, and campaigns that included a video, did better. Strikingly, students and postdoctoral researchers had higher success rates than professors, and campaigns led by women had higher success rates than campaigns led by men. The authors were careful not to claim that these associations are causal; many unobserved factors, such as how much effort a creator put into outreach, could explain them. But the pattern is suggestive. The attributes that drive grant success, such as seniority and publication record, did not drive crowdfunding success. Effort, communication and connection to an audience appeared to. This fits the earlier #SciFund analysis. Byrnes, Ranganathan and their colleagues, writing in PLOS ONE in 2014, found that the amount a campaign raised tracked the reach of the researcher's outreach, for example the number of people who visited the campaign page and the size of the researcher's existing online following. A researcher who had spent years talking to the public about her work had, in effect, already done most of the fundraising before the campaign opened. A researcher who had not was starting from nothing. A useful comparison comes from studies of crowdfunding in general. Ethan Mollick, whose 2014 study in the Journal of Business Venturing was one of the first systematic analyses of Kickstarter, found that project quality signals and the size of the founder's social network both predicted success, and that most funded projects delivered what they promised, though often late. A later study of delivery by Mollick, based on a large survey of Kickstarter backers and released in 2015, estimated that around nine percent of successfully funded projects failed to deliver their rewards at all. Research projects are different from product launches in many ways, but the lesson carries over: funding is only the beginning, and delay is the norm rather than the exception. The real currency Put together, the history and the evidence point to a conclusion that shapes the rest of this book. The currency of scientific crowdfunding is not the quality of the proposal as an expert would judge it. It is trust, and trust is built from three things: an audience that already knows the researcher, a project that ordinary people can understand and care about, and a record of honest communication. That conclusion has an uncomfortable implication. Crowdfunding does not level the playing field as completely as its early advocates hoped. It removes some barriers, such as institutional eligibility and seniority, but it introduces others, such as the need for an audience and the ability to communicate. A brilliant researcher who has never spoken to the public, works on a question that cannot be explained in a paragraph and has no network of supporters will struggle. The advantages that crowdfunding confers flow to those who can build a relationship with a community. The implication is also an opportunity. Unlike seniority or institutional prestige, the ability to build an audience can be learned and practiced, and it compounds. A researcher who runs one honest, well-communicated campaign emerges with more backers, more followers and more credibility than she started with, and her second campaign begins from a stronger position. Several researchers who ran early campaigns have described exactly this pattern. The money from the first campaign was modest. The community it built was worth far more. When to consider crowdfunding Before going further, it is worth setting out the conditions under which crowdfunding is a sensible choice. None of them is absolute, but a project that meets most of them is a reasonable candidate. · The project needs a small, well-defined amount of money for direct costs, typically in the range of a few hundred to a few tens of thousands of dollars. · The question is genuinely exploratory: it is worth asking even though the answer is uncertain, and conventional funders are unlikely to support it until someone has taken a first look. · The research can be explained to a curious non-specialist in a few sentences, and the connection between the money and the answer is concrete. · The researcher, or someone on the team, is willing to spend real time on communication: before the campaign, during it and for the whole life of the project. · The research can be carried out within any required ethical and regulatory oversight, and the researcher knows what that oversight is. · The researcher is prepared to share the results publicly, including results that are negative, disappointing or ambiguous. A project that fails several of these tests may still be worth doing, but it is probably not worth crowdfunding. The next chapter is about how to take an idea that almost meets them and shape it so that it does. Chapter 2: Shaping a Project the Crowd Can Back Most crowdfunding campaigns are decided before they launch, and most of that decision happens when the researcher chooses what, exactly, to ask the crowd to fund. A good research idea is not automatically a good crowdfunding project. The same underlying question can be framed as a campaign that is vague, expensive and open-ended, or as one that is specific, affordable and finished within months. The second will raise money more easily, and it will also be far easier to deliver honestly. This chapter is about the craft of turning a research idea into that second kind of project. The unit of funding Grant proposals typically describe a programme of work: several aims, spread over several years, connected by a larger hypothesis. That structure makes sense when a panel is deciding whether to invest in a research direction. It makes little sense for a crowd. A backer is not investing in a research direction. She is paying for a specific piece of work to happen, and she wants to know what that piece of work is, what it will cost, and what she will learn at the end. The right unit of funding for a crowdfunding campaign is therefore not a programme but a step: one discrete, well-bounded activity that produces one identifiable output. Consider the difference between two framings of the same idea. A marine biologist interested in how microplastics affect filter-feeding invertebrates might describe her research as "understanding the impacts of microplastic pollution on coastal ecosystems." That is a fine description of a career. As a crowdfunding pitch, it is hopeless: it has no end, no clear cost, and no obvious output. Alternatively, she might propose to "collect mussels from five sites along a single estuary, from the harbour mouth to the upper reaches, and count the microplastic particles in their tissue, to find out whether contamination falls off with distance from the city." That is a project. It has a place, a method, a sample size, a result that will exist in a few months, and an answer that anyone can understand, whichever way it comes out. The shift from programme to step is the single most important move in designing a crowdfunded project. It has several consequences. It forces the researcher to identify what is genuinely the first thing to do. Many large research ideas contain a small, cheap experiment that would tell the researcher whether the rest is worth pursuing. Crowdfunding is ideal for exactly that experiment. It makes the budget concrete. A step can be costed item by item. A programme can only be estimated. It makes success and failure legible. At the end of a step, the researcher can say what was done and what was found. At the end of a programme, there is always more to do. And it makes the campaign repeatable. A researcher who funds step one through a campaign can report the results to the same backers and, if the results warrant it, launch step two. Several researchers have built sequences of campaigns this way, each funded partly by the backers of the last. The high-risk question Crowdfunding is often described as a way to fund high-risk research, and that is true, but the phrase needs care. There are two very different kinds of risk in research, and they call for different treatment. The first is scientific risk: the possibility that the hypothesis is wrong, that the effect does not exist, that the measurement shows nothing. This kind of risk is the whole point of exploratory research, and it is exactly what conventional funders are reluctant to take on. A crowdfunded project can embrace it openly. Backers can be told, in plain terms, that the answer might be no, and that a no is still worth knowing. Many are delighted to fund a genuine gamble, provided it is described as one. The second is execution risk: the possibility that the researcher cannot carry out the work at all, because the method does not function, the equipment cannot be obtained, the permits are not granted, the samples cannot be collected or the team does not have the skills. This kind of risk is not a feature. A backer who funds an experiment expects the experiment to be done, whatever its result. A project that might never produce any result at all, not because the hypothesis fails but because the work cannot be carried out, is a much weaker candidate for public money. The design goal, then, is to maximize the proportion of risk that is scientific and minimize the proportion that is execution. A project in which the question is uncertain but the method is proven is a good crowdfunding project. A project in which the question is uncertain and the method is also untested is a poor one, unless the untested method is itself the question and success or failure of the method is the result being reported. The Glowing Plant campaign of 2013 is a vivid example of confusing the two. The project's backers were promised seeds of a plant that would glow visibly in the dark. Whether such a plant could be engineered with the tools then available was a genuine scientific question, and a fascinating one. But the campaign framed the outcome as a deliverable, a product that backers would receive, rather than as an experiment that might fail. When the engineering proved much harder than expected, with the introduced genes producing only dim light and the pathway difficult to transfer, the project had no honest way to fulfil its promises. The scientific risk had been sold as an execution certainty. Tests for a crowdfundable project Before committing to a campaign, it helps to put the project through a set of questions. Each is simple; together they catch most of the common failures. Can the question be stated in one sentence that a curious fourteen-year-old would understand? Not simplified to the point of inaccuracy, but stripped of jargon. "Do mussels closer to the city carry more microplastic?" passes. "Characterizing spatial heterogeneity in anthropogenic particulate burden in Mytilus" does not, though it describes the same study. Can the answer be delivered, in some form, within a period backers will tolerate? For most campaigns, that means results within six to eighteen months of funding. Longer projects can be crowdfunded, but they need a plan for intermediate outputs that backers will see along the way. Is there a clear link between the money and the work? A backer should be able to see that her money buys specific things: sequencing, travel, reagents, equipment, participant payments. A budget that consists mainly of salary for an unspecified amount of time is harder to support and harder to account for. Is the answer interesting whichever way it comes out? A project whose only publishable, communicable outcome is a positive result creates pressure to find one. A project whose answer is interesting either way, such as whether contamination declines with distance or does not, removes that pressure and makes honest reporting easier. Can the researcher actually do it? The researcher, or the team, should have the skills, the access to equipment, the permissions and the time. If any of these is uncertain, the uncertainty should either be resolved before launch or stated plainly on the campaign page. Is the project within ethical and regulatory bounds, and are the approvals in hand or clearly obtainable? Research involving human participants, animals, protected species, controlled substances, genetically modified organisms or sensitive data is subject to oversight that does not disappear because the funding source is unusual. Chapter 8 discusses this in detail. The practical point here is that approvals should be planned before launch, not after. Would the researcher be comfortable having every backer read the final report? This is the simplest test of all, and it catches a surprising number of problems. If the honest answer is no, because the project is really a pretext for something else, or because the promised outcomes are exaggerated, or because the researcher does not intend to report negative results, the project should be rethought. Types of project that work well Experience across the research platforms suggests several recurring patterns of project that suit the model. The pilot. A small study designed to generate the preliminary data that a larger grant application needs. Pilots are ideal because they are exactly what conventional funders are least willing to pay for and most want to see. A researcher can say to backers, truthfully, that their money is buying the evidence that may unlock a much larger investment later. The field season. A defined period of data collection in a particular place: surveying a population, sampling a site, recording an event. Field seasons have natural boundaries, concrete costs such as travel, permits and equipment hire, and strong narrative appeal. Backers can follow the expedition in real time. The sample run. A batch of laboratory analysis on material the researcher already has: sequencing, isotope analysis, dating, chemical assays. These projects are cheap to describe, cheap to cost and relatively low in execution risk, because the samples already exist. The instrument or tool. Building a piece of equipment, writing software or developing a method that the researcher, and often others, will use. Tools work well when they will be released openly, since backers can see a lasting public benefit beyond the single study. The community study. Research carried out with and for a particular community, such as patients with a rare disease, residents of a polluted neighbourhood or practitioners of a craft. These projects often have a ready-made audience that cares intensely about the question. They also carry the heaviest ethical responsibilities, discussed in Chapter 8. The replication or re-analysis. Repeating an influential study, or re-analysing public data with a new question. These are chronically underfunded because they carry little prestige, and yet many members of the public find them compelling once the stakes are explained. Types of project that work badly Some projects are poor fits regardless of how well the campaign is run. Anything promising a treatment or cure. Patients and families facing serious illness are among the most generous backers of research, and also among the most vulnerable to overpromising. A campaign that implies, even indirectly, that its results will lead to a treatment for a particular person or disease creates expectations research cannot meet. Campaigns connected to disease can be entirely legitimate, but they must be scrupulous about describing what the work can and cannot achieve. Clinical trials of interventions. Testing a treatment in people requires regulatory approval, trial registration, insurance, safety monitoring and a scale of funding that crowdfunding rarely reaches. There are rare exceptions run by established organizations with full oversight, but for an individual researcher this is not the right tool. Large capital projects. Equipment costing hundreds of thousands of dollars, facilities, long-term salaries. Crowdfunding rarely reaches these sums, and when it does, the obligations to backers become correspondingly heavy. Work that cannot be explained. Some important research is simply too technical to communicate without substantial background. That is no criticism of the research. It means the research should be funded some other way. Projects whose real purpose is a product. A study designed mainly to validate a commercial product, or a campaign whose rewards are the real attraction, is a pre-sale, not research funding. General-purpose platforms serve pre-sales well, but they should be described as such, and the research framing should not be used to lend them credibility. Reshaping an idea Many ideas fail these tests in their first form and pass them after reshaping. The reshaping usually involves some combination of four moves. The first is narrowing. A study of five species becomes a study of one. A survey of a whole region becomes a survey of a single valley. A comparison of ten conditions becomes a comparison of two. The narrowed project is less ambitious but far more deliverable, and the broader version can follow if the first result is interesting. The second is sequencing. The project is divided into stages, and the campaign funds only the first. The researcher explains the full plan so that backers see where the work is heading, but promises only what the first stage will produce. The third is de-risking. The researcher identifies the parts of the plan most likely to fail for practical reasons, and either resolves them before launch, for example by running a quick test of the method or securing the permit, or restructures the project so that their failure still produces a reportable result. The fourth is reframing the output. Instead of promising a discovery, the researcher promises a measurement. Instead of promising a working device, she promises a tested prototype and a report of how it performed. The output is something she controls, the doing of the work and the reporting of what happened, rather than something she does not, the answer nature gives. Consider how these moves might apply to a hypothetical proposal from an independent researcher interested in whether a traditional fermented food has antimicrobial properties. In its first form, the idea is to "discover new antibiotics in traditional foods." That promise is enormous, open-ended and misleading, since the chance of finding a clinically useful antibiotic in any single study is very small. Narrowed, it becomes a study of one food from one region. Sequenced, the first stage is simply culturing the microbes present and testing extracts against a panel of harmless laboratory bacteria. De-risked, the researcher first confirms that she has access to a laboratory with appropriate biosafety arrangements and has run a small trial of the culturing method. Reframed, the promised output is not "new antibiotics" but a public report and open dataset describing which microbes live in the food and whether any of them inhibit the growth of test bacteria in the dish. The reshaped project is smaller, more honest and much more likely to be funded, and its result, whether positive or negative, is something the researcher can deliver. The shape of the promise Every choice in this chapter is, at bottom, a choice about what the researcher will promise. The question, the scope, the output and the timeline together define the promise that backers will hold the researcher to. The best time to get that promise right is before a single dollar has been raised, when changing it costs nothing. A good promise is specific enough that backers know what they are paying for, modest enough that the researcher can keep it whatever nature decides, and interesting enough that people want it kept. The rest of the campaign, from the platform to the budget to the video, is built on that foundation. Chapter 3: Platforms, Models and the Rules of the Game Once a project has been shaped, the researcher faces a set of practical choices that are easy to make carelessly and hard to reverse: which platform to use, which funding model to adopt, whether to run the campaign in her own name or through an institution, and what legal and financial obligations follow. These choices determine who sees the campaign, how the money flows, what happens if the target is not met, and what the researcher owes, formally and informally, to the people who give. This chapter works through them in turn. Four models of crowdfunding The word "crowdfunding" covers several quite different arrangements, distinguished by what backers receive in return for their money. Four are relevant to research. In donation-based crowdfunding, backers give money and receive nothing tangible in return beyond thanks and, usually, updates on how the money is used. This is closest to charitable giving, and when the recipient is a registered charity or a qualifying institution, contributions may be tax-deductible for the donor. Most research-specific platforms operate largely on this model. In reward-based crowdfunding, backers receive something in exchange for their contribution: a postcard from the field site, a named acknowledgment in the eventual paper, a print of a microscope image, a video call with the researcher, a copy of a book. This is the model of the general creative platforms such as Kickstarter. Rewards can boost participation, but they add cost, work and obligations, and they change the legal character of the transaction, since a reward is something closer to a purchase than a gift. In equity crowdfunding, backers receive a share in a company. This is regulated as a securities offering. In the United States, the Jumpstart Our Business Startups Act of 2012 created a framework for it, and the Securities and Exchange Commission's Regulation Crowdfunding rules came into effect in 2016. Equity crowdfunding is relevant only to research carried out inside a company with commercial prospects, and it brings the full apparatus of investor disclosure and financial regulation. In lending or debt-based crowdfunding, backers lend money that is to be repaid, sometimes with interest. This is almost never appropriate for research, since research does not generate the cash flows needed to repay a loan. For the kind of exploratory research this book is about, the relevant models are donation-based and reward-based, and the choice between them has consequences explored below. Table 1 sets out the main differences across all four. Table 1. Crowdfunding models compared for research use. Model What backers receive Typical platforms Main obligations on the researcher Fit for exploratory research Donation-based Thanks, updates and access to results Research platforms such as Experiment; university giving sites; general donation sites Use funds as described; report honestly; observe charity and tax rules where applicable Strong: matches the gift-like nature of basic research Reward-based A tangible or experiential reward scaled to the contribution General creative platforms such as Kickstarter and Indiegogo Deliver rewards as promised; consumer-protection duties; tax on income in many jurisdictions Workable for projects with a natural product or artefact; risky when the research outcome is the reward Equity A share in a company Regulated equity portals Securities disclosure, investor reporting, financial regulation Only for commercial ventures; poor fit for open research Lending Repayment, sometimes with interest Peer-to-peer lending sites Repayment on schedule Almost never appropriate All-or-nothing versus keep-what-you-raise A second dimension cuts across the models: what happens if the campaign does not reach its target. Under an all-or-nothing rule, backers are charged only if the target is met. If the campaign falls short, no money changes hands and the researcher receives nothing. Kickstarter and Experiment both work this way. Under a flexible or keep-what-you-raise rule, the researcher receives whatever has been pledged, whether or not the target is reached. Indiegogo has offered this option, and most general donation platforms work this way by default. The all-or-nothing rule can seem harsh, but it serves both parties well in research. For backers, it guarantees that their money will only be spent if there is enough to do the work. Nobody wants to fund forty percent of a sequencing run. For researchers, it provides a clear deadline and a clear stake, and it guards against the uncomfortable position of having raised enough money to be obliged to do something but not enough to do it properly. The rule also has a strategic effect that researchers should understand. Studies of crowdfunding in general, including work by Venkat Kuppuswamy and Barry Bayus on Kickstarter, have found that support tends to cluster at the beginning and end of campaigns, and that backers respond to how close a campaign is to its goal. Under all-or-nothing, the deadline creates urgency, and the approach to the target often draws in final contributions from people who want to push it over the line. This makes the choice of target critically important, a subject taken up in Chapter 4. A flexible rule makes sense only when the project can be scaled smoothly to whatever sum is raised, for example a survey whose sample size can grow or shrink with the budget. Even then, the researcher should state in advance what will be done at different funding levels, so that backers know what their money will buy if the target is missed. Research platforms versus general platforms The choice of platform involves a trade-off between audience and fit. General-purpose platforms such as Kickstarter and Indiegogo attract enormous numbers of visitors and have well-developed tools for campaign pages, rewards and payments. But their audiences are browsing for games, gadgets, films and design objects, and a research project competes for attention with products that are easier to understand and more immediately rewarding. Their rules are also designed for creative projects and products. Kickstarter, for instance, has long required that projects create something to share with others and has restricted certain categories; its prohibition of genetically modified organisms as rewards after the Glowing Plant campaign is one example of how policies can change in response to research projects that sit uneasily with the platform's model. Research-specific platforms offer a better fit. Their pages are designed around the elements a research project needs: a question, methods, a budget, a team, a section for progress notes. Their audiences are self-selected for interest in science. Experiment, the largest, has staff review submitted projects for clarity, scientific accuracy and feasibility before they go live, which provides a modest form of quality control that backers can rely on. Its current terms, as published on its site, are an eight percent platform fee plus payment-processing fees of roughly three to five percent, charged only when a project is fully funded. Some projects on Experiment, those affiliated with qualifying non-profit organizations, are labelled as tax-deductible for United States donors. The trade-off is that research platforms bring relatively little traffic of their own. The evidence reviewed in Chapter 1 suggests that most backers of research campaigns come from the researcher's own networks rather than from browsing the platform. So the choice of platform matters less for finding backers than for how the campaign is presented, how the money is handled and what rules apply. The researcher should expect to bring her own audience wherever she goes. University platforms and institutional routes Many universities now operate their own crowdfunding sites, usually run by the development or advancement office that handles alumni giving. These have particular advantages and disadvantages. On the positive side, money raised through a university site is usually treated as a gift to the institution, which means that donors can typically claim tax deductions where the law allows, the money is held and accounted for by the institution, and the researcher is protected from having the funds treated as personal income. University sites often come with support from communications staff, access to alumni mailing lists and the credibility of the institution's name. On the negative side, university sites are often slower and more bureaucratic. Some restrict who may run a campaign, for example to faculty or to projects approved by a department. Some charge an administrative fee or an overhead levy. They may have limited reach beyond the institution's own supporters, and they may impose restrictions on how projects are described. For researchers at universities, a further question arises even when they use an external platform: whose money is it? Many institutions have policies requiring that funds raised for research conducted by their employees, or using their facilities, be routed through the institution. Some treat crowdfunded money as a gift, with minimal overhead; others treat it as a form of sponsored research, with the full apparatus of indirect cost recovery. A researcher who raises money personally on an external platform for work done in a university laboratory may find herself in breach of institutional policy, or may find that the university claims a share of the funds. The time to discover this is before launch. A short conversation with the institution's research office or development office usually settles the question. Graduate students face a particular version of this issue. Their research is often supervised and conducted in facilities they do not control, and their right to raise money in their own name varies between institutions. Their supervisors should be involved in the decision, both because the supervisor may have obligations to the institution and because the supervisor's support, and network, is often valuable to the campaign. Independent researchers Researchers without institutional affiliation face a different set of issues. They are free of institutional policies, but they lose the protections and infrastructure that institutions provide. Money raised by an individual on a crowdfunding platform is, in many jurisdictions, treated as taxable income to that individual, particularly when rewards are offered, and particularly when the sums are significant. The rules differ widely between countries and are not always clear. In the United States, payment processors report certain transactions to the tax authority, and the tax treatment of crowdfunding receipts depends on whether they are gifts, income or something else. An independent researcher should seek advice from an accountant familiar with the relevant jurisdiction before launching a campaign of any size, and should build the expected tax into the budget. Some independent researchers address these issues by working through a fiscal sponsor: a registered non-profit that accepts funds on behalf of a project and administers them for a fee. Fiscal sponsorship can make donations tax-deductible, provide financial accountability and give the project a legal home. Some independent research organizations have been established partly to serve this function for researchers outside universities. Independent researchers also need to arrange for the facilities, oversight and insurance that institutions ordinarily supply. A study involving human participants needs ethical review regardless of who funds it, and an independent researcher may have to use a commercial or independent review board. Laboratory work may require access to rented facilities, community laboratories or collaborations with institutions. These arrangements have costs and lead times that belong in the campaign plan. Legal and consumer-protection obligations Crowdfunding sits in a patchwork of law that varies between countries and is still developing. A few general points apply widely. Platforms set terms of use that bind campaign creators. Kickstarter's terms, for example, require creators who cannot fulfil their promises to make a good-faith effort to complete the project, communicate honestly with backers about what went wrong, account for how the money was spent and offer refunds where appropriate. These terms are contractual. They are enforceable in principle, even if enforcement is rare. Consumer-protection law may also apply, particularly to reward-based campaigns. In 2015 the United States Federal Trade Commission brought its first action against a crowdfunding creator, a man who had raised money on Kickstarter for a board game, failed to deliver it and spent much of the money on personal expenses. The case established that regulators regard crowdfunding promises as potentially enforceable representations to consumers. State attorneys general have brought similar actions. Research campaigns rarely attract this kind of scrutiny, but the principle is clear: promises made on a campaign page can have legal weight. Charity law applies when donations flow through registered charities, including universities and fiscal sponsors. Funds given for a specified purpose may be legally restricted to that purpose. If the project cannot be carried out, the charity may be obliged to seek donors' consent before redirecting the money, or to return it. None of this should deter a researcher from crowdfunding. The obligations it imposes are, for the most part, obligations an honest researcher would honour anyway: do what you said, tell people what happened, account for the money and offer a remedy if things go wrong. But it does mean that the promises on a campaign page should be written with care, and that a researcher should know which rules govern her particular campaign before it launches. Making the choice Pulling this together, most researchers will find that their situation points fairly clearly to one route. A university researcher with a modest project and a supportive institution will often do best on the institution's own platform, or on a research platform with the funds routed through the institution, after checking policy with the research office. A graduate student with a well-defined project and a supervisor's support will often do well on a research platform, with the supervisor's agreement and an understanding of how the funds will be held. An independent researcher will usually need either a fiscal sponsor or a clear understanding of the personal tax consequences, and will need to budget for the review, facilities and insurance that an institution would otherwise supply. A researcher whose project has a natural, deliverable artefact, such as a book, a tool, a dataset with public value or a documentary, may reasonably choose a general reward-based platform, provided she keeps a sharp line between the reward and the research outcome. In every case, the choice should be made with the promise in mind. The platform, the model and the institutional route are the vehicles for that promise, and each shapes how it will be understood and enforced. Hashtags: #ScientificCrowdfunding #ResearchCrowdfunding #PublicScience #CommunityBackedResearch #ResearchCampaigns #CrowdfundingPlatforms #ExperimentPlatform #DonationBasedCrowdfunding #RewardBasedCrowdfunding #AllOrNothingFunding #ResearchCommunication #ScienceOutreach #PublicEngagement #BackerCommunity #CampaignDesign #ResearchBudgeting #CampaignPitch #PitchVideo #AudienceBuilding #PilotResearch #ExploratoryResearch #OpenResearch #ResearchTransparency #CrowdfundedScienceEthics #FutureOfScientificCrowdfunding

Download the Book (PDF): Introduction Somewhere in almost every research university there is a freezer, a server or a filing cabinet holding an invention that works. It has been published. It has been cited. It may have been presented to a room of industry visitors who nodded, took a business card and never called. The inventor knows it could matter outside the lab: a diagnostic that would catch a disease earlier, a catalyst that would make a chemical process cheaper, a material that would let a battery survive more cycles. And yet nothing happens. The paper goes into the literature, the graduate student who did the work graduates, and the invention settles into the long sleep of things that were proved possible and never made real. The gap between a working result in a laboratory and a product that someone will pay for has a name in policy circles: the valley of death. The phrase is dramatic, and deliberately so. It describes a stretch of development where the work is too applied for most research funders, too risky for most corporate buyers and too early for most investors. Public money pays for discovery. Private money pays for scale. In between lies a zone where nobody's job description obliges them to pay, and where most promising technologies quietly die. This book is for the academic who wants to get an invention across that zone. It is written for the principal investigator with a result she believes in, the postdoctoral researcher weighing whether to found a company rather than chase a faculty post, and the graduate student who has noticed that the thing they built for their thesis is better than anything on the market. It is also written for the people around them: department heads, technology transfer staff, and the early investors and advisers who have to work with scientists and are sometimes baffled by how differently scientists think. The argument of this book The central claim of what follows is simple, and it explains most of the frustrations academic inventors experience. The valley of death is not primarily a shortage of money. It is a shortage of the right kind of evidence. A scientist spends a career learning to produce one kind of proof: evidence that a phenomenon is real, that a mechanism works as proposed, that a result is reproducible and novel. That is the evidence peer reviewers demand and grant panels reward. But every audience on the far side of the valley wants a different kind of proof. A patent examiner wants evidence that the invention is new, not obvious, and described well enough for someone else to make it. A licensing officer wants evidence that a company somewhere will pay for the rights. A grant agency funding translation wants evidence that the next experiment will retire a specific technical risk. A venture capitalist wants evidence that the technology can become a business returning many times the money put in, within the life of a fund. A university conflict-of-interest committee wants evidence that the inventor's financial stake will not bend the science, harm students or endanger research subjects. These are not the same evidence, and the gap between them is where inventions stall. A paper showing that a molecule binds its target with remarkable affinity is a triumph in a journal and nearly useless in a pitch meeting, where the questions are about manufacturability, toxicity, the size of the patient population, the regulatory path and whether anyone else owns the chemistry. The inventor who treats each new audience as a harder version of peer review, and responds by generating more of the same evidence, will keep losing. The inventor who learns to see each stage as a translation problem, converting what is known into what the next gatekeeper needs to believe, has a real chance. Seen this way, each of the institutions in this book stops being an obstacle and becomes a converter. The technology transfer office converts a discovery into owned, licensable property. The patent converts an idea into an exclusive right that an investor can value. Translational grants convert technical uncertainty into data. The founding team converts a licence into an organisation that can act. Venture capital converts demonstrated potential into the money needed to realise it. And a well-run conflict-of-interest process converts a dangerous tangle of loyalties into an arrangement that lets the inventor stay a scientist while becoming a founder. None of these conversions happen automatically, and each one has its own grammar. What the book covers, and in what order The chapters follow the rough order in which an academic inventor meets each problem, though in practice they overlap and loop back. The first chapter looks at the valley of death itself: why it exists, what economists and policymakers have said about it, and why deep-technology inventions, those grounded in new science or engineering rather than in new business models, fall into it more often than software or consumer products do. It introduces the idea, which runs through the rest of the book, that each audience on the far side of the valley needs its own kind of evidence. The second chapter turns to the university's technology transfer office, the institution most academic inventors meet first and understand least. It explains who owns an academic invention and why, how the Bayh-Dole Act reshaped university patenting in the United States and influenced policy elsewhere, what an invention disclosure is for, and how to work with a transfer office rather than against it. The third chapter is about patents: what they protect, what they cannot protect, and the specific ways in which academic habits, especially the rush to publish and the conference talk given before a filing, can destroy rights that would otherwise have existed. The fourth chapter moves from a single patent to a portfolio and a licence, explaining how a university decides between licensing to an established company and backing a spinout, and what the terms of a spinout licence typically contain. The fifth chapter deals with non-dilutive funding: the grants, prizes and public programmes that pay for translation without taking equity. These are often the most valuable money a deep-tech company ever receives, and they are badly misunderstood. The sixth chapter covers the founding of the company itself: who should run it, how equity should be divided, what role the academic founder should take and the mistakes that most often poison a young company before it raises its first round. The seventh chapter is about venture capital: how venture funds actually work, why their economics shape every question an investor asks, how to pitch a scientific company to people who are not scientists, and what a term sheet means. The eighth chapter addresses conflicts of interest and commitment, the area where academic inventors most often get into real trouble, and where the stakes include research integrity, student welfare and, in clinical research, human lives. A note on scope A short book on a subject this large has to leave things out. The emphasis here is on the United States, the United Kingdom and continental Europe, because that is where most of the relevant policy and most of the academic spinout activity has been concentrated, though many of the same patterns hold in Canada, Australia, Israel, Singapore and elsewhere. The book concentrates on inventions that need new science or engineering to reach the market, in fields such as therapeutics, diagnostics, medical devices, advanced materials, energy, semiconductors, quantum technologies and robotics. Pure software start-ups founded by academics face some of the same questions, but they rarely depend on patents or on long translational development, and much of this book would matter less to them. The book also assumes a reader who is willing to learn a new language without surrendering the old one. A recurring theme is that the best academic founders do not stop being scientists. They add a second competence alongside the first. They learn to hear what a licensing officer, an examiner or an investor is actually asking, and they answer in terms that audience can use, while protecting the integrity of the research that made the invention possible in the first place. Much of what follows touches on law and finance: ownership of intellectual property, patent procedure, licence terms, share structures, investment agreements and regulations governing financial interests in research. It is offered as general information about how these systems usually work, to help an inventor understand the landscape and ask better questions. It is not legal, patent or investment advice. Rules differ between countries, between institutions and over time, and the specific facts of an invention matter enormously. Before signing, filing or investing anything, an inventor should consult a qualified patent attorney, a lawyer who acts for them personally rather than for the university or an investor, and, where money is involved, a qualified financial adviser. Why this matters It is easy to treat academic commercialisation as a side issue, a way for universities to earn a little licensing income or for a few scientists to become rich. That view misses what is at stake. Many of the technologies that define modern life began in academic laboratories and crossed into industry through the mechanisms this book describes: recombinant DNA, the search algorithm that built Google, the lithium-ion chemistries that power phones and cars, messenger RNA vaccines, CRISPR gene editing and a long list of medicines. Each of those crossings was contingent. Each could have failed at several points, and the history of science is full of inventions of similar promise that did. Every invention that dies in the valley represents public research money that produced knowledge but no benefit anyone could use. Every one that crosses, by contrast, can create jobs, treatments, cleaner industries and the tax revenue that funds the next generation of research. The inventor who learns to cross is doing more than building a company. They are completing the social bargain that justified the research funding in the first place. The path is hard and the odds are poor. Most spinouts fail, as most start-ups do. But failure in the valley is not random. It follows recognisable patterns: rights lost before anyone thought to file, licences negotiated badly, funding sought from the wrong source at the wrong time, founding teams that could not function, pitches that answered the wrong questions, conflicts left unmanaged until they became scandals. Each of these is avoidable, and the chapters that follow explain how. Chapter 1: Why Good Science Dies The phrase "valley of death" entered American science policy in the 1990s and was popularised in Congressional testimony and reports about the gap between federally funded research and commercial development. Its most careful early treatment came in a 2002 study for the National Institute of Standards and Technology by Lewis Branscomb and Philip Auerswald, Between Invention and Innovation. They argued that the image of a valley, a single chasm between two well-funded plateaus, was slightly misleading. The terrain looked more like a sea of uncertainty in which some projects swam and most sank, where survival depended on a patchwork of angel investors, corporate partnerships, government programmes and luck. Yet the core of the metaphor has survived because inventors recognise it. There is a stage of development where nobody is obliged to pay, and it is exactly the stage where an academic invention usually sits. To cross it, it helps to understand why it exists. The valley is not an accident of bad policy that a better funding scheme could simply abolish. It follows from the different logics of the institutions on either side. Two plateaus with different rules On one side sits the research system. Public funders, charities and universities pay for work whose value lies mainly in knowledge. The economist Kenneth Arrow set out the classic reasoning in 1962: knowledge is hard to own, easy to copy once revealed, and uncertain in its payoff, so private markets will produce less of it than society would want. Governments therefore fund basic research directly. The research system rewards novelty, rigour and publication. It is organised around the individual investigator and the grant cycle, which typically runs three to five years. Its outputs are papers, trained people and, occasionally, inventions. On the other side sits the commercial system. Companies and investors pay for work whose value lies in products that customers will buy. They reward predictability, cost control, scale and return on capital. Established companies fund development when the technical risk is modest and the market is clear, because their shareholders expect steady returns and their internal budgets favour projects likely to reach revenue within a few years. Venture investors accept more risk, but they need a credible path to very large returns within the life of a fund, commonly ten years. The valley opens because an academic invention usually fails the entry test of both systems at once. It has already produced its main contribution to knowledge, so research funders are no longer interested in paying for the unglamorous work of making it reliable, manufacturable and cheap. Nobody writes a high-impact paper showing that a promising catalyst still works after ten thousand hours, or that a device can be assembled with a tolerance a factory can hold. Yet that work is exactly what the commercial side needs to see before it will commit money. The invention is too applied for one side and too risky for the other. Technology readiness and the missing middle Engineers describe this gap with technology readiness levels, a scale developed at NASA in the 1970s and 1980s and later adopted by the US Department of Defense, the European Commission's research programmes and many national funders. The scale runs from level 1, where basic principles have been observed, to level 9, where the actual system has been proven in operational use. Most academic research sits between levels 1 and 3: principles observed, a concept formulated, a proof of concept shown experimentally. Most commercial investment begins around levels 6 or 7, where a prototype has been demonstrated in a relevant or operational environment. Levels 4 and 5, validation in the laboratory and then in a relevant environment, are the floor of the valley. This is where the question changes from "does it work?" to "does it work reliably, outside the conditions that made it work the first time, at a cost and scale that matter?" It is expensive, slow, intellectually unfashionable and full of unpleasant surprises. It is also where the value of the invention is actually determined. The readiness scale was designed for engineered systems, and it maps imperfectly onto drugs, where the milestones are set by regulators and clinical trials, or onto materials, where scale-up often reveals entirely new chemistry. But the underlying point generalises. There is a middle phase of development in which technical risk is still high and the evidence that would persuade a commercial buyer does not yet exist, and nobody on either plateau sees paying for it as their job. Why deep technology falls further The valley is not equally deep for every kind of invention. A new mobile application can go from idea to paying customers in months with a few hundred thousand dollars. Its technical risk is low, because the tools are well understood, and its main uncertainty is whether customers want it. A new battery chemistry, cancer therapy, semiconductor process or fusion component faces technical risk, market risk and often regulatory risk all at once, and it needs years and tens or hundreds of millions of dollars to resolve them. The term deep tech, which became common in the 2010s, describes this second class: ventures built on substantial scientific or engineering advances rather than on new business models applied to existing technology. Deep-tech ventures share several features that make the valley wider. They need capital before they have customers. A software company can often sell a rough version of its product early and use revenue to fund development. A company developing a new medicine may spend a decade and hundreds of millions of dollars before selling anything at all. They carry technical risk that cannot be resolved by talking to customers. Customer discovery tells you whether people want the thing; it cannot tell you whether the physics will cooperate at scale. Some questions can only be answered by building, and building is expensive. They often require specialised infrastructure: clean rooms, pilot plants, animal facilities, clinical sites, regulatory expertise. These are costly to build and hard to rent. Their markets are frequently dominated by large incumbents who control distribution, manufacturing and customer relationships. A new diagnostic may be scientifically superior and still fail because hospital purchasing, reimbursement codes and clinical guidelines are all built around the existing test. And their timelines are long. Venture funds are usually structured to return money within about ten years, with most investments made in the first few. A technology that needs twelve years to reach meaningful revenue is awkward for a fund of that shape, regardless of its eventual value. None of this makes deep tech a bad investment. Some of the most valuable companies of the last half century, from Genentech to the major semiconductor equipment makers, are deep-tech companies. But it means the evidence a deep-tech inventor needs to produce before anyone will fund them is harder, slower and more expensive to generate than the evidence a software founder needs, and the stretch of development during which nobody wants to pay for it is longer. The evidence problem The claim of this book is that the valley is at heart an evidence problem. Money follows evidence; the reason nobody pays for the middle phase is that nobody yet has the evidence that would justify paying, and generating that evidence is itself the thing that costs money. Breaking the circle requires seeing clearly what evidence each audience needs. An academic career trains one kind of evidence very well. A good scientist knows how to design a controlled experiment, how to rule out alternative explanations, how to show that a result is statistically robust and how to write it up so that peers accept it. These skills remain essential for commercialisation, because an invention that is not real cannot be rescued by any business plan. But they are only the first rung. The table below sets out, in simplified form, what each of the main audiences an academic inventor meets is really trying to establish, and the kind of evidence that persuades them. The categories overlap and the details vary by field, but the pattern matters more than any single cell. Table 1 compares these audiences. Table 1. What different audiences need to believe. Audience Core question Evidence that persuades Evidence that does not Journal reviewer Is it true and new? Controlled experiments, reproducibility, novelty against literature Market size, cost estimates Patent examiner Is it new, non-obvious and fully described? Comparison with prior art, enabling description, working examples Citation counts, impact factor Licensing officer Will anyone pay for rights? Named industry interest, defensible claims, a plausible product Elegance of mechanism Translational funder Will this project retire a defined risk? Milestones, go/no-go criteria, a credible team Broad future applications Venture investor Can this return many times the fund's money? Large market, strong team, protected position, path to scale A single impressive dataset Conflict committee Will financial interest bend the research or harm people? Disclosure, separation of roles, independent oversight Assurances of good intent The last column is where academic inventors lose time. Faced with an unpersuaded investor, many respond by producing more of the evidence they already know how to produce: another figure, a higher-resolution image, a tighter error bar. It rarely works, because the investor was not doubting the science. They were doubting whether anyone would buy the product, whether the team could build a company, or whether a larger competitor already had a patent covering the same ground. How inventions actually die It is worth being specific about the ways academic inventions fail in the valley, because they are more varied than the image of a single funding gap suggests. Several patterns recur. Some die before anyone realises they could be commercialised. The inventor publishes, the rights are lost, and by the time someone sees the potential the knowledge is in the public domain. This is not always bad, since open knowledge has great value, but when a product needed private investment to reach users, the loss of patent protection can mean the product never gets made. Some die in the technology transfer office. The office, faced with more disclosures than it can patent, decides the invention is too early or the market too uncertain, and declines to file. The inventor, not understanding why or what might change the decision, gives up. Some die in a licence. The rights go to an established company that has no strong incentive to develop them, perhaps because the technology competes with its existing products or because internal priorities shift. The invention sits on a shelf, protected by patents that keep anyone else from developing it. Some die in the lab, during the middle phase, because the technology simply does not scale. The effect that was dramatic in a small sample vanishes in a larger one; the material that performed beautifully in grams cracks in kilograms; the drug that worked in mice is toxic in larger animals. This is honest failure, and it is the valley working as it should: most early ideas do not survive contact with reality, and finding that out quickly and cheaply is a success of sorts. Some die in the company, from causes that have nothing to do with the technology: a founding team that cannot agree, an equity split that leaves nobody motivated, a chief executive who cannot raise money, a board that runs out of patience. Investors routinely say they back teams more than technologies, and early failure in deep-tech companies is often a team failure. And some die of conflict. A professor's company and laboratory become entangled; students are drawn into unpaid company work; a clinical trial is compromised by a financial interest; a public scandal forces the university to withdraw. These are the rarest failures and the most damaging, because they harm people beyond the venture itself. Each of these failure modes has its own remedy, and most of the rest of this book is organised around them. But the common thread is that at each point an invention needed evidence it did not have, or needed its evidence translated for an audience that could not read it. Timescales, honestly stated Academic inventors routinely underestimate how long commercialisation takes. The misjudgement comes partly from optimism and partly from the grant cycle, which trains people to think in three-year blocks. In therapeutics, the path from a promising academic discovery to an approved medicine commonly takes well over a decade, with clinical development alone often running seven years or more, and the great majority of candidates that enter human trials never reach approval. In medical devices the timelines are usually shorter, but regulatory clearance, clinical evidence and reimbursement can still take many years. In advanced materials and energy, the gap between a laboratory demonstration and a product at industrial scale has historically been measured in decades; lithium-ion batteries, for instance, took roughly two decades from the foundational chemistry of the 1970s and early 1980s to commercial cells in 1991, and far longer to reach the scale and cost that made electric vehicles practical. Semiconductor process innovations often need a similar arc from laboratory to fabrication plant. This does not mean the academic founder must wait fifteen years for any reward. Companies can be sold, licensed or partnered long before a final product exists, and investors are frequently repaid when a larger company acquires a spinout after it has passed a key milestone. But it does mean that the founding decision should be made with a realistic sense of the commitment involved. Founding a deep-tech company is not a sabbatical project. Where the crossing points are If the valley is an evidence problem, then crossing it is a matter of generating the right evidence in the right order, as cheaply as possible, and finding people willing to pay for each step. Historically, several institutions have grown up to do exactly that, and each of them is the subject of a later chapter. University technology transfer offices exist to capture and license the intellectual property that research produces, and to steer inventions towards companies that can develop them. Patents turn inventions into assets that investors can value and that protect the long, expensive development needed to cross the valley. Translational and small-business grants, from programmes such as the US Small Business Innovation Research scheme, the European Innovation Council, national innovation agencies and medical research charities, pay for the middle phase without taking ownership. Venture capital, especially the subset of firms that specialise in deep technology, takes on high risk in exchange for a share of the upside. And spinout companies are themselves a mechanism for crossing, because they create an organisation whose only job is to push one technology forward, unlike a university laboratory, which must keep producing new knowledge, or a large company, which must balance many priorities. No single one of these institutions spans the valley. The successful crossings are almost always assembled from several of them in sequence: a disclosure leads to a patent, the patent supports a translational grant, the grant produces data, the data support a licence to a new company, the company raises seed money, the seed money funds a demonstration, and the demonstration attracts a larger round. At each handoff there is a translation task. The inventor who understands what each institution needs, and why, can manage those handoffs far better than one who treats each as an unwelcome bureaucratic hurdle. A different way of seeing the inventor's job The most important shift an academic inventor can make is to stop thinking of commercialisation as something that happens after the science is done. In the research system, the paper is the end of a project. In the commercialisation system, the paper is closer to the beginning, and it can even be an obstacle, if it disclosed the invention before rights were protected or oversold results that later work cannot reproduce. The shift is not towards being less rigorous. Investors and partners are, if anything, more unforgiving of irreproducible results than journals, because they lose money on them. A 2011 report from scientists at Bayer and a 2012 commentary by researchers at Amgen both described how often industry teams failed to reproduce published preclinical findings, and those reports sharpened a lesson experienced investors had already learned: that a striking academic result is a hypothesis, not a product. The inventor who welcomes independent replication, runs blinded validation and publishes negative results about their own technology's limits earns a credibility that no pitch deck can buy. The shift is towards a wider view of what counts as evidence, and towards treating every stage as a chance to generate the proof the next audience will need. That begins before the first patent is filed, with the inventor's relationship to the institution that almost certainly owns the invention, whether the inventor realises it or not. Chapter 2: Who Owns the Invention, and the Office That Manages It Many academic inventors are surprised, sometimes angry, to learn that the invention they conceived, built and published does not belong to them. In most research universities in the United States, the United Kingdom and much of Europe, an invention made by an employee in the course of their research, using university resources, belongs to the university. The inventor is named on the patent, shares in any income and often has a strong say in what happens next. But the legal owner, the party with the right to license, sell or abandon the invention, is usually the institution. Understanding why that is, and how the office that manages university inventions actually works, is the first practical step in commercialisation. A remarkable amount of friction between inventors and universities comes from mismatched expectations rather than genuine conflict. How universities came to own inventions In the United States, the modern system was shaped by the Patent and Trademark Law Amendments Act of 1980, universally known as the Bayh-Dole Act after its Senate sponsors, Birch Bayh and Bob Dole. Before 1980, inventions arising from federally funded research generally belonged to the federal government unless an agency granted a waiver, and the government licensed them non-exclusively. Few companies were willing to invest in developing a technology that any competitor could license on the same terms, and a large proportion of government-owned patents went unused. Bayh-Dole allowed universities, small businesses and non-profit institutions to elect to retain title to inventions made with federal funding, on conditions. The institution must disclose the invention to the funding agency, decide within a set period whether to take title, file patent applications in a timely way, try to commercialise the invention, give preference to small businesses when licensing, and share royalties with the inventor. The government keeps a non-exclusive licence to use the invention for its own purposes and holds so-called march-in rights, allowing it to require further licensing if the owner fails to take effective steps towards practical application or in certain other circumstances. March-in has been petitioned many times, especially over drug prices, but no agency has ever exercised it. The law is widely credited with, and sometimes blamed for, the growth of university patenting and licensing since 1980. Before it, only a handful of universities ran serious patent operations; the Wisconsin Alumni Research Foundation, founded in 1925 to manage Harry Steenbock's vitamin D patents, and Research Corporation, founded in 1912, were notable early exceptions. Afterwards, nearly every research university created a technology transfer office. Scholars have argued about how much of the change was caused by Bayh-Dole and how much by concurrent developments, including the biotechnology revolution and a 1980 Supreme Court decision, Diamond v. Chakrabarty, that allowed patents on genetically engineered organisms. But the institutional model it established, in which universities own and license academic inventions and share income with inventors, has been imitated in various forms across much of the world. Bayh-Dole does not itself hand an invention to the university. In 2011, in Stanford v. Roche, the US Supreme Court held that the Act does not automatically vest title in the institution; the inventor owns their invention in the first instance, and the university obtains title through an assignment from the inventor. That is why university employment agreements and intellectual property policies matter so much. After the decision, many universities revised their agreements to use present-tense assignment language, stating that the employee "hereby assigns" future inventions, rather than a mere promise that they "agree to assign", because the case turned partly on that distinction. Ownership outside the United States In the United Kingdom, inventions made by employees in the course of their normal duties belong to the employer under the Patents Act 1977, and universities generally own inventions made by their academic staff. Students are a different matter, since they are not employees, and university regulations often require them to assign rights when they work on funded projects or use university facilities. In Germany, a long tradition known as the professor's privilege gave university professors ownership of their own inventions, but it was abolished in 2002, and German universities now claim inventions under the Employee Inventions Act. Sweden, by contrast, retains a form of the professor's privilege: Swedish university teachers generally own their own inventions, a notable exception among major research economies. Other countries, including Italy, have moved in both directions over the years. The details differ, and every institution's policy says something slightly different about students, visiting researchers, consulting work, software, copyright and materials. The first practical step for any academic inventor is to read their own institution's intellectual property policy, their employment contract, and the terms of any grant or industry agreement that funded the work. Grants and sponsored research agreements frequently contain their own provisions on ownership and licensing, and a company that funded part of the research may already hold an option to license the result. The invention disclosure The invention disclosure is the formal document by which an inventor tells the university about an invention. It is usually a form, submitted to the technology transfer office, asking for a description of the invention, the names of all contributors, the funding sources, any planned or past publications and presentations, and any known commercial interest. Inventors often treat the disclosure as paperwork. It is better understood as the first commercial document in the life of the invention, and a well-written one changes what happens next. The transfer office uses it to decide whether to spend money on a patent application, and in a busy office that decision is made quickly, often by a licensing officer who is not an expert in the inventor's field and who is managing dozens of other cases. A disclosure that helps its own cause has several features. It states plainly what the invention does that existing approaches cannot, in terms a non-specialist can follow. It distinguishes the invention from the closest known work, including the inventor's own earlier publications. It identifies the evidence that the invention works and is honest about what has not yet been shown. It names the likely applications and, where possible, specific companies or kinds of companies that might be interested, including any that have already expressed interest. And it gives accurate dates for any public disclosure, past or planned. The list of inventors deserves care. Inventorship under patent law is not the same as authorship on a paper. A person is an inventor only if they contributed to the conception of at least one claimed element of the invention; a technician who carried out experiments under instruction, or a senior colleague who secured the funding, is not an inventor merely for that reason. Getting inventorship wrong, in either direction, can make a patent vulnerable, and disputes over it are a common source of bitterness in laboratories. The honest course is to describe each person's contribution accurately and let the patent attorney apply the legal test to the claims as finally drafted. Timing matters as well. The disclosure should be filed well before any planned publication, conference talk, poster, thesis deposit or even a detailed seminar to an outside audience. The reasons are explained in the next chapter, but the short version is that in most of the world, a public disclosure before a patent filing destroys the right to patent. Students, collaborators and shared inventions Ownership becomes complicated as soon as more than one institution, or more than one kind of person, is involved, and academic inventions very often involve both. A graduate student funded by a fellowship rather than a salary may not be an employee at all, and whether the university owns their contribution depends on what they signed when they enrolled or joined a funded project. A visiting researcher may remain employed by a home institution that has its own claim. A collaborator at another university almost certainly does. An industry scientist seconded to the laboratory brings their employer's claims with them. When an invention has contributors from several institutions, each institution typically owns an undivided share, and the institutions must agree which of them will lead on patenting and licensing. These inter-institutional agreements are routine, but they take time, and an invention cannot easily be licensed to a spinout until they are in place. Under US law, each co-owner of a patent may in principle exploit it without the consent of the others, which is precisely why a prospective licensee will insist that all owners join in, or that one has authority to act for all. The practical lesson is to establish, at the start of any collaboration, who is contributing what and under which agreements. It is far easier to settle ownership when an invention is hypothetical than when it has become valuable. Students deserve particular care. They are less able to protect their own interests than faculty, their contribution can be substantial, and a dispute over whether a student is an inventor, or whether they can use their own thesis work in a later company, can damage both the student's career and the venture. A laboratory head who makes the rules explicit protects everyone, including themselves. What a transfer office actually does Technology transfer offices go by many names: tech transfer offices, innovation offices, commercialisation units, knowledge exchange teams. Some are departments within the university; others, such as Oxford University Innovation or Imperial College's former commercialisation arm, have been set up as wholly owned companies. Some universities have outsourced the function or shared it among several institutions. Their core work, however, is similar. The office receives disclosures and assesses them. It decides which inventions to protect, commissions patent attorneys to draft and file applications, and manages the resulting portfolio, paying renewal fees and responding to examiners. It markets inventions to potential licensees, negotiates licence and option agreements, and, increasingly, helps to form spinout companies. It manages material transfer agreements, confidentiality agreements and parts of sponsored research contracts. It collects and distributes licensing income. And, in many universities, it runs or supports entrepreneurship programmes, proof-of-concept funds and relationships with investors. The office usually does this with a small staff. A licensing officer may carry a portfolio of many active inventions, and patent budgets are always limited. Most offices therefore apply some form of triage. They file provisional or priority applications on a relatively wide range of inventions, which is cheap, and then decide within a year which of them merit the much larger costs of international filing. Why good inventions get declined A transfer office declining to file, or dropping an application after the first year, is one of the most demoralising experiences in academic commercialisation. It helps to understand the reasons. The most common is that nobody appears likely to pay. A patent costs money to obtain and more to maintain, and a university cannot recover that cost unless someone takes a licence. If the office cannot identify a plausible licensee, whether an existing company or a spinout with a credible team, it may reasonably conclude that the money is better spent elsewhere. Another is that the invention is too early. A promising observation that has not yet been shown to work in a useful setting may be patentable, but the claims will be narrow or speculative, and the patent may expire before a product exists. Offices sometimes advise waiting until the data are stronger, provided the inventor can hold off publishing. A third is that the claims would be weak: the prior art is crowded, the invention is an incremental improvement, or it falls into a category where patents are hard to obtain or easy to design around. A fourth is that the rights are encumbered. An industry sponsor may hold an option, a funder may impose conditions, or the invention may depend on another party's patents. And sometimes the office is simply wrong, or overwhelmed, or lacks expertise in the field. It happens. But an inventor who responds to a decline by complaining rarely changes the outcome. One who responds by addressing the stated reason often does. If the concern is market interest, a letter from a company saying it wants to evaluate the technology changes the calculation. If the concern is prematurity, a specific experiment that would demonstrate utility, and a commitment to delay publication until it is done, may persuade the office to file. If the concern is cost, a translational grant or an early-stage investor willing to cover patent expenses can shift the balance. Where the university decides not to pursue an invention, many policies allow the rights to be released to the inventor, subject to conditions such as repayment of costs and, for federally funded inventions in the United States, approval from the funding agency. Inventors in this situation should ask. Revenue sharing and what it signals Universities share licensing income with inventors, and Bayh-Dole requires this for federally funded inventions without specifying the proportion. The formulas vary widely. Some institutions pay inventors a third of net income; some use sliding scales that give inventors a larger share of the first slice of income and a smaller share thereafter; some split the remainder between the inventor's department, their laboratory and the central university. Net income usually means income after the office recovers its patent costs, which can take some time. The sharing formula matters less than inventors tend to think, for a simple reason: most inventions never generate significant income. Licensing income across the university sector is highly skewed, with a small number of blockbuster inventions accounting for a large share of the total. The Cohen-Boyer recombinant DNA patents, licensed non-exclusively by Stanford and the University of California from 1980 until they expired in the late 1990s, generated about a quarter of a billion dollars. Stanford's licence of the PageRank search algorithm to Google, in exchange partly for shares, yielded shares that the university sold in 2005 for about 336 million dollars. Such cases are exceptional, and many transfer offices do not cover their own operating costs from licensing income. What the revenue formula does signal is how the university thinks about the relationship. An office that treats licensing income as a central source of revenue will negotiate differently from one that treats knowledge transfer as a public mission and accepts modest returns in exchange for more inventions reaching use. Both views exist, often in the same institution, and inventors benefit from knowing which one they are dealing with. Working with the office rather than against it The relationship between inventor and transfer office works best when each side understands the other's constraints. The office has a limited budget, legal obligations to funders and the university, and a portfolio of competing cases. The inventor has deep technical knowledge, networks in the field and usually the strongest motivation to see the invention used. Several practices make the relationship productive. Disclose early, well before publication, so that the office has time to act. Keep the office informed about planned talks, papers and collaborations; an office that is surprised by a conference abstract cannot protect the invention. Bring market intelligence: the names of people in industry who have asked about the work, the companies that dominate the relevant market, the problems customers complain about. Respond quickly to questions from patent attorneys, since delays can cost priority. And be honest about limitations, because an office that discovers late that a key result was not reproducible will be warier the next time. It also helps to understand the language of the office. A licence grants rights to use a patent or other intellectual property, which may be exclusive, meaning only the licensee may use it, or non-exclusive, and may be limited to particular fields of use, territories or periods. An option gives a company the right, for a fee and a limited time, to evaluate a technology and then negotiate a licence on terms defined in advance. A material transfer agreement governs the sharing of physical research materials, such as cell lines or compounds, and often contains clauses about ownership of any resulting inventions. A confidentiality or non-disclosure agreement allows the invention to be discussed with a company without destroying patent rights, though confidential discussions should still be kept to the minimum needed. When the relationship does break down, the most useful escalation routes are usually internal: the director of the office, a faculty committee on intellectual property, or the vice-president or pro-vice-chancellor responsible for research. Universities vary a great deal in how responsive they are, and in some countries governments have pressed them to be faster and more generous with spinouts. But the inventor who arrives with a clear account of what they want, why, and what evidence supports it will usually get further than one who arrives with a grievance. The office's first real decision concerns whether to protect the invention at all, and on what terms. That decision turns on patent law, which is the subject of the next chapter. Chapter 3: Patents for Scientists A patent is a strange instrument, and scientists often misunderstand it in ways that cost them dearly. It is not a certificate of scientific merit, and it does not give its owner the right to make or sell anything. It is a time-limited right to exclude others from making, using, selling or importing the claimed invention in the country that granted it, in exchange for a public description of how the invention works. The bargain is disclosure for exclusivity: the inventor teaches the public, and in return the state lets them stop others from using the teaching for about twenty years. For a deep-tech venture, that right to exclude is often the single most important asset in the company. It is what allows an investor to believe that if the technology works, a larger competitor cannot simply copy it after the start-up has spent years and many millions of dollars proving it. Without that belief, the long and expensive work of crossing the valley is very hard to finance. Understanding patents is therefore not a legal nicety for an academic founder. It is part of understanding the business. What can be patented Most patent systems require that an invention be patentable subject matter, new, inventive and capable of industrial application or, in US terms, useful. It must also be described clearly and completely enough that a skilled person could carry it out. The subject-matter requirement excludes certain categories. Laws of nature, natural phenomena and abstract ideas cannot be patented as such in the United States, and the European Patent Convention excludes discoveries, scientific theories, mathematical methods and computer programs "as such", along with methods of treatment of the human body. These exclusions matter a great deal to academic inventors, because much academic research consists precisely of discovering natural phenomena. A series of US Supreme Court decisions sharpened the line. In Mayo v. Prometheus in 2012, the Court held that a method of adjusting drug dosage based on the level of a metabolite in the blood was an unpatentable application of a natural law. In Association for Molecular Pathology v. Myriad Genetics in 2013, it held that naturally occurring DNA sequences, even when isolated, could not be patented, although synthetic complementary DNA could. In Alice v. CLS Bank in 2014, it tightened the treatment of software and business methods. The practical consequence for academic founders, especially in diagnostics and computational methods, is that a discovery about how nature works must usually be tied to a specific, concrete application that goes beyond the discovery itself. Novelty means the invention has not been disclosed to the public anywhere in the world before the relevant date, in any form: a paper, a poster, a thesis on a library shelf, a conference talk, a preprint, a website, a product on sale, even a sufficiently detailed conversation with someone not bound by confidentiality. Inventive step, or non-obviousness, means that the invention would not have been obvious to a person skilled in the field, given everything that was already known. Much patent prosecution consists of arguing about this second test. The publication problem The single most damaging habit academic inventors bring to patenting is publishing first. It is also the most understandable, because publication is how academic careers are built, and delaying a paper can mean being scooped. The rules differ between jurisdictions in a way that matters enormously. The European Patent Convention, like many national systems, applies what is called absolute novelty: any public disclosure before the filing date counts as prior art, including the inventor's own. There are narrow exceptions, for instance for disclosures made in breach of confidence or at certain officially recognised international exhibitions, but a journal article, conference talk or preprint published before filing will generally destroy the right to a European patent. The United States is more forgiving. Since the America Invents Act moved the country to a first-inventor-to-file system for applications filed from 16 March 2013, US law has provided a one-year grace period for disclosures made by the inventor, or by others who obtained the information from the inventor. An inventor who publishes can therefore still file a US application within twelve months. Japan, South Korea, Canada, Australia and several other countries also have grace periods, though their terms differ. But relying on a grace period is dangerous. It preserves rights only in the countries that offer it, which often excludes Europe and China, and it leaves the inventor exposed if someone else publishes or files on a related idea in the meantime. A grace period is a safety net for mistakes, not a strategy. The sensible practice is to file first and publish second. Once a priority application has been filed, the inventor can usually publish immediately, since the filing date fixes the novelty date for everything described in it. The delay needed is often only a few weeks, provided the inventor warns the transfer office early. The damage comes from last-minute surprises: an abstract submitted for a conference in three weeks' time, a thesis defence open to the public, a preprint posted on impulse. Two further traps deserve mention. First, what counts as a public disclosure is broad. A poster at an internal departmental day with outside visitors, a grant application later published in summary, a slide deck posted online, or a talk to an industry audience without a confidentiality agreement may all qualify. Second, the priority application protects only what it actually describes. If the inventor files on one version of the invention and then publishes additional data or variations not included in the filing, those additions are unprotected. The filing must be complete enough to support the claims that will eventually matter. The anatomy of a patent A patent document has two parts that matter most: the description, sometimes called the specification, and the claims. The description explains the invention, its background and how to carry it out, with examples. It must be sufficient for a skilled person to reproduce the invention without undue experimentation; in US terms, it must satisfy the enablement and written description requirements. This is where academic data become important. Working examples, experimental results and a range of embodiments give the patent attorney material to support broad claims. A thin description, based on a single experiment, supports only narrow claims. The claims define the legal scope of the right. They are numbered sentences, written in a stylised language, each setting out a combination of features. Anything that includes every feature of a claim infringes it; anything that lacks even one feature does not. The first claim is usually the broadest, and later claims add features that narrow the scope, providing fallback positions if the broad claim is found invalid. Scientists often read a patent by looking at the description, which resembles a paper. Investors and lawyers read it by looking at the claims, because the claims are the property. A patent with a brilliant description and narrow claims may be commercially worthless, since a competitor can design around it by changing one feature. A patent whose claims cover the only practical way to achieve a commercially important result can underpin an entire industry. Claims come in different types. Composition or product claims cover a thing, such as a molecule, a material or a device. Method claims cover a process of making or using something. Use claims, available in some jurisdictions, cover a new use of a known substance, such as a new medical indication for an existing drug. Composition claims are usually the most valuable, because they cover the product however it is made or used, but they are also the hardest to obtain when the thing itself is already known. The filing sequence Patent filing is a sequence of decisions spread over several years, each involving more cost. Academic inventors should understand the sequence because the key decisions arrive at predictable times and because the costs escalate sharply. Table 2 sets out the typical route an international patent family takes, using the common pattern of a US provisional or national priority filing followed by an international application under the Patent Cooperation Treaty. Table 2. A typical international filing sequence. Stage Timing from first filing What it does Relative cost Priority filing (e.g. US provisional or UK application) Month 0 Fixes the priority date; not examined in the provisional case Low International (PCT) application By month 12 Preserves the option to file in most countries; search report and opinion Moderate International publication Around month 18 Application becomes public None directly National and regional phase entry Around month 30 (31 at some offices) Separate applications in chosen countries or regions High, rising with translations Examination and grant Often 3 to 6 years from filing Claims negotiated with each patent office High, ongoing Maintenance and renewal Through expiry, usually 20 years from filing Keeps each patent in force Accumulating The first filing is cheap relative to what follows. A US provisional application can be filed for a modest official fee and is not examined; it expires after twelve months, and its only purpose is to establish a priority date for a later full application. Many universities file provisionals liberally, sometimes drafted largely from the inventor's manuscript, as a way to preserve rights before publication. That is better than nothing, but a provisional that simply attaches a paper may not support the claims that later matter, and the priority date only protects what is actually described. The twelve-month deadline is the first major decision point. By then the office must decide whether to file a full application, usually an international application under the Patent Cooperation Treaty, which is administered by the World Intellectual Property Organization and covers over 150 contracting states. The PCT application does not itself produce an international patent; no such thing exists. It preserves the right to seek patents in member countries and provides an international search report and a preliminary opinion on patentability, which is valuable information about the strength of the claims. The second major decision point comes at about thirty months from the priority date, when the application must enter the national or regional phase in each country or region where protection is wanted. This is where costs rise steeply, because each office charges its own fees, many require translations, and each will examine the application separately with local attorneys. A typical decision might be to enter the United States, the European Patent Office, Japan and China, and perhaps a few other markets, depending on where the product will be made and sold. The European Union's Unitary Patent system, which began operating in June 2023 together with the Unified Patent Court, has changed the European step. An applicant who obtains a European patent can now request unitary effect covering the participating EU member states in a single right, rather than validating the patent country by country, and can enforce it in a single court. Not all European countries participate, and the system introduces the risk that a single central challenge could revoke protection across all of them at once, but it has made broad European coverage simpler and, for many applicants, cheaper. Over the life of a patent family, the accumulated cost of filing, prosecution, translation and maintenance across several major jurisdictions commonly reaches the low hundreds of thousands of dollars or more. That is why university offices triage so carefully at the twelve and thirty month points, and why a licensee or spinout is usually expected to take over the costs once a licence is signed. Priority, competition and interference Because most of the world grants patents to the first to file, timing can decide everything when several groups are working on the same problem. The most famous recent example is CRISPR gene editing. The University of California, representing Jennifer Doudna, Emmanuelle Charpentier and colleagues, filed a priority application in May 2012 describing the CRISPR-Cas9 system. The Broad Institute and the Massachusetts Institute of Technology, representing Feng Zhang's group, filed later in 2012 but claimed the use of the system in eukaryotic cells and obtained US patents first through an accelerated examination route. Because the Broad's applications were filed before the America Invents Act changed the rules, the dispute proceeded under the old first-to-invent system, as a series of interference proceedings about who had conceived the invention first. The US Patent Trial and Appeal Board ruled for the Broad on the eukaryotic claims in 2022, and in 2025 the Federal Circuit sent part of that decision back to the board for further consideration of how conception should be assessed. The European picture developed separately, with its own oppositions and revocations. The lesson for academic inventors is not that they should expect a decade of litigation. Most inventions are never contested at all. It is that the filing date and the precise wording of claims can matter as much as the science, and that in a crowded field a few weeks' delay, or a priority document that failed to describe a key embodiment, can determine who owns a platform technology. Freedom to operate A patent gives its owner the right to exclude others. It does not give the owner the right to practise the invention, because doing so may infringe someone else's patent. A new drug formulation may be patented, but if the drug molecule itself is covered by another company's composition patent, the formulation cannot be sold until that patent expires or a licence is obtained. Freedom to operate is the question of whether a product can be made and sold without infringing others' valid patents in the countries that matter. It is a separate question from patentability, and it is answered by a separate analysis: a search of patents in force, followed by legal review of the claims against the planned product. Academic inventors rarely think about it, because research use enjoys some limited protection in some jurisdictions, and because a laboratory is not selling anything. Investors think about it constantly, because a spinout that discovers a blocking patent after raising money may have to license it on unfavourable terms, redesign the product or stop. A full freedom-to-operate opinion is expensive and is usually commissioned only when a product design is reasonably settled. But an early landscape search, identifying the main patent holders in the field and any obvious blocking rights, is cheap relative to the cost of discovering a problem late, and it helps the inventor design around obstacles while the technology is still flexible. When patents are the wrong tool Patents are not always the best form of protection. For some inventions, especially manufacturing processes that cannot be discovered by examining the final product, keeping the method secret may protect it longer and more cheaply than patenting it, since a trade secret does not expire as long as it stays secret, and a patent publishes the method for competitors to study. For software and data-driven methods, patents may be hard to obtain after the US decisions described earlier, and speed, data and know-how may matter more. For some research tools, a university may decide that open licensing or no patent at all will serve the public better and create more value than exclusivity. Most deep-tech ventures end up protected by a combination: core patents on the key composition or device, further patents on improvements and applications, trade secrets around manufacturing, and accumulated know-how in the team. The patent, however, remains central for most of them, because it is the one form of protection that can be transferred cleanly by licence, valued by an investor and enforced against a well-funded competitor. How a single filing grows into a portfolio, and how that portfolio moves from the university into a company, is the subject of the next chapter. Hashtags: #ResearchCommercialization #AcademicInnovation #DeepTechSpinouts #TechnologyTransfer #ValleyOfDeath #AcademicPatents #IntellectualProperty #BayhDoleAct #InventionDisclosure #PatentStrategy #PatentPortfolio #FreedomToOperate #TechnologyLicensing #SpinoutLicensing #TranslationalFunding #NonDilutiveFunding #TechnologyReadinessLevels #ProofOfConcept #DeepTechEntrepreneurship #FounderEquity #VentureCapital #TermSheets #ConflictOfInterest #UniversitySpinouts #FutureOfResearchCommercialization