Download the Book (PDF): Introduction In the autumn of 1936 the Literary Digest, then one of the most widely read magazines in the United States, mailed roughly ten million straw-poll ballots to Americans whose names it had gathered from its own subscriber lists, telephone directories, and automobile registrations. Something over two million came back. On the strength of that enormous return the magazine predicted that Alf Landon would defeat Franklin Roosevelt comfortably. R

Download the Book (PDF): Introduction Every analyst who has opened a large public survey file has met the same small crisis. The file arrives with thousands of respondents, hundreds of questions, and a handful of columns whose purpose is not obvious: a final weight, a stratum code, a primary sampling unit code, and perhaps eighty or a hundred and sixty further columns labelled as replicate weights. The documentation says these must be used. The deadline says otherwise. So the

Download the Book (PDF): Introduction Every systematic review rests on a claim that is rarely stated aloud: that the studies it analyses are the studies that exist. A meta-analysis can use flawless statistics, a risk-of-bias assessment can be performed in duplicate by trained reviewers, and a GRADE table can be assembled with care, but if the search that fed them missed a third of the relevant trials, every downstream number is an estimate of something other than what the rev

Download the Book (PDF): Introduction A systematic review is a strange kind of scientific study. It has no participants of its own, collects no new measurements, and runs no experiment. Its raw material is other people's published and unpublished work, and its central act is a search. Yet it sits at the top of most evidence hierarchies, gets cited more than almost anything else, and is routinely used to decide what treatments health systems will pay for, what advice clinician

Download the Book (PDF): Introduction Somewhere in a mobile examination trailer parked outside a county fairground, a technician draws blood from a volunteer, labels the tube, and logs the time. Somewhere in a Swedish tax office, a clerk updates an address. In a warehouse in Stockport, a robot retrieves one of several million frozen aliquots so that a machine in another building can count five thousand proteins in it. In 1950, an enumerator in rural Georgia wrote down the occ

Download the Book (PDF): Introduction A researcher submits a paper. The regression table is clean, the coefficient on the variable of interest is negative, and the asterisks are there: p < 0.01. A reviewer asks whether the relationship is causal. The author replies that the model controls for age, sex, income, education, region, and a quadratic in experience. The reviewer, satisfied that a serious effort has been made, recommends acceptance. Six years later a different team,

Download the Book (PDF): Introduction Every year, thousands of published documents announce that a group of experts has agreed. A panel of clinicians agrees on which outcomes every trial in a disease should measure. A group of public health specialists agrees on what governments should do next in a pandemic. A committee of speech therapists, psychologists and paediatricians agrees on what to call children whose language does not develop as expected. Very often, the phrase tha

Download the Book (PDF): Introduction In 2006 the structural biologist Geoffrey Chang and his colleagues retracted five papers, three of them from Science. The papers described the three-dimensional structures of membrane transport proteins, and they had been cited hundreds of times. Other laboratories had built on them, designed experiments around them, and in some cases struggled to reconcile their own data with them. The cause of the collapse was not fraud, not contaminate

Download the Book (PDF): Introduction Most scientists meet grant accounting the way travellers meet customs officers: briefly, reluctantly, and with a vague sense that something could go wrong without their quite knowing what. The proposal is written in a rush of scientific ambition, the budget is assembled in the last seventy-two hours before the deadline by a department administrator working from a spreadsheet template, and the award notice arrives months later carrying a s

Download the Book (PDF): Introduction Every structural equation model is an argument about where covariance comes from. That sentence is the whole of this book compressed, and most of the trouble researchers get into with the method comes from forgetting it. Consider what actually happens when a researcher fits a model in AMOS or lavaan. She has a data set: perhaps four hundred teachers who answered twenty questionnaire items about workload, emotional exhaustion, collegial su

Download the Book (PDF): Introduction In the spring of 2022 a neurologist at Vanderbilt University named Matthew Schrag, working on a separate question entirely, began looking closely at the Western blots in a group of papers on amyloid beta, the protein at the centre of the dominant theory of Alzheimer's disease. Among them was a paper published in Nature in 2006 that described a specific assembly of the protein, called Aβ56, and reported that it impaired memory in rats. The

Download the Book (PDF): Introduction In the late 1980s residents of West Harlem began organizing against the environmental burdens their neighborhood had been asked to carry: a sewage treatment plant on the Hudson waterfront, and a disproportionate share of Manhattan's diesel bus depots. Northern Manhattan's children were being hospitalized for asthma at rates far above the city average. Residents did not need a study to tell them the air was bad; they could smell it and the