Download the Book (PDF): Introduction Every structural equation model is an argument about where covariance comes from. That sentence is the whole of this book compressed, and most of the trouble researchers get into with the method comes from forgetting it. Consider what actually happens when a researcher fits a model in AMOS or lavaan. She has a data set: perhaps four hundred teachers who answered twenty questionnaire items about workload, emotional exhaustion, collegial support, and intention to leave the profession. From those responses the software computes a covariance matrix, a square grid of numbers describing how each item moves with every other item. Twenty items produce 210 unique variances and covariances. That matrix is the evidence. Everything else, the latent constructs, the arrows, the standardized coefficients printed to three decimal places, the indirect effect with its bootstrapped confidence interval, is a hypothesis about how that matrix came to be. The researcher's diagram says, in effect: these four items share variance because they all reflect a common cause called exhaustion; these five share variance because they reflect support; exhaustion depends on workload and is buffered by support; intention to leave depends on exhaustion. Once the diagram is drawn and the parameters estimated, the model implies a covariance matrix of its own, the pattern of associations that would appear if the story were exactly true. Estimation chooses parameter values that make the implied matrix as close as possible to the observed one. Fit indices summarize how close it came. Modification indices point to where it fell short. Seen this way, several things that confuse newcomers become clearer. A model can fit well and still be wrong, because many different stories can generate the same covariance matrix. A model can fit badly for reasons that have nothing to do with the structural hypotheses the researcher cares about, because most of the matrix's cells concern relationships among indicators of the same construct. A significant indirect effect does not show that mediation occurred, because covariances carry no information about temporal order. And a fit index crossing a published threshold does not certify anything, because the threshold was derived under conditions that may not resemble the researcher's model at all. Why the method is worth the effort None of this is an argument against structural equation modeling. It is an argument for using it with a clear head, because what the method offers is real and not easily obtained any other way. Its first gift is the separation of measurement from theory. Psychological, educational, and organizational research deals almost entirely in constructs that cannot be observed directly: self-efficacy, reading comprehension, transformational leadership, organizational commitment, anxiety. We observe responses to items, scores on tests, ratings by supervisors, and we treat them as imperfect windows onto the constructs. Ordinary regression on summed scale scores pretends the windows are clear. Measurement error in predictors then biases coefficients, usually toward zero for a single predictor and unpredictably when several correlated predictors are involved. A latent variable model estimates the error explicitly and relates the constructs to one another as if they had been measured without it. The gain is not cosmetic. Two constructs that correlate at .45 as observed scale scores can correlate at .60 or more once unreliability is removed, and a regression coefficient that looked trivial can look substantial. Its second gift is the ability to test a whole theory at once. A regression answers one question about one outcome. A structural model lets a researcher write down a system: workload affects exhaustion, exhaustion affects commitment and turnover intention, support moderates the first path. The system has testable consequences beyond its individual coefficients, because the paths it omits are claims too. A model that says workload affects turnover intention only through exhaustion predicts a particular pattern among the three constructs, and the data can contradict it. Its third gift is flexibility. The same framework handles confirmatory factor analysis, path analysis, mediation, multiple-group comparison, measurement invariance, latent growth curves, and latent interactions. Researchers who learn the logic once can apply it to a remarkable range of designs. Why the method goes wrong The same flexibility makes structural equation modeling easy to misuse. It is possible, with a few clicks in AMOS or a few lines of lavaan syntax, to produce output that looks authoritative and means very little. The recurring failures are well documented in the methodological literature and in any reviewer's memory. Researchers treat fit indices as pass marks, reporting that a model "fit well" because its CFI exceeded .95 and its RMSEA fell below .06, without asking whether those cutoffs were designed for models like theirs. They add correlated errors suggested by modification indices until the numbers improve, then report the final model as if it had been hypothesized in advance. They estimate mediation models on cross-sectional survey data and describe the resulting indirect effects in causal language. They ignore the fact that alternative models, with arrows pointing the other way, would fit exactly as well. They bundle measurement problems and structural problems into a single fit statistic and cannot tell which kind of misfit they have. These are not failures of software. AMOS and lavaan both compute what they are asked to compute, correctly. They are failures of understanding what the numbers are answers to. What this book does This book is built around a single claim: that a structural equation model is a set of explicit hypotheses about covariance, and that every practical decision in the method, from how constructs are specified to how fit is judged to how misfit is repaired, becomes clearer once the researcher keeps that claim in view. The chapters follow the order in which a careful analysis actually proceeds. The first chapter develops the core logic: how a diagram implies a covariance matrix, what identification means, and why degrees of freedom are the currency of testability. The second treats path analysis with observed variables, the historical root of the method, and shows both what it can do and why measurement error limits it. The third and fourth chapters turn to latent constructs: first the conceptual work of specifying a measurement model and judging its reliability and validity, then the practical work of confirmatory factor analysis, including estimation choices for non-normal and ordinal data, missing data, and measurement invariance. The fifth and sixth chapters address the structural questions that draw most researchers to the method in the first place: mediation, moderation, and their combination. They take seriously both the statistical machinery and the causal assumptions the machinery cannot supply. The seventh chapter explains what each of the common fit indices measures, where the famous cutoffs came from, and why methodologists have spent a quarter century warning against using them mechanically. The eighth chapter is about diagnosis: how to locate the source of misfit using residuals and modification indices, how to respecify without deceiving oneself, and how to report what was done so that readers can judge it. Throughout, the book shows how the ideas map onto the two programs most widely used in the target fields. AMOS, distributed by IBM alongside SPSS, is built around drawing path diagrams and remains common in management, marketing, and education departments. lavaan, an open-source package for R created by Yves Rosseel and described in the Journal of Statistical Software in 2012, is built around a compact model syntax and has become the default in much of psychology. Neither is presented as superior. They estimate the same models by the same methods, and a researcher who understands the model can move between them. Where they differ in defaults or capabilities, the difference is noted, because defaults quietly shape results. The reader assumed here knows ordinary regression and the idea of a correlation, has perhaps run a factor analysis, and wants to use structural equation modeling responsibly in a thesis, a grant, or a journal article. No matrix algebra is required, though a few simple formulas appear where they illuminate what a statistic is doing. The goal is not to replace the standard textbooks, several of which are listed at the end, but to give a reader the judgment that makes those textbooks usable: a sense of what questions the method can answer, what it cannot, and how to tell the difference when the output is on the screen. Chapter 1. Covariance as Evidence The single most useful habit a structural equation modeler can develop is to look at a model diagram and ask: what pattern of associations does this picture predict? Everything in the method follows from the answer. Estimation is the search for parameter values that make the prediction match the data. Testing is the comparison of the prediction with the data. Identification is the question of whether the data contain enough information to pin the parameters down. This chapter builds that habit from the ground up. From arrows to associations Begin with the smallest interesting case. Suppose a researcher believes that parental involvement (X) raises a student's academic self-concept (M), which in turn raises end-of-year achievement (Y). She measures all three directly, say as standardized scale scores, and draws a chain: X points to M, M points to Y. There is no arrow from X to Y. In standardized form, the model has two path coefficients, call them a for X to M and b for M to Y. The rules for reading associations off such a diagram were worked out by the geneticist Sewall Wright in a series of papers beginning around 1918 and set out in his 1921 paper "Correlation and Causation." Wright's tracing rules say that the correlation between any two variables equals the sum, over every permissible route connecting them, of the products of the coefficients along each route. A permissible route may go backward along arrows and then forward, but not forward and then backward, and it may pass through each variable only once. Apply the rules. The correlation between X and M is a, since there is one route. The correlation between M and Y is b. The correlation between X and Y is the product a × b, since the only route runs from X through M to Y. So the model predicts three correlations from two parameters, and the third is not free: it must equal the product of the other two. That is a testable claim. If the observed correlation of X with M is .40 and of M with Y is .50, the model insists that X and Y correlate at .20. If they actually correlate at .35, something is wrong with the story. Perhaps parental involvement also affects achievement directly, through homework help or school choice, in addition to its effect through self-concept. The missing arrow was a hypothesis, and the data can reject it. Now add the direct path c′ from X to Y. The model has three parameters and three correlations to explain. It will reproduce them exactly, whatever the data are. The model is no longer wrong in any detectable way, which also means it is no longer testable. It has become a reparameterization of the correlation matrix, a way of redescribing the data rather than making a claim about them. This small example contains the three ideas that organize the rest of the method. A model implies a pattern of associations. The pattern is testable only to the extent that the model has fewer free parameters than there are associations to explain. And the omitted arrows, not the drawn ones, are where the testable content lives. The model-implied covariance matrix Real models are larger, and they work with covariances rather than correlations, so that variables retain their original scales. But the logic is identical. Given a model and a set of values for its parameters, one can compute the covariance matrix the model implies. Methodologists write this as Σ(θ), the covariance matrix Σ expressed as a function of the parameter vector θ. The observed sample covariance matrix is S. Estimation finds the values θ̂ that make Σ(θ̂) as close as possible to S, according to some definition of "close." The most common definition is maximum likelihood. Under the assumption that the observed variables follow a multivariate normal distribution, maximum likelihood chooses the parameter values under which the observed data would have been most probable. In practice, the software minimizes a discrepancy function, usually written F, which is zero when the implied and observed matrices are identical and grows as they diverge. Both AMOS and lavaan use maximum likelihood by default for continuous data. Other estimators, discussed in Chapter 4, use different definitions of closeness that are better suited to non-normal or ordinal data. What matters conceptually is that the observed covariance matrix, along with the means if the model includes them, is the complete evidence base. With complete data and a normal-theory estimator, two data sets with identical covariance matrices and means will produce identical parameter estimates, identical standard errors, and identical fit statistics, regardless of whether they came from a randomized experiment, a longitudinal panel, or a single survey administered on one afternoon. The method does not know how the data were collected. Any causal interpretation must come from the design and from the researcher's substantive knowledge, not from the covariances themselves. This point is easy to agree with and hard to remember. It will return in every later chapter. Counting information: degrees of freedom With p observed variables, the covariance matrix contains p(p + 1)/2 unique elements: p variances on the diagonal and p(p − 1)/2 covariances off it. Five variables give 15 elements; ten give 55; twenty give 210. If the model also estimates means, add p more. The model's degrees of freedom are the number of unique elements minus the number of freely estimated parameters. When degrees of freedom are positive, the model is overidentified: it makes more predictions than it has parameters, so it can fail. When they are zero, the model is just-identified: it reproduces the data perfectly and cannot be tested as a whole, though its individual parameters can still be estimated and tested. When they are negative, the model is underidentified: there are more unknowns than pieces of information, and no unique solution exists. Degrees of freedom are the currency of testability. Each one represents a constraint the model imposes on the data, a place where it says "this covariance must equal that combination of parameters," and each constraint is an opportunity for the data to disagree. A researcher who adds a correlated error term to improve fit spends a degree of freedom. The fit improves, necessarily, but the model now says less. A model with many degrees of freedom that still fits well has survived many tests. A model with one or two has survived very few. This is why the phrase "the model fit the data" must always be read alongside the degrees of freedom. A just-identified path model always fits perfectly and proves nothing by doing so. A confirmatory factor model with 200 degrees of freedom and acceptable fit has passed a demanding examination, though, as later chapters will show, not necessarily the examination the researcher cared about. Identification Counting is necessary but not sufficient. A model can have positive degrees of freedom overall and still contain parameters that the data cannot determine. Identification asks whether each free parameter has a unique value implied by the population covariance matrix. If two different parameter values would produce the same implied matrix, no amount of data could tell them apart, and the parameter is not identified. The most common identification problem in practice concerns the scale of latent variables, which is taken up in Chapter 3. A latent variable has no natural units. Is exhaustion measured on a scale from zero to one, or zero to a hundred? The data cannot say. So the modeler must fix the scale, either by setting one factor loading to 1.0, which gives the latent variable the units of that indicator, or by setting the latent variance to 1.0, which standardizes it. AMOS does the first automatically when a latent variable is drawn with its indicators; lavaan's cfa() and sem() functions also fix the first loading of each factor to one by default, and the argument std.lv = TRUE switches to fixing the variances instead. The two choices produce identical fit and equivalent solutions; they differ only in the units in which the estimates are expressed. Other identification problems are subtler. A factor with only two indicators is not identified on its own; it can be estimated only if it correlates with other factors in the model, and even then the solution may be unstable. A single-indicator factor requires the researcher to fix the indicator's error variance to some value, usually derived from a known reliability. Nonrecursive models, which contain feedback loops or reciprocal paths, require instrumental variables that affect one variable in the loop but not the other, a condition that must be justified substantively. Models with correlated errors between indicators that load on different factors can become unidentified in ways that no simple rule detects. Two practical warnings follow. First, software will sometimes produce estimates for a model that is empirically underidentified: formally identified, but with data that leave some parameter almost undetermined. Symptoms include enormous standard errors, warnings about a non-positive-definite information matrix, and estimates that swing wildly when the model is changed slightly. lavaan issues explicit warnings in such cases; AMOS reports that the model is probably unidentified and may suggest constraints. These warnings deserve to be read rather than suppressed. Second, identification is a property of the model and the population, not of any particular estimation run. A researcher who is unsure whether a novel model is identified can fit it to a covariance matrix generated from known parameter values and check whether the software recovers them. Writing the model down A structural equation model can be expressed as a diagram, as a set of equations, or as program syntax. The three are equivalent, and fluency in moving among them protects against errors. The diagram conventions are nearly universal. Observed variables appear as rectangles, latent variables as ovals or circles. A single-headed arrow represents a directional effect, a regression coefficient or factor loading. A double-headed curved arrow represents a covariance between exogenous variables or between error terms. Each endogenous variable, meaning any variable with an arrow pointing into it, has an error or disturbance term, usually drawn as a small circle with an arrow into the variable. In AMOS the diagram is the model: the researcher draws it on a canvas with the program's tools, and AMOS translates the drawing into equations. This makes the method visually accessible, but it also means that omissions are easy to miss. An error term left undrawn is an error variance fixed to zero, a strong and usually unintended claim. lavaan takes the opposite route. The model is typed as a short block of text, and the diagram, if wanted, is drawn afterward by a separate package such as semPlot. The syntax uses a handful of operators that map directly onto diagram elements, and Table 1 sets out the correspondence along with how the same element is created in AMOS. Table 1. Core model elements in diagrams, lavaan syntax, and AMOS. Element Diagram lavaan operator Example AMOS action Factor loading Arrow from oval to rectangle =~ exhaust =~ e1 + e2 + e3 Draw latent variable with indicators Regression path Arrow between variables ~ quit ~ exhaust + support Draw single-headed arrow Covariance or variance Curved two-headed arrow ~~ e1 ~~ e2 Draw double-headed arrow Intercept or mean Not usually drawn ~ 1 quit ~ 1 Enable means and intercepts Defined parameter Not drawn := ind := a*b User-defined estimand plugin Parameter label Label on arrow name* exhaust ~ a*workload Name the parameter in its properties One practical consequence of the difference in interfaces is that lavaan makes certain defaults invisible in a different way than AMOS does. When the sem() function is used, lavaan automatically estimates residual variances for all observed and latent endogenous variables and, by default, allows exogenous latent variables to covary. An AMOS user must draw the curved arrows between exogenous latent variables explicitly; an omitted one is a covariance fixed at zero. Researchers moving between programs should always check the full parameter list in the output, which both programs provide, rather than trusting that the model estimated is the model intended. What the estimates mean Once estimated, the model yields several kinds of numbers, and it pays to be precise about what each represents. Unstandardized coefficients are in the original units. A path of 0.35 from workload to exhaustion, with both measured on five-point scales, means that a one-point increase in workload is associated with a 0.35-point increase in expected exhaustion, holding the other predictors of exhaustion constant. Standard errors and significance tests are computed for the unstandardized estimates. Standardized coefficients rescale everything to unit variance. They make paths comparable within a model, and for factor loadings they indicate how strongly each indicator reflects its factor. But standardized coefficients depend on the variances in the particular sample, so they are poor guides for comparing groups or studies whose variances differ. When a researcher wants to say that an effect is stronger among novice teachers than experienced ones, the comparison should be made on unstandardized coefficients with the latent scales anchored equivalently in both groups, a requirement discussed under measurement invariance in Chapter 4. The squared multiple correlation, R², for each endogenous variable gives the proportion of its variance accounted for by its predictors in the model. For an indicator of a latent factor, R² is the proportion of that item's variance attributable to the factor, a quantity sometimes called item reliability. Error variances are estimated parameters, and they should be inspected. A negative error variance, known as a Heywood case, is impossible in a population and signals a problem: a misspecified model, an indicator that is nearly redundant with another, a factor with too few indicators, or a small sample. Heywood cases are not to be patched silently by fixing the variance to a small positive number; they are symptoms, and the underlying cause needs to be found. A model is an argument It may seem that this chapter has dwelt on elementary mechanics. But the mechanics carry a conceptual point that shapes everything later. A structural equation model makes claims of two kinds. It claims that certain parameters are nonzero, and those claims are tested by the significance of individual estimates. It also claims that certain parameters are zero, the omitted arrows, and those claims are tested by overall fit. Researchers tend to focus on the first kind, because the paths they drew represent their hypotheses. But the second kind is where the model earns its right to be called confirmatory. A model in which every variable is connected to every other says nothing that the correlation matrix did not already say. The model is also an argument about direction, and here the covariance matrix is silent. The chain from parental involvement to self-concept to achievement implies exactly the same correlations as the chain running in reverse, from achievement to self-concept to involvement, and as a model in which self-concept is a common cause of the other two. All three models have one degree of freedom and all three predict that the correlation between the endpoints equals the product of the other two. They are equivalent models, observationally indistinguishable, and no fit statistic can choose among them. The choice must rest on theory, on the design of the study, and on knowledge that lies outside the data. Chapter 8 returns to equivalent models, because they are the most frequently ignored threat to interpretation in published structural equation modeling. The remaining chapters take these ideas into increasingly realistic territory: first path models with observed variables, then latent constructs, then the structural relationships among them, then the judgment of fit and the diagnosis of failure. In each, the question to hold in mind is the one this chapter began with. What pattern of associations does this model predict, and what, exactly, would count against it? Chapter 2. Path Analysis with Observed Variables Structural equation modeling began without latent variables. Sewall Wright developed path analysis to study inheritance in guinea pigs and the determinants of birth weight and bone size in livestock, using measured variables and diagrams of hypothesized influences. The method passed into economics as simultaneous equation modeling and into sociology in the 1960s, where Otis Dudley Duncan's work on status attainment made path diagrams a standard way of representing how family background, education, and occupation link across a life. Only in the early 1970s, with Karl Jöreskog's development of the LISREL model and program, were path analysis and factor analysis joined into the general framework now called structural equation modeling. Path analysis with observed variables remains useful in its own right. Many research questions in education and management involve variables measured well enough, or by single indicators unavoidably, that a latent variable model would add complexity without benefit: grade point average, years of experience, firm size, number of absences, test scores from standardized instruments with published reliabilities. More important for this book, path analysis is where the logic of direct, indirect, and total effects is easiest to see, and where the limits of observed-variable models become clear enough to motivate everything that follows. Systems of regressions A path model is a set of regression equations estimated simultaneously. Each endogenous variable is regressed on the variables that point into it. Consider a model from organizational research: transformational leadership (L) is hypothesized to raise employees' psychological empowerment (E), which in turn raises both job satisfaction (S) and discretionary effort, often labelled organizational citizenship behavior (C). Satisfaction also affects citizenship behavior directly. Leadership has no direct path to satisfaction or citizenship behavior. Written as equations, the model says E depends on L; S depends on E; and C depends on E and S. Each equation has its own disturbance term, representing everything that affects the outcome but is not in the model. If the disturbances are assumed to be uncorrelated, and the model is recursive, meaning there are no feedback loops and all causal flow runs one way, the path coefficients could be estimated by running three separate ordinary regressions. The estimates would be the same as those from maximum likelihood estimation of the system. What the simultaneous approach adds is the overall test: the model omits two paths, L to S and L to C, and it assumes that the disturbances are uncorrelated. Four observed variables provide ten variances and covariances; the model estimates one exogenous variance, four path coefficients, and three disturbance variances, eight parameters in all, leaving two degrees of freedom, one for each omitted path. The chi-square test with two degrees of freedom asks whether those omissions are consistent with the data. This is worth dwelling on because it illustrates what "confirmatory" means. The researcher's theory is that leadership works entirely through empowerment. The two missing arrows are the theory. If the data show that leadership predicts satisfaction even after empowerment is accounted for, the chi-square will register it, and the modification index for the L to S path will be large. The model has made a falsifiable claim. Direct, indirect, and total effects Path models distinguish among three kinds of effect, and the distinction is the foundation of mediation analysis. The direct effect of one variable on another is the coefficient on the arrow connecting them, if there is one. In the leadership model, the direct effect of empowerment on citizenship behavior is the coefficient on the E to C path. An indirect effect runs through one or more intervening variables and equals the product of the coefficients along the route. Empowerment affects citizenship behavior indirectly through satisfaction: the E to S coefficient multiplied by the S to C coefficient. Leadership affects citizenship behavior through two indirect routes: L to E to C, and L to E to S to C. Each has its own product. The total effect is the sum of the direct effect and all indirect effects. The total effect of leadership on citizenship behavior, in this model, is the sum of the two indirect products, since there is no direct path. Both AMOS and lavaan compute these quantities. In AMOS, the "Indirect, direct and total effects" option in the Analysis Properties dialog produces tables of all three; to test a specific indirect effect when a variable has several, AMOS requires a user-defined estimand, since its default indirect effect sums all routes. In lavaan, the researcher labels the relevant paths and defines each indirect effect explicitly with the := operator, for example writing ind1 := a*b after labelling the paths a and b. The explicit approach has a virtue: it forces the researcher to state which indirect effect is of interest, rather than accepting an aggregate that may combine theoretically distinct mechanisms. The decomposition of effects is algebraically straightforward. Its interpretation is not, and Chapter 5 is devoted to the difference. For now, note that the words "direct" and "indirect" are relative to the model. A direct effect is whatever part of the association is not carried by the mediators included in the model. Add another mediator, and part of the former direct effect becomes indirect. The direct effect is a residual, not a mechanism. Assumptions that the diagram hides A path diagram looks like a description of how the world works. What it actually encodes is a set of statistical assumptions, some of them strong, that the researcher must be prepared to defend. The first is that the disturbances of different endogenous variables are uncorrelated unless the model says otherwise. This assumption means that no omitted variable affects two endogenous variables at once. In the leadership model, if an unmeasured factor such as an employee's general positive affect raises both empowerment and satisfaction, then the disturbances of E and S are correlated, and the estimated E to S path absorbs a spurious component. The coefficient will be too large, and the indirect effect of leadership through satisfaction will be overstated. This is the omitted-confounder problem that afflicts all observational research, but path diagrams make it easy to forget. The diagram shows arrows the researcher believes in; it does not show the unmeasured common causes the researcher has assumed away. A useful discipline, when drawing any path model, is to ask of every pair of endogenous variables: is there anything that could cause both of these that is not in the model? If the answer is yes, and it usually is, the estimates between them should be regarded as upper or lower bounds, not as effects. The second assumption is correct functional form: the relationships are linear and additive unless the model specifies otherwise. A path coefficient summarizes the average linear association. If empowerment raises satisfaction strongly at low levels and hardly at all at high levels, the coefficient will describe neither regime well. The third assumption is correct direction. The arrow from empowerment to satisfaction could as easily run the other way: satisfied employees may feel more empowered. With cross-sectional data, as Chapter 1 showed, models differing only in the direction of such arrows can fit identically. The fourth assumption, and the one that motivates latent variable modeling, is that the variables are measured without error. The problem of measurement error In ordinary regression, error in the outcome variable is harmless to the coefficients; it inflates standard errors but does not bias estimates. Error in a predictor is another matter. Classical measurement error in a single predictor attenuates its coefficient toward zero, by a factor equal to the predictor's reliability. If empowerment is measured with reliability .70, the estimated effect of empowerment on satisfaction will be, on average, about 70 percent of its true size. With several correlated predictors, the damage is less predictable. Suppose satisfaction and empowerment both predict citizenship behavior, and empowerment is measured with more error than satisfaction. The regression will understate the role of empowerment, and because the two predictors are correlated, some of the variance that truly belongs to empowerment will be credited to satisfaction. The coefficient for the better-measured predictor can be biased upward. In mediation models this has a specific consequence that has been demonstrated repeatedly: measurement error in the mediator attenuates the estimated indirect effect and inflates the estimated direct effect. A researcher who finds "partial mediation" may be looking at complete mediation observed through an unreliable measure. The standard reliability coefficient for scale scores, Cronbach's alpha, does not help as much as its ubiquity suggests. Alpha estimates reliability correctly only when all items measure the construct equally well, an assumption called tau-equivalence that rarely holds; otherwise it underestimates reliability of the sum score, and it can be distorted by correlated errors. More fundamentally, knowing the reliability does not remove the error from the analysis. It only quantifies it. A small calculation shows how large the distortion can be. Suppose that, in the population, empowerment correlates .50 with satisfaction and .30 with citizenship behavior, and satisfaction correlates .40 with citizenship behavior, with all three measured perfectly. Regressing citizenship behavior on empowerment and satisfaction gives standardized coefficients of about 0.13 for empowerment and 0.33 for satisfaction. Now suppose empowerment is measured with reliability .60, a figure not unusual for short scales, while satisfaction is measured with reliability .90. The observed correlations shrink: empowerment with satisfaction becomes about .37, and empowerment with citizenship behavior about .23. Rerunning the regression on these attenuated correlations gives roughly 0.11 for empowerment and 0.34 for satisfaction. The poorly measured predictor has lost about a fifth of its coefficient, and the well-measured one has gained slightly despite its own small measurement error, even though nothing about the true relationships has changed. In a larger model with more correlated predictors, the redistribution can be larger and can even change signs. There are two remedies within the observed-variable framework. The first is to correct the path model for known unreliability by specifying each scale score as the single indicator of a latent variable, fixing the indicator's loading to one and its error variance to (1 − reliability) × observed variance. This is a legitimate technique, particularly when a construct is measured by a well-established instrument whose reliability is known from large samples, and it is easily specified in either program. In lavaan, for a scale score with observed variance 0.80 and reliability .85, the error variance would be fixed at 0.12 by writing a line such as emp_score ~~ 0.12*emp_score alongside a single-indicator factor definition. In AMOS, the error variance is fixed by entering the value in the parameter's properties. The second remedy is to model the items directly, allowing the software to estimate each item's relationship to the construct and each item's error. That is the subject of Chapters 3 and 4, and it is the step that makes structural equation modeling distinctive. Nonrecursive models and reciprocal effects Most published path models are recursive. But some theories genuinely involve reciprocal influence: job satisfaction and job performance may each affect the other; students' motivation and achievement may reinforce one another. A model with a two-way arrow structure, satisfaction pointing to performance and performance pointing to satisfaction, is called nonrecursive. Such models are not identified without additional information. The standard solution uses instrumental variables: for each variable in the loop, at least one predictor that affects it directly but has no direct effect on the other variable in the loop. For the satisfaction and performance loop, one might use pay fairness as an instrument that affects satisfaction but not performance directly, and cognitive ability as an instrument that affects performance but not satisfaction directly. The quality of the estimates depends entirely on the validity of those exclusion claims, and they are often hard to defend. Weak instruments, those only modestly correlated with the variable they are meant to predict, produce unstable estimates with very large standard errors. In practice, reciprocal effects are better studied with longitudinal data, in which the influence of satisfaction at time one on performance at time two, and of performance at time one on satisfaction at time two, can be estimated in a cross-lagged panel model. Even these models have been criticized in recent years because the traditional cross-lagged panel model confounds stable between-person differences with within-person change. Hamaker, Kuiper, and Grasman's 2015 paper in Psychological Methods proposed the random-intercept cross-lagged panel model, which separates the two by giving each person a stable trait component, and it is now widely used for questions about reciprocal processes within individuals. Its lesson generalizes: a path coefficient in a panel model answers a question about some level of analysis, and the researcher must know which. Sample size and estimation in path models Path models with observed variables are relatively undemanding of sample size compared with latent variable models, since they estimate fewer parameters. A common heuristic calls for at least ten to twenty cases per estimated parameter, but heuristics of this kind have weak foundations. Sample size requirements depend on the size of the effects, the reliability of the measures, the number of parameters, the distribution of the data, and the purpose of the analysis. A simulation study by Wolf, Harrington, Clark, and Miller, published in Educational and Psychological Measurement in 2013, found that required sample sizes for structural equation models ranged from as few as 30 to more than 400 depending on such features, and that no single rule of thumb held across conditions. The more defensible approach is a power analysis for the specific parameters or fit test of interest, which can be carried out by simulation in lavaan or with dedicated tools, and which forces the researcher to state expected effect sizes in advance. Maximum likelihood assumes multivariate normality. With observed variables that are skewed, such as counts of absences or measures of rare behaviors, the parameter estimates remain consistent but the standard errors and chi-square statistic can be badly wrong. Robust corrections, discussed in Chapter 4, address this and are available in both programs, though they are easier to request in lavaan. AMOS's primary alternative for non-normal data is bootstrapping, which it implements well and which is valuable particularly for indirect effects, whose sampling distributions are not normal even when the variables are. What path analysis teaches Path analysis with observed variables is, in one sense, a set of regressions with a diagram attached. Its enduring value is conceptual. It teaches that a model is defined as much by its omitted paths as by its included ones; that effects decompose into direct and indirect components that depend on what else is in the model; that every arrow embeds assumptions about confounding and direction that the data cannot check; and that measurement error, in predictors and mediators especially, distorts exactly the coefficients researchers most want to interpret. That last lesson leads directly to latent variables. If the quantities of interest are constructs rather than scale scores, and the scale scores contain error, then the model should represent the constructs and their measurement explicitly. Doing so requires a measurement model, and specifying a measurement model well is harder, and more consequential, than most researchers expect. Chapter 3. Latent Constructs and the Measurement Model A latent variable is a claim that something unobserved explains why certain observed things go together. When a researcher says that eight questionnaire items measure teacher self-efficacy, the statistical content of that statement is precise: the eight items covary because each responds to the same underlying quantity, and once that quantity is held constant, the items should no longer covary at all. The latent variable is the common cause that accounts for the shared variance. What remains in each item after the common cause is removed is unique variance, a mixture of random error and item-specific content that has nothing to do with self-efficacy. That definition is worth stating plainly because the language of latent variables tempts researchers into a looser view, in which a construct is whatever a set of items happens to be labelled. The looser view produces measurement models that fit poorly for reasons the researcher cannot diagnose, and structural estimates that mean something different from what the construct's name suggests. The discipline of the measurement model begins with taking the common-cause claim seriously. The reflective model and its implications The standard latent variable model in structural equation modeling is the reflective, or common factor, model. Each indicator is written as a linear function of the factor plus an error: the item score equals an intercept, plus a loading times the factor, plus a unique term. The arrows run from the construct to the items. Variation in self-efficacy produces variation in the item responses, not the other way around. The model makes a strong and testable prediction, known as local independence: indicators of the same factor are uncorrelated once the factor is controlled. For a single factor with four indicators, this implies that the six covariances among the items are fully explained by four loadings and the factor variance. With the scale fixed, that is four free parameters to account for six covariances, and the model has two degrees of freedom. With three indicators the model is just-identified and cannot be tested in isolation. With two it is underidentified on its own. This is the source of the widely repeated advice that each factor should have at least three indicators, and preferably four or more: not a matter of reliability alone, but of whether the model's defining claim can be checked. Several consequences follow from the reflective model, and each is a useful test of whether it suits a given construct. Indicators should be interchangeable in principle. Dropping one indicator of a well-defined reflective construct should not change what the construct means, only how precisely it is measured. If removing an item would change the construct's content, the reflective model is probably wrong for it. Indicators should correlate positively with one another, after any reverse-coded items are recoded. If two items supposedly reflecting the same factor are uncorrelated, a single common cause cannot explain them. Indicators should relate to other variables in the same way, apart from differences in strength. If one self-efficacy item correlates strongly with years of experience and another not at all, the items are responding to different things. Formative constructs and composites Not every construct fits the reflective model. Socioeconomic status is often measured by income, education, and occupational prestige. It makes little sense to say that a person's latent status causes their income and their education; rather, income and education are components that together define status. A rise in income raises status without any expectation that education will follow. Such a construct is called formative, or, when the weights are fixed rather than estimated, a composite. Formative indicators need not correlate. Removing one changes the construct's meaning. Local independence does not apply. Forcing a formative construct into a reflective measurement model typically produces poor fit, low and uneven loadings, and a factor whose meaning drifts toward whichever indicator dominates the covariance structure. Formative constructs raise identification problems of their own in covariance-based structural equation modeling: a formatively measured latent variable is identified only if it emits paths to at least two other variables, and its meaning then depends partly on what those outcome variables are. A lively methodological debate over the past two decades has questioned whether formatively measured latent variables are coherent at all, with some methodologists arguing that such constructs are better treated simply as weighted composites of observed variables. The practical advice is conservative. If the construct is a sum or index of its components by definition, as with a checklist of stressful life events or a firm's portfolio of human resource practices, treat it as an observed composite and model it as such. Reserve latent variables for constructs whose indicators are plausibly effects of a common cause. A related tradition, partial least squares structural equation modeling, is widely used in management and information systems research and treats all constructs as weighted composites. It optimizes explained variance in outcome constructs rather than fit to the covariance matrix, and it has distinct software such as SmartPLS. Its relationship to covariance-based modeling has been a source of heated exchanges in the methods literature. This book concerns covariance-based modeling as implemented in AMOS and lavaan; researchers choosing between the approaches should recognize that they answer different questions and that the fit indices of later chapters do not transfer to partial least squares. Scaling the latent variable A latent variable has no inherent units, and the model cannot be estimated until some are assigned. Chapter 1 noted the two common choices. Fixing the first loading to one, the default in both AMOS and lavaan, gives the factor the metric of that marker indicator: a one-unit difference in the factor corresponds to a one-unit expected difference on the marker item. Fixing the factor variance to one, requested in lavaan with std.lv = TRUE, standardizes the factor and frees all loadings. The choice does not affect fit, but it affects interpretation and occasionally estimation. If the marker indicator is weakly related to the factor, the marker-variable approach gives a factor with a poorly defined scale and can produce estimation difficulties. In multiple-group models, fixing the factor variance to one in every group imposes equal variances, which may be untrue; the marker-variable approach, or fixing the variance only in a reference group, is then appropriate. A third option, effects coding, constrains the loadings of each factor to average one, so the factor takes on the average metric of its indicators; it is useful when a researcher wants latent means and variances expressed on the original response scale. Reliability, properly estimated The measurement model yields direct estimates of how well each indicator reflects its factor, and from these one can compute reliability for the construct as a whole. The standardized loading of an item, squared, is the proportion of its variance explained by the factor. An item with a standardized loading of .80 has 64 percent common variance; one with .50 has only 25 percent. Items with standardized loadings below about .40 contribute little and may be measuring something else, though whether to retain them depends on the construct's content and not only on the numbers. For the reliability of a composite, the model-based coefficient most often recommended is omega, associated with the psychometrician Roderick McDonald. For a unidimensional factor with uncorrelated errors, omega is the squared sum of the unstandardized loadings divided by that quantity plus the sum of the error variances. Unlike Cronbach's alpha, omega does not assume equal loadings. When loadings are unequal, as they almost always are, alpha underestimates the reliability of the sum score, though usually not by much; when errors are correlated, alpha can be misleading in either direction. Dunn, Baguley, and Brunsden made the case for moving from alpha to omega in the British Journal of Psychology in 2014, and the argument is now widely accepted. In lavaan, the companion package semTools provides reliability coefficients including omega from a fitted model through its compRelSEM() function. AMOS does not report omega directly; users compute it from the estimated loadings and error variances, which is simple arithmetic. A short example makes the difference concrete. Suppose four items measuring emotional exhaustion have standardized loadings of .85, .80, .60, and .45. Their error variances in standardized form are one minus the squared loadings: .28, .36, .64, and .80. The sum of loadings is 2.70, its square 7.29, and the sum of error variances 2.08, so omega is 7.29 divided by 9.37, about .78. Cronbach's alpha computed on the same items, assuming the model holds exactly, comes out close to .76. The difference is small here, as it usually is, but the two coefficients diverge more as loadings become more unequal. More instructive is what the loadings reveal that neither coefficient does. The fourth item shares only about 20 percent of its variance with the factor. It may be poorly worded, it may tap a different facet of exhaustion, or it may be responding to something else entirely. Omega and alpha both summarize the scale; the loadings show which item is carrying less of the construct and invite a look at its content. A related quantity, composite reliability, is computed from the standardized solution in the same way and appears frequently in management research. Along with it, researchers often report average variance extracted, the mean of the squared standardized loadings for a factor. Fornell and Larcker, in a 1981 paper in the Journal of Marketing Research, proposed that average variance extracted should exceed .50, meaning that the factor accounts for more of its indicators' variance than error does. That standard is widely cited and should be understood as a guideline, not a law; a construct measured by many moderately loading items can be reliable as a composite while falling short of it. Discriminant validity A measurement model with several factors makes a second kind of claim: that the constructs are distinct. If the estimated correlation between two latent factors approaches one, the data do not support treating them as different things, regardless of what they are called. Three approaches are common. The first compares a model in which the two factors are distinct with a model in which their correlation is fixed at one, or in which their indicators load on a single factor; a significant loss of fit when the constraint is imposed indicates that the constructs can be distinguished. With large samples, this test detects trivial differences, so it establishes that the factors are not identical but not that they are meaningfully distinct. The second, proposed by Fornell and Larcker, requires that each factor's average variance extracted exceed its squared correlation with any other factor. The logic is that a construct should share more variance with its own indicators than with another construct. The third, the heterotrait-monotrait ratio proposed by Henseler, Ringle, and Sarstedt in the Journal of the Academy of Marketing Science in 2015, compares the average correlation between items of different constructs with the average correlation among items of the same construct. Their simulations suggested that the Fornell-Larcker criterion often fails to detect a lack of discriminant validity, and that ratios above about .85 or .90 signal a problem. The heterotrait-monotrait ratio is available in semTools through the htmt() function. Whatever the criterion, the substantive question is whether two constructs that the theory treats as different are empirically separable in the measures used. In organizational research, constructs such as job satisfaction, affective commitment, and engagement are often correlated above .70 as latent variables, and a structural model that treats one as a cause of another may be relating a construct to a near-duplicate of itself. The measurement model is the place to discover this, before the structural paths are interpreted. Correlated errors and method variance The reflective model assumes that indicators' unique parts are uncorrelated. Real questionnaires violate this in predictable ways. Items with similar wording, such as two items both beginning with "I feel confident that," share something beyond the construct. Negatively worded items often share variance with each other that has more to do with response style than content. Items measured by the same method, such as self-report on the same occasion, share method variance that inflates their correlations with one another, including correlations across constructs. Podsakoff, MacKenzie, Lee, and Podsakoff reviewed these sources of common method bias in the Journal of Applied Psychology in 2003, and their paper remains the standard reference. Its central message is that procedural remedies, such as measuring predictor and outcome from different sources or at different times, are more effective than statistical ones. Among statistical remedies, adding a method factor on which all items load, alongside their substantive factors, is common but can create identification and interpretation problems. The popular Harman single-factor test, which checks whether one factor accounts for most of the variance, is widely regarded as an inadequate diagnostic. Correlated errors between specific items should be specified when there is an a priori reason: items that share a stem, repeated measurement of the same item across occasions, or items from the same subscale nested within a broader construct. In longitudinal models, the error of each item at time one should typically be allowed to correlate with the same item's error at later occasions; failing to do so misattributes item-specific stability to the construct. What should not be done is to add correlated errors because a modification index suggests them, with no substantive rationale. Chapter 8 returns to why. Parcels, higher-order factors, and bifactor models Three extensions of the basic measurement model come up often enough in applied work to need brief treatment. Item parceling combines items into small composites, such as averaging three items at a time, and uses the parcels as indicators. It reduces the number of parameters and often improves fit. Little and colleagues have defended parceling when the goal is to estimate structural relations among constructs whose unidimensionality is well established. Critics point out that parceling can hide misspecification in the item-level model, so that good fit with parcels is weak evidence that the items measure what they are said to. A reasonable position is that parceling is defensible for a well-validated unidimensional scale and indefensible when the measurement model is itself in question. A higher-order factor model treats first-order factors as indicators of a broader factor: for example, emotional exhaustion, depersonalization, and reduced accomplishment as facets of burnout. With three first-order factors, the second-order structure is just-identified and fits exactly as well as a model with three correlated first-order factors, so it cannot be tested against that model; with four or more it imposes testable constraints. A bifactor model lets every item load on a general factor and on one specific factor, with general and specific factors uncorrelated. It has become popular for questions about whether a multidimensional scale can be scored as a whole. Bifactor models tend to fit well, sometimes because their flexibility lets them absorb noise, and their specific factors are often weakly defined once the general factor is extracted. Indices such as omega hierarchical, which estimate how much of the total score's variance reflects the general factor, are more informative than fit comparisons for deciding whether a total score is meaningful. The measurement model as foundation Everything in the structural part of a model depends on the measurement part. The meaning of a latent variable is defined by its indicators and the pattern of their loadings. Structural paths are estimated between these latent variables, and their magnitudes depend on how much error the measurement model has removed. Discriminant validity problems in the measurement model become multicollinearity problems in the structural model. Misspecification anywhere tends to spread. This is why careful practice treats the measurement model as a hypothesis to be tested on its own before any structural paths are added, and why the next chapter concerns the confirmatory factor analysis that does the testing. Hashtags: #StructuralEquationModeling #SEM #PathAnalysis #LatentConstructs #LatentVariableModeling #CovarianceStructure #MeasurementModel #ConfirmatoryFactorAnalysis #CFA #ModelIdentification #DegreesOfFreedom #DirectEffects #IndirectEffects #MediationAnalysis #MeasurementError #ReflectiveMeasurement #FormativeConstructs #FactorLoadings #CompositeReliability #DiscriminantValidity #ModelFit #FitIndices #ModificationIndices #AMOS #FutureOfStructuralEquationModeling

Download the Book (PDF): Introduction Most scientists meet grant accounting the way travellers meet customs officers: briefly, reluctantly, and with a vague sense that something could go wrong without their quite knowing what. The proposal is written in a rush of scientific ambition, the budget is assembled in the last seventy-two hours before the deadline by a department administrator working from a spreadsheet template, and the award notice arrives months later carrying a stack of terms and conditions that almost nobody reads end to end. Then the money starts to move, and for three or five years the principal investigator spends it on people, reagents, instruments and travel while an entirely separate machine inside the university records, allocates, reconciles and reports every dollar. For most PIs, most of the time, the two worlds never collide. The experiments run, the reports go in, the award closes. But when they do collide the consequences are out of all proportion to the sums involved. A postdoc's salary charged to the wrong grant for eight months becomes a cost transfer that an auditor flags as late and unexplained. A promise in the proposal that the department would "provide a technician at no cost to the project" becomes a cost-sharing commitment the university must document to the penny. A freezer bought in the last month of an award, for no reason anyone can articulate beyond "the money was there", becomes an unallowable cost the institution must refund. And at the far end of the spectrum, patterns of misstatement across many awards become False Claims Act settlements measured in millions of dollars, announced by the Department of Justice and reported in Science and Nature. This booklet is written for the scientist in the middle of that picture: a principal investigator, or someone about to become one, who wants to understand the financial architecture of sponsored research well enough to make good decisions and avoid preventable trouble, without becoming an accountant. It concentrates on federally funded research at American universities, because that is where the rules are most elaborate, most codified and most consequential, and because the principles carry over well to foundations, industry sponsors and research systems elsewhere. It uses the vocabulary your sponsored programs office uses, and explains it, so that the next conversation with that office goes faster and better. The argument of this book The central claim is simple, and everything else in the book is an elaboration of it: a grant budget is not an estimate; it is a set of representations the institution makes to the government, and nearly every compliance failure is a gap between what was represented and what the records later show. That framing changes how a PI should think about almost every financial decision. When you list a percentage of your effort in a proposal, you are representing that you will devote that effort. When you describe a cost as direct, you are representing that it can be specifically and consistently identified with this project. When you name a collaborator's institution as a subrecipient, you are representing that it will carry out part of the programmatic work under your institution's oversight. When you offer the department's contribution of a microscope or a technician, you are representing a commitment the university must later prove it kept. Auditors, inspectors general and, in the worst cases, federal prosecutors do not ask whether the science was good. They ask whether the representations were true. Seen this way, compliance stops being a thicket of arbitrary rules and becomes something more tractable: the discipline of making only promises you can keep, and keeping records that show you kept them. The rules themselves, which run to hundreds of pages across the federal Uniform Guidance, agency policy statements and institutional procedures, turn out to be mostly consistent applications of a few principles. Costs must be necessary for the project, reasonable in amount, allocable to the project in proportion to the benefit it receives, treated consistently with how the institution treats similar costs elsewhere, and documented. Most of what follows is a working out of those principles in the places where PIs actually make decisions. What the book covers The chapters follow the path of money through an award, from the rules that govern it to the audit that tests it. Chapter 1 sets out the regulatory framework: the federal Uniform Guidance at 2 CFR Part 200, the agency-specific overlays from NIH and NSF, the institution's own policies, and the four or five tests every cost must pass. It also explains why the 2024 revision of the Uniform Guidance, effective for awards made from October 2024, changed several thresholds that matter to PIs. Chapter 2 turns to direct costs: salaries and effort, fringe benefits, the NIH salary cap, supplies, travel, participant support, and the categories of expense that are ordinarily indirect but can sometimes be charged directly. The emphasis throughout is on building a budget you can defend line by line. Chapter 3 explains indirect costs, formally called facilities and administrative costs or F&A. It shows what the rate actually pays for, how it is negotiated, why it is applied to a modified base rather than to the whole budget, and why the attempt in 2025 to replace negotiated rates with a flat cap ended up in the federal courts. Chapter 4 deals with equipment: where the line between supplies and equipment falls, what depreciation means in a university context, who owns an instrument bought with grant funds, and how to avoid the classic end-of-award purchase that auditors love to question. Chapter 5 addresses cost sharing, the most misunderstood commitment in research finance, and explains why the most expensive words in many proposals are "at no cost to the sponsor". Chapter 6 covers subawards: how to tell a subrecipient from a vendor, what your institution must do to monitor the partner it pays, and how the rules for international collaboration changed at NIH in 2025. Chapter 7 follows the award through its active life: effort reporting, cost transfers, rebudgeting, no-cost extensions and closeout, which is where most small problems either get fixed or harden into findings. Chapter 8 examines audits and penalties: how the single audit works, what the Office of Inspector General looks for, how False Claims Act cases arise, and what the largest university settlements of the past two decades actually involved. The conclusion draws these threads into a practical stance for a working PI, and argues that the institutions and investigators who suffer least from audits are not those who know the most rules, but those whose budgets and records were designed from the outset to tell a single true story. A note on scope and currency Research finance is not static. The thresholds in the Uniform Guidance were revised in 2024. NIH's salary cap rises most years. The rules for foreign subawards at NIH were rebuilt in 2025. The fight over indirect cost rates that began in February 2025 produced appellate decisions in 2026 and a policy debate that is not over. This booklet describes the rules as they stand in 2026, flags the places where they have recently moved, and tries to explain the reasoning behind them so that a reader can interpret the next change rather than merely memorise the current number. Two cautions apply throughout. First, your institution's policies may be stricter than federal rules, and when they are, they govern. A university that sets its equipment capitalisation threshold at $5,000 applies that figure, not the federal ceiling of $10,000. Second, nothing here substitutes for the specific terms of your award. The notice of award, the program solicitation and the agency's policy statement override general principles where they conflict. The value of understanding the principles is that you will know which questions to ask, and of whom, before rather than after the money is spent. Chapter 1. The Rules Behind the Money A research grant looks, from the investigator's side, like a gift with strings attached: the agency gives money, the scientist does research, the strings are paperwork. Legally it is something else. A federal grant is a form of financial assistance governed by statute, by government-wide regulation, by agency policy and by the terms of the individual award, and the institution that accepts it takes on binding obligations about how every dollar will be spent and recorded. The principal investigator is not usually the legal recipient. The university is. But the PI is the person whose decisions generate almost every cost, and so the PI is, in practice, the first line of compliance. Understanding the structure of the rules is worth an hour of any investigator's time, because it answers a question that otherwise produces endless frustration: why does the sponsored programs office say no to things that seem obviously reasonable? Usually the answer is that the request collides with one of a small number of principles that sit at the top of the hierarchy, and that no amount of scientific justification can move. The layers of authority At the top sits federal law. Statutes create the agencies, appropriate their money and occasionally impose specific restrictions, such as the annual appropriations provision that limits how much of an individual's salary NIH grants may pay, or the provisions that have, since fiscal year 2018, barred the Department of Health and Human Services from unilaterally changing the way indirect cost rates are applied to NIH awards. Statutes are rarely read by PIs, but they explain why some rules are immovable: an agency cannot waive what Congress has required. Beneath statute sits the government-wide regulation that governs almost all federal grants to universities, nonprofits, states and local governments: Uniform Administrative Requirements, Cost Principles, and Audit Requirements for Federal Awards, codified at Title 2 of the Code of Federal Regulations, Part 200. Everyone in research administration calls it the Uniform Guidance. The Office of Management and Budget issued it in December 2013 to consolidate eight older circulars into a single framework, including Circular A-21, which had governed university cost principles since the 1950s, and Circular A-110, which set administrative requirements. It became effective for new awards in December 2014 and has been revised several times since, most substantially in a revision published in April 2024 and effective for awards and amendments made on or after 1 October 2024. The Uniform Guidance is divided into subparts. Subpart D covers post-award requirements such as financial management, property, procurement, subrecipient monitoring and closeout. Subpart E contains the cost principles, the rules about what can be charged and how. Subpart F contains the audit requirements. Several appendices matter to universities, most importantly Appendix III, which sets out how institutions of higher education identify and allocate indirect costs. Each federal agency then adopts the Uniform Guidance in its own part of the regulations and publishes policy documents that explain how it applies it. For biomedical researchers the central document is the NIH Grants Policy Statement, which is incorporated by reference into every NIH award. For NSF-funded researchers it is the Proposal and Award Policies and Procedures Guide, usually called the PAPPG, which governs both what goes into a proposal and how the award is managed. The Departments of Energy and Defense, NASA, the Department of Agriculture and others have their own equivalents. These documents do not override the Uniform Guidance, but they fill in details and exercise options the Uniform Guidance leaves open. NSF's prohibition on voluntary committed cost sharing, for example, and NIH's specific rules about the salary cap and cost transfers, live at this level. Next comes the individual award. The notice of award or grant agreement specifies the budget, the period of performance, the reporting schedule and any special conditions. A program solicitation may impose rules that apply only to that competition, such as a mandatory cost-share percentage or a cap on the number of months of senior personnel salary. The award terms are the most specific layer, and where they conflict with more general guidance, they generally control. Finally, the institution's own policies apply to every award it accepts. Universities are required to have written policies and procedures for many things the Uniform Guidance governs, and they are free to be stricter than federal rules. Many are. A university may require prior approval for purchases that federal rules would allow without it, or set a lower capitalisation threshold for equipment, or require monthly rather than annual reconciliation of grant ledgers. When a PI hears "the federal rules allow it but our policy doesn't", this is the layer speaking, and it binds just as firmly as the others. For contracts rather than grants, a different body of rules applies. Research contracts with federal agencies are governed by the Federal Acquisition Regulation, which has its own cost principles in Part 31 and its own compliance culture. Contracts are less common than grants for most academic scientists, but they are frequent in defence and energy research, and they bring stricter requirements around deliverables, invoicing and cost accounting. Where this booklet speaks of awards, it means grants and cooperative agreements unless it says otherwise. The tests every cost must pass The heart of the cost principles is a short section, 2 CFR 200.403, that sets out the factors affecting the allowability of costs. Read it once and most of the rest of Subpart E falls into place. To be allowable, a cost must be necessary and reasonable for the performance of the award and allocable to it. It must conform to any limitations or exclusions in the cost principles or the award. It must be consistent with policies that apply uniformly to both federally financed and other activities of the institution. It must be accorded consistent treatment, meaning that a cost cannot be charged directly to an award if a cost of the same kind in similar circumstances has been charged as indirect. It must be determined in accordance with generally accepted accounting principles. It must not be used to meet cost-sharing requirements on another federal award. And it must be adequately documented. In practice, PIs will spend most of their time on four of these tests: reasonableness, allocability, consistency and documentation. Reasonableness, defined at 200.404, asks whether the cost, in its nature and amount, does not exceed what a prudent person would incur in the circumstances prevailing when the decision was made. The test looks at whether the cost is of a type generally recognised as ordinary and necessary, whether it reflects sound business practice and arm's-length bargaining, whether market prices for comparable goods were considered, and whether the individuals concerned acted with prudence. A premium-priced centrifuge chosen for no stated reason when a comparable instrument costs half as much is a reasonableness problem. So is business-class travel when economy is available, or a catered working lunch for a lab of twelve that happens to cost as much as a restaurant dinner. Allocability, at 200.405, asks whether the cost is chargeable to this particular award in accordance with the relative benefits received. A cost is allocable to an award if it is incurred specifically for that award, if it benefits both that award and other work and can be distributed in reasonable proportion to the benefit, or if it is necessary to the overall operation of the institution and is assignable in part to the award. The first two are direct costs; the third is the domain of indirect costs. Allocability is the principle most often violated in the ordinary life of a lab, because it is so easy to charge a shared purchase to whichever grant has the most money left, rather than dividing it among the projects that actually use it. The Uniform Guidance anticipates this temptation and forbids it explicitly: a cost allocable to one award may not be charged to another to overcome funding deficiencies, avoid restrictions or for other reasons of convenience. Consistency, which appears in both 200.403 and in the rules on direct and indirect costs, asks whether the institution treats like costs alike. If a university's normal practice is to recover departmental administrative salaries through its indirect cost rate, it cannot also charge those salaries directly to a federal grant except in the specific circumstances the rules allow. Otherwise the government would pay for the same thing twice: once through the rate and once through the direct budget. This is why so many seemingly sensible direct charges, from office supplies to general-purpose laptops to the department's administrative assistant, draw scrutiny. They are the kinds of cost that are ordinarily in the indirect pool. Documentation is the least intellectual of the tests and the most frequently failed. A cost that is necessary, reasonable and allocable is still unallowable if the institution cannot show, with records created at or near the time, what was bought, why, for which project, and who approved it. Auditors routinely disallow costs not because they doubt the purchase was legitimate but because the file does not prove it. The practical lesson is to write the justification when the cost is incurred, not when the auditor asks. What is expressly unallowable Beyond the general tests, Subpart E contains some fifty sections addressing specific items of cost, from advertising to travel. Most confirm that normal research expenses are allowable under conditions. A smaller number identify costs that are unallowable regardless of how the general tests come out. These are worth knowing by heart, because charging them to a federal award is not an error of judgement but a clear violation. Alcoholic beverages are unallowable. Entertainment costs, including amusement, diversion, social activities and associated costs such as tickets and meals, are unallowable unless they have a programmatic purpose and are authorised in the approved budget or with prior written approval. Fundraising and investment management costs are unallowable. Lobbying is unallowable, as are most costs of organising or influencing legislation. Bad debts, contributions and donations made by the institution, fines and penalties resulting from violations of law, and goods or services for personal use are unallowable. Costs of alumni activities, commencement and convocation, and student activities are generally unallowable as direct or indirect costs, subject to limited exceptions. Membership dues in social or country clubs are unallowable; membership in professional and scientific societies generally is allowable. The pattern is instructive. Expressly unallowable costs are, for the most part, costs that benefit the institution or individuals rather than the research, or costs whose public funding would be politically indefensible. When a PI proposes a charge that falls near one of these categories, such as a laboratory end-of-year dinner, a gift for a departing technician, or the reception after a symposium, the right question is not whether it helped morale but whether it is on the list. If it is, other funds must pay. Why the 2024 revision matters to investigators The April 2024 revision of the Uniform Guidance was the most substantial since 2014, and several of its changes land directly on decisions PIs make. It applies to new awards and to funding amendments made on or after 1 October 2024, which means that for several years many investigators will hold awards under both the old and the new thresholds at the same time. Knowing which rule applies to which award is a task for the sponsored programs office, but knowing that the difference exists is the PI's responsibility. The threshold for equipment rose from $5,000 to $10,000. An item with a useful life of more than one year and a per-unit acquisition cost at or above the lesser of the institution's capitalisation level or $10,000 is now equipment; below that, it is a supply. Because most universities set their own capitalisation thresholds, the practical effect depends on whether your institution has raised its own figure. Many did so in 2024 and 2025; some did not. The de minimis indirect cost rate, available to organisations that do not have a negotiated rate, rose from 10 percent to up to 15 percent of modified total direct costs. This matters to PIs mainly through their subrecipients: a small company or community organisation receiving a subaward can now recover more of its overhead without negotiating a rate. The portion of each subaward included in the modified total direct cost base, on which indirect costs are calculated, rose from the first $25,000 to the first $50,000 over the life of the subaward. The result is that a prime institution now recovers indirect costs on more of each subaward it manages, which recognises the real work of subrecipient monitoring. The threshold at which a non-federal entity must undergo a single audit rose from $750,000 in annual federal expenditures to $1,000,000. Some small subrecipients that previously had single audits no longer do, which shifts more of the monitoring burden onto the prime institution. Several other thresholds moved in the same direction: the value of residual unused supplies at the end of an award above which the government must be compensated rose from $5,000 to $10,000, and the ceiling on fixed amount subawards without prior approval rose from $250,000 to $500,000. The revision also clarified language throughout, replaced "should" with "must" where requirements were intended to be mandatory, and addressed matters such as the treatment of data and information technology costs. The general direction of these changes is to reduce administrative burden by raising thresholds that had not been adjusted for inflation in a decade. None of them loosens the basic tests of allowability. Where the PI stands in all of this Institutions assign responsibilities differently, but some features are nearly universal. The PI is responsible for the scientific and technical direction of the project and for ensuring that expenditures are appropriate to it. The PI approves or initiates purchases, hires and assigns staff, certifies or confirms effort, reviews monthly financial statements, and is usually the first person asked to explain any questioned cost. The department administrator prepares budgets, processes transactions and reconciles accounts. The central sponsored programs office submits proposals, negotiates and accepts awards, handles prior approvals, and interprets policy. The post-award or research accounting office invoices sponsors, draws funds, prepares financial reports and manages closeout. The internal audit function and external auditors test whether all of this works. The risk in this division of labour is that each party assumes someone else is checking. A PI who approves charges without reading them because the department administrator is competent, and an administrator who processes whatever the PI approves because it is the PI's grant, together produce a system in which nobody actually applies the tests of allowability. Most institutions now make the PI's role explicit through training requirements, signed acknowledgements at proposal submission, and periodic certification of expenditures and effort. These are not formalities. They are the institution's way of establishing, for later reference, that the person closest to the costs accepted responsibility for them. A useful way to hold all of this in mind is to imagine the award's financial record being read, years later, by someone who knows nothing about your science and everything about the rules. Every charge should make sense to that reader on its face, or have a note beside it that makes it make sense. Every promise in the proposal should have a matching trace in the records. Everything that follows in this booklet is a way of making that imagined reading uneventful. Chapter 2. Direct Costs: Building a Budget You Can Defend Direct costs are the costs that can be identified specifically with a particular project and assigned to it with a high degree of accuracy. That is the definition in 2 CFR 200.413, and it sounds simple enough: the salary of the graduate student who runs the experiments, the antibodies she uses, the flight to the conference where she presents the results. The complications arise at the edges, where a cost benefits several projects, or where it is of a kind the institution normally treats as overhead, or where the proposal promised one thing and the project needs another. A budget that can be defended is one in which every line has a traceable reason, a basis for its amount, and a plan for how it will be documented when spent. This chapter works through the main categories in roughly the order they appear on a typical federal budget form, with attention to the points where investigators most often go wrong. People: salary, effort and the cap For most research awards, personnel costs are the largest single category, often between half and three quarters of direct costs. They are also the category most exposed to audit, because the link between a salary charge and the work it paid for depends entirely on the institution's system for recording how people spend their time. The basic principle, in 2 CFR 200.430, is that compensation for personal services is allowable to the extent that it is reasonable for the services rendered, conforms to the institution's established written policy applied consistently to federal and non-federal activities, and is supported by records that accurately reflect the work performed. The share of an individual's salary charged to an award must match the share of that individual's total professional effort devoted to the award. The key word is total. Effort is expressed as a percentage of everything a person does for the institution in return for their institutional base salary, including research, teaching, clinical work, administration, committee service and the writing of new proposals. It is not a percentage of a forty-hour week. A faculty member who works sixty hours a week and spends fifteen of them on a particular project is devoting 25 percent effort to it, not 37.5 percent. Effort is also not the same thing as time in the lab: a PI who spends the morning analysing data for the project at home and the afternoon teaching is dividing effort between the project and instruction, wherever the work happens. Institutional base salary is the annual compensation the institution pays for an individual's appointment, whether that time is spent on research, teaching or other activities. It excludes income earned outside the institution, such as consulting or honoraria, and for most universities excludes supplements that are not part of the base appointment. Institutional base salary cannot be increased because a grant is paying for it. The rate used to charge a federal award must be the same rate the institution pays for all the individual's other duties. For NIH awards, an additional limit applies. Since the 1990s, appropriations acts have limited the direct salary that NIH grants may pay to any individual to the rate of Level II of the federal Executive Schedule. For 2026 that figure is $228,000 a year, up from $225,700 in 2025. The cap is not a limit on what an investigator may earn. It is a limit on the rate at which NIH will reimburse that salary. A faculty member whose institutional base salary is above the cap can still devote any percentage of effort to an NIH project, but the salary charged to the grant is calculated as that percentage of the capped rate, and the institution must pay the difference from non-federal funds. The worked example in Table 1 shows how the arithmetic runs for a hypothetical investigator. Table 1. Applying the NIH salary cap to an investigator above the cap (illustrative). Item Amount Institutional base salary (12-month) $260,000 Committed effort on NIH award 25% Salary that effort represents $65,000 2026 NIH cap (Executive Level II) $228,000 Salary chargeable to the award (25% of cap) $57,000 Difference paid from institutional funds $8,000 The $8,000 difference in that example is not cost sharing in the formal sense unless the institution chooses to treat it as such, but it is a real cost the department must fund, and a PI above the cap who commits heavy effort to multiple NIH awards can create a significant liability for the department. For faculty on nine-month appointments the cap is prorated, which for 2026 gives $171,000 for the academic year. NSF takes a different approach. Its policy, set in the PAPPG, is that salary for senior personnel is normally limited to no more than two months of their regular salary in any one year, counting all NSF-funded grants together. Compensation above that amount must be disclosed in the proposal budget and justified, and specifically approved in the award. The two-month rule is the source of a common confusion: it limits the salary NSF pays, but it does not limit the effort a faculty member devotes during the academic year, which may be substantial and uncompensated by NSF. Fringe benefits follow salary. Most universities charge fringe benefits through a negotiated fringe benefit rate, applied as a percentage of salary to each salary charge, with different rates for different employee categories, such as faculty, staff, postdocs and students. Because the rate is negotiated with the federal government, the PI does not choose it. The budgeting task is simply to apply the correct rate to each category and to remember that when salary moves, fringe moves with it. Graduate students present particular issues. Their compensation typically consists of a stipend or salary, tuition remission, and sometimes health insurance. Tuition remission is allowable as part of compensation where it is consistent with institutional policy and the student is performing necessary work on the project. For NIH research grants, the combined compensation for a graduate student, including salary, fringe and tuition remission, is limited to the amount paid to a first-year postdoctoral scholar at the NIH training stipend level in effect when the grant is awarded. Tuition remission is excluded from the base on which indirect costs are calculated, which is one reason its budgeting treatment varies between institutions. Materials, services and travel Materials and supplies are allowable when they are necessary to carry out the project, and the costs should be net of any discounts or rebates. The principal budgeting question is specificity. A proposal line that reads "laboratory supplies, $40,000" is less defensible than one that explains the expected consumption of reagents, animals, sequencing kits and consumables per experiment and multiplies by the number of experiments planned. Agencies rarely challenge supply budgets at proposal time, but a well-reasoned budget justification becomes a useful document later, when the PI needs to show that spending was consistent with the approved plan. Supplies purchased for general use across a lab must be allocated among the projects that benefit. A case of pipette tips used by every project in the lab should not be charged entirely to one grant. Most institutions allow reasonable allocation methods, such as distribution by headcount on each project or by proportion of use, provided the method is documented and applied consistently. The failure mode is the lab in which each month's purchases go to whichever award has the most remaining balance, which is a textbook allocability violation even if every item was genuinely used for research. Computing devices merit a specific note. Under 2 CFR 200.453, computing devices, meaning machines used to acquire, store, analyse, process and publish data, are allowable as supplies when they cost less than the equipment threshold, provided they are essential and allocable to the award, even if not solely dedicated to it. A laptop used by a graduate student for data analysis on the project is therefore generally allowable as a direct cost. A laptop for the PI that will be used for teaching, email and administration as much as for research is harder to defend, and many institutions require a specific justification or a partial allocation. Services, including core facility charges, sequencing, animal care per diems and analytical services, are allowable when necessary and when the rates charged are consistent. University core facilities that bill internal users are service centres, and they must set their rates to recover no more than their actual costs over time and charge all users, federal and non-federal, the same rate. A PI does not need to understand the rate-setting method, but should know that core rates are themselves subject to federal rules and that it is not permissible to negotiate a special internal rate for a federally funded project. Travel is allowable for employees on official business related to the award, under the institution's written travel policy, provided costs are reasonable and consistent with what the institution normally allows. The Uniform Guidance requires that airfare be the lowest reasonable commercial fare, typically economy, and that higher classes be justified by specific circumstances such as medical need or the unavailability of economy seats that would require travel at unreasonable hours. Foreign travel on federal awards is subject to the Fly America Act, which requires the use of US flag air carriers, or foreign carriers operating under code-share arrangements with them, with limited exceptions. Many agencies require that foreign travel be identified in the proposal. Conference registration fees are allowable where attendance is related to the project, and meals included in registration should be deducted from any per diem claimed. Participant support and other special categories Participant support costs are direct costs for items such as stipends, subsistence allowances, travel allowances and registration fees paid to or on behalf of participants or trainees, but not employees, in connection with conferences, workshops or training projects. They are defined at 2 CFR 200.1 and carry two rules that surprise PIs. First, they are excluded from the base on which indirect costs are calculated. Second, NSF and several other agencies require prior approval to rebudget funds out of the participant support category into other categories. The logic is that participant support is money the sponsor intended to pass through to people outside the institution, and the sponsor wants to control whether it is diverted. A PI who plans a summer workshop, reduces it in scale, and spends the savings on a postdoc without approval has made an unallowable rebudget, however sensible the decision was scientifically. Consultant costs are allowable for professional services by individuals who are not employees of the institution, where the services are necessary and the fees are reasonable. The consultant must be distinguished from a subrecipient, a distinction considered in Chapter 6, and from an employee. A faculty member at the same institution is not normally eligible to be paid as a consultant on an award from the same institution, because the compensation belongs within their institutional base salary. Publication costs, including page charges and open-access fees, are allowable where they relate to the project. Following changes in federal public access policy, agencies increasingly expect investigators to budget for making publications and data available. NIH's public access policy, updated to require that accepted manuscripts be made publicly available on PubMed Central without an embargo, took effect in July 2025; data management and sharing costs have been an explicit allowable budget item at NIH since its Data Management and Sharing Policy took effect in January 2023. Budgeting these costs deliberately, rather than hoping residual funds will cover them, is now a normal part of a defensible budget. Animal and human subject costs, including per diems for animal care and payments to research participants, are allowable, with the latter often requiring particular documentation to protect participant privacy while demonstrating that payments were made. Costs that are usually indirect The trickiest direct-cost questions concern items that are ordinarily part of indirect costs. The general principle is consistency: a cost of a type the institution normally recovers through its F&A rate cannot be charged directly unless the circumstances are genuinely different. The Uniform Guidance addresses the most common case explicitly. Under 2 CFR 200.413(c), salaries of administrative and clerical staff should normally be treated as indirect. Direct charging may be appropriate only if all of four conditions are met: the services are integral to the project or activity; the individuals involved can be specifically identified with the project; the costs are explicitly included in the budget or have the prior written approval of the federal agency; and the costs are not also recovered as indirect costs. "Integral" means essential to the project's success, as in a large multi-site clinical trial that needs a dedicated coordinator, or a centre grant with substantial administrative workload specific to the award. A departmental administrator who spends part of her time processing purchases for the project does not meet the test; those services are what the administrative component of the F&A rate pays for. Similar reasoning applies to office supplies, postage, local telephone service, memberships, and general-purpose software. These are presumed to be indirect unless the project creates a need that is distinct from ordinary departmental operations. A project that runs a mailed survey to ten thousand households can charge postage directly. A lab that sends ordinary correspondence cannot. The practical test a PI can apply is simple. Ask whether the cost would exist if this project did not. If it would, because it is part of running a department or a laboratory in general, it is probably indirect. If it arises only because of the project and scales with the project's activities, it is probably direct. The test is not perfect, but it anticipates most of the questions the sponsored programs office will ask. Writing the budget justification Every federal proposal budget is accompanied by a justification: a narrative explaining each category and the basis for its amount. Investigators often treat it as a formality and fill it with generic text. This is a mistake, because the justification is part of the application the agency approves, and it becomes a reference point for what the institution represented it would do. A good justification names each person, their role, their effort and the basis for their salary. It explains supplies in terms of the experiments that will consume them. It identifies each trip, its purpose and its approximate cost. It names each piece of equipment, explains why it is needed and why existing institutional resources cannot serve. It explains every subaward and what the partner will do. It explains any cost that might look unusual to a reviewer who knows the rules, such as direct-charged administrative support, in terms that show why it meets the applicable test. For NIH modular budgets, which apply to many research grant applications requesting $250,000 or less in direct costs per year and are requested in $25,000 modules, the application does not include a detailed categorical budget, but the institution still expects the PI to have one internally, and the personnel justification remains a required element. A modular budget is a simplification of what is submitted, not of what is spent. A budget built this way does two things. It gives the reviewer confidence that the project has been planned seriously, and it gives the PI, years later, a document that explains why the money was spent as it was. The second function matters more than most investigators realise. When an auditor asks why a cost was incurred, the most persuasive answer is a sentence written before the cost existed, predicting that it would. Chapter 3. Indirect Costs: What the Rate Pays For No part of research finance generates more resentment among scientists than indirect costs. An investigator who wins a grant of, say, $1.5 million in total costs may discover that only $1 million or so is available to spend on the project, with the rest going to the university as "overhead". It is natural to see this as a tax, and natural to suspect that the money funds administrators rather than science. Both perceptions are understandable and both are largely mistaken, and a PI who understands why will make better budgeting decisions and argue more effectively with both the institution and the sponsor. Indirect costs, formally facilities and administrative costs or F&A, are real costs of doing research that cannot practically be assigned to individual projects. The laboratory building in which the research takes place has to be heated, cooled, lit, cleaned, secured and maintained. It has to be paid for, either through depreciation of its construction cost or through interest on the debt used to build it. Its fume hoods, vivarium, shared instruments and hazardous-waste systems have to operate. Someone has to process the purchase orders, run the payroll, maintain the accounting system, prepare the financial reports, ensure compliance with animal welfare and human subjects regulations, and handle export controls and research security. None of this can be charged to a single grant with any precision, but all of it is needed for any grant to be performed. The F&A rate is the mechanism by which each project pays its proportionate share. What goes into the pools Appendix III to the Uniform Guidance, which applies to institutions of higher education, divides indirect costs into two broad groups, each made up of several cost pools. The facilities group includes depreciation on buildings, building improvements and equipment; interest on debt associated with those buildings and equipment; operation and maintenance expenses such as utilities, custodial services, repairs, security, and environmental health and safety; and library expenses. These costs are driven by space and by the physical infrastructure of research. The administrative group includes general administration, meaning institution-wide functions such as the president's office, accounting, payroll, purchasing and legal services; departmental administration, meaning the administrative work performed in academic departments, including a portion of the time of faculty chairs and administrative staff; sponsored projects administration, meaning the offices that handle proposals, awards, compliance and post-award accounting; and student administration and services, to the limited extent those functions benefit research. Each pool is allocated among the institution's major functions, principally instruction, organised research, other sponsored activities and other institutional activities, using bases specified in Appendix III. Building costs are allocated using detailed space surveys that assign each room, by square footage, to the functions it serves. Administrative costs are allocated using expenditure bases. The result, after a long and detailed calculation, is the portion of each pool attributable to organised research. Dividing that total by the organised research base produces the F&A rate for research. Two features of the pool structure are worth noting. First, the administrative component of the rate for universities has been capped at 26 percentage points since 1991, when OMB revised Circular A-21 following controversies over university indirect cost recovery, most prominently at Stanford. Any administrative cost above that cap is unrecovered and must be paid by the institution from other sources. For many research-intensive universities, actual administrative costs substantially exceed the cap. Second, some institutions are entitled to a utility cost adjustment of 1.3 percentage points, added to reflect the higher energy consumption of research space, a provision now carried in Appendix III. The implication for PIs is that most of the rate is facilities. In typical negotiated rates at research universities, the facilities component is often similar to or larger than the administrative component, and the administrative portion is held down by the cap. When faculty imagine that indirect costs fund administrators, they are thinking of a portion of the rate that is, by rule, limited and in practice under-recovered. Rates, bases and the arithmetic of a budget An institution's F&A rate is negotiated with its cognizant agency for indirect costs, which for most universities is either the Department of Health and Human Services, through its Cost Allocation Services, or the Department of Defense, through the Office of Naval Research. The institution prepares a detailed proposal based on a recent fiscal year's actual costs; the cognizant agency reviews and negotiates it; and the result is a negotiated indirect cost rate agreement. The agreement is binding on all federal agencies, which must accept the negotiated rate except where a statute, regulation or agency-level policy approved by OMB provides otherwise, or where a program has a published limit on indirect costs. Rate agreements typically contain several rates: an on-campus research rate, an off-campus research rate, rates for instruction and other sponsored activities, and sometimes special rates for particular facilities. They also specify whether the rates are predetermined, meaning fixed for a period and not adjusted after the fact; provisional, meaning used until final rates are set; fixed with carry-forward, meaning adjusted in a later period for differences between estimated and actual costs; or final. Most research universities operate under predetermined rates negotiated for several years at a time. The rate is not applied to the total budget. It is applied to a base, which for universities is modified total direct costs, or MTDC. Under the definition in 2 CFR 200.1, MTDC consists of all direct salaries and wages, applicable fringe benefits, materials and supplies, services, travel, and up to the first $50,000 of each subaward, regardless of the period of performance of the subawards under the award. It excludes equipment, capital expenditures, charges for patient care, rental costs, tuition remission, scholarships and fellowships, participant support costs and the portion of each subaward in excess of $50,000. The excluded items are either capital costs, which are paid for in full as direct costs or recovered separately through depreciation, costs that pass through the institution without consuming much of its infrastructure, or costs, such as rent for off-campus space, that would duplicate what the facilities component pays for. Table 2 shows how this works for a hypothetical one-year budget at an institution with a 57 percent on-campus research rate. Table 2. Calculating indirect costs on a modified total direct cost base (illustrative, 57% rate). Budget item Direct cost In MTDC base? Amount in base Salaries and fringe benefits $210,000 Yes $210,000 Materials, supplies and services $45,000 Yes $45,000 Travel $8,000 Yes $8,000 Equipment (one instrument) $62,000 No $0 Tuition remission $18,000 No $0 Subaward (first year of $120,000 total) $40,000 First $50,000 only $40,000 Totals $383,000 $303,000 At a 57 percent rate, the indirect costs on that budget are about $172,700, and total costs are about $555,700. The effective rate as a proportion of total direct costs is about 45 percent, lower than the negotiated rate because of the excluded items. In the second year, if the subaward continues with $40,000 more, only $10,000 of it enters the base, because the first $50,000 is counted once across the subaward's life. Investigators sometimes treat MTDC exclusions as a budgeting strategy: shift money into equipment or participant support to reduce indirect costs and free more for direct spending within a fixed total. There is nothing wrong with budgeting for genuinely needed equipment, but a budget distorted to minimise the base invites the question of whether each item is actually necessary and correctly classified. More to the point, the institution's F&A recovery on a given award does not change what the research costs the institution to support. Every exclusion is paid for by someone. Where the PI touches the rate Although no PI negotiates the rate, investigators contribute to the data from which it is built, usually without realising it. The most important contribution is the space survey. Because the facilities component is allocated by square footage, the institution periodically asks departments to report how each room is used: for organised research, instruction, other sponsored activities, departmental administration or other institutional purposes. A laboratory used entirely for federally funded research is assigned to organised research; a room used half for teaching labs and half for research is split. The accuracy of those assignments determines how much of the building's depreciation, interest and operating costs flows into the research rate. A survey that overstates research use inflates the rate and is exactly the kind of error that cognizant agency reviewers and auditors look for. A survey that understates it leaves recoverable costs unclaimed. The PI's role is simply to answer the survey accurately, which usually means knowing which funding sources pay for the people working in each room. The second contribution is the organised research base itself. Salaries and other costs charged to sponsored research form the denominator of the rate calculation. When costs that belong to research are recorded as instruction or departmental activity, or the reverse, the rate is distorted. The same effort records that support salary charges on individual awards therefore also support the institution-wide calculation. This is one reason the 2001 clarification on voluntary uncommitted cost sharing, discussed in Chapter 5, mattered so much to universities: it confirmed that uncommitted faculty effort did not have to be added to the research base, where it would have diluted the rate. The third is timing. Rate agreements run for fixed periods, and proposals are budgeted using the rates in effect, or expected to be in effect, during each year of the project. Under the Uniform Guidance and long-standing federal policy, the rate in effect at the time of the initial award generally applies for the life of that competitive segment, even if the institution negotiates a new rate in the meantime. A PI planning a multi-year budget should use whatever rate the sponsored programs office provides for each year and should not assume that the rate on a colleague's older award applies to a new one. On-campus, off-campus and the Columbia lesson The distinction between on-campus and off-campus rates has produced one of the most instructive enforcement cases in research finance. The on-campus rate applies to research performed in facilities owned by the institution or for which it pays depreciation, use charges or rent that is included in its facilities costs. The off-campus rate, much lower because it omits most facilities costs, applies when the research is performed in space the institution does not own and whose costs are not in its pools, such as leased space whose rent is charged directly, or facilities owned by another entity. In July 2016, Columbia University agreed to pay $9.5 million to settle a False Claims Act case concerning this distinction. According to the US Attorney's Office for the Southern District of New York, Columbia had applied its on-campus rate of 61 percent, rather than its off-campus rate of 26 percent, to some 423 NIH grants for research performed between 2003 and 2015 in buildings owned by New York State and New York City, principally at the New York State Psychiatric Institute. The government alleged that Columbia did not own or operate those facilities, did not for most of the period pay the state for their use, and did not disclose the non-ownership to NIH. For a PI, the lesson is not that one should memorise the rate agreement. It is that the location of the research is a representation. Proposals ask where the work will be performed. If the answer changes during the project, because a lab moves into leased space, a clinical study is performed at a partner hospital, or fieldwork occupies most of the award period, the applicable rate may change, and the sponsored programs office needs to know. Many rate agreements also specify that when a project is performed off campus for more than a stated portion of its duration, the off-campus rate applies to the whole project or to the relevant portion. Getting this right is the institution's job, but the PI is the only person who knows where the research is actually happening. The fight over the rate In February 2025, NIH issued a notice, NOT-OD-25-068, announcing that it would cap indirect cost rates for institutions of higher education at 15 percent for new awards and for existing awards going forward, replacing negotiated rates. Had the policy taken effect, research universities whose negotiated rates were typically in the 50 to 65 percent range would have lost a large fraction of their indirect cost recovery on NIH awards almost overnight. The response was immediate. Associations of universities and medical colleges, joined by a coalition of state attorneys general, sued in the US District Court for the District of Massachusetts. The court issued a temporary restraining order within days, then a preliminary injunction, and in April 2025 a final judgment vacating the policy. The plaintiffs' central argument was that the policy violated both the Uniform Guidance's requirements for deviating from negotiated rates and the appropriations rider that has, since fiscal year 2018, prohibited HHS from modifying the indirect cost provisions in effect for NIH. In January 2026 the US Court of Appeals for the First Circuit affirmed, holding that Congress had deliberately prevented NIH from displacing negotiated rates in this way. Similar caps announced during 2025 by NSF, the Department of Energy and the Department of Defense were also challenged in court and blocked, and appropriations language for fiscal year 2026 extended protections to additional agencies. The litigation settled the question of whether an agency could impose a flat cap by notice. It did not settle the policy debate about whether the negotiated-rate system is the right one. Critics argue that it is opaque, that it rewards institutions with expensive facilities, and that the government cannot easily see what its indirect payments buy. Defenders argue that the rates reflect audited actual costs and that the alternative is for universities to subsidise federal research even more heavily than they already do through the administrative cap and cost sharing. In 2025 a coalition of higher education associations known as the Joint Associations Group, including the Council on Governmental Relations, the Association of American Universities and others, published an alternative called the Financial Accountability in Research, or FAIR, model. Its central proposal is to replace the single negotiated rate with a structure in which more research-specific support costs would be budgeted and justified project by project, alongside a fixed percentage for general institutional operations. The final version was released in the second half of 2025. Whether it, or some other reform, will be adopted by the government remained an open question as of 2026. For the working PI, the practical point is this. The indirect cost rate on your award is not a number the university chose arbitrarily, and it is not a number you can negotiate away to make your budget more competitive at a federal agency. It is the product of an audited, negotiated process, and it is legally binding. But the system around it is under more pressure than at any point in the past three decades, and a PI who understands what the rate pays for is better placed to take part in the argument about what should replace it. Foundations, industry and limited rates Not every sponsor pays the negotiated rate. Many private foundations cap indirect costs at a fixed percentage, often 10 to 15 percent of direct costs, and some pay none at all. Certain federal programs have statutory or regulatory limits, such as the 8 percent limit on indirect costs for NIH training grants and fellowships of certain kinds, and limited rates for some Department of Agriculture programs. Industry sponsors typically pay a full rate, often higher than the federal research rate, because universities treat industry work as outside the federal negotiation and price it accordingly. When a sponsor pays less than the negotiated rate, the difference is unrecovered indirect cost. The costs themselves do not disappear; the institution pays them from other resources, typically unrestricted funds, endowment income or tuition. Many institutions have policies requiring that proposals to sponsors with low indirect rates be approved at a senior level, precisely because every such award increases the institution's subsidy of research. In some cases, discussed in Chapter 5, unrecovered indirect costs can be counted as cost sharing on federal awards. In all cases, a PI who understands that indirect costs are real costs will see why the university cares so much about the rate on each award, and why a large foundation grant with no overhead is, from the institution's perspective, not free money but a purchase of research partly paid for by the university itself. Hashtags: #ScientificGrantBudgeting #FinancialCompliance #ResearchFinance #GrantManagement #PrincipalInvestigator #SponsoredResearch #UniformGuidance #FederalResearchGrants #DirectCosts #IndirectCosts #FacilitiesAndAdministrativeCosts #CostAllowability #CostAllocability #CostReasonableness #CostSharing #EffortReporting #NIHGrants #NSFGrants #Subawards #BudgetJustification #ResearchAccounting #AuditCompliance #GrantCloseout #ResearchAdministration #FutureOfResearchFinance

Download the Book (PDF): Introduction In 2006 the structural biologist Geoffrey Chang and his colleagues retracted five papers, three of them from Science. The papers described the three-dimensional structures of membrane transport proteins, and they had been cited hundreds of times. Other laboratories had built on them, designed experiments around them, and in some cases struggled to reconcile their own data with them. The cause of the collapse was not fraud, not contaminated samples, and not a flawed theory. It was a program written in the laboratory that swapped two columns of data, inverting the sign of a set of measurements before they went into the structure calculation. Years of careful crystallography had been passed through a few lines of code that nobody outside the group had examined and nobody inside it had tested against a case with a known answer. The story is often told as a cautionary tale about one unlucky lab. It is better understood as an ordinary event that happened to be visible. Most research now runs through software. When the Software Sustainability Institute surveyed 417 researchers at fifteen research-intensive UK universities in 2014, 92 per cent said they used research software, 69 per cent said their research would not be practical without it, and 56 per cent said they wrote their own. Of those who wrote their own, around one in five had received no training in software development at all. Surveys in the United States and elsewhere since then have found much the same pattern. The laboratory's most heavily used instrument is frequently the one its members were never taught to build. This booklet is about building that instrument well. Its controlling argument is simple to state and, in practice, surprisingly hard to act on: the engineering practices that make research code trustworthy are the same practices that make it cheap to change, and a researcher who adopts them in small, proportionate steps pays less for reliability than they are already paying, invisibly, for its absence. Version control, automated tests, modular design, continuous integration and containerised environments are usually presented to academics as a tax — a set of professional niceties that software companies can afford and that a PhD student with a deadline cannot. The chapters that follow argue the reverse. These practices are how you stop losing afternoons to the question "which version of the script produced the third plot in the paper?", how you make a reviewer's request for an extra analysis a morning's work rather than a fortnight's, and how you make it possible for a new student to pick up a project when its author leaves. Who this is for The reader I have in mind writes code as part of doing research rather than as the research itself. You might be a doctoral student in ecology whose thesis rests on a few thousand lines of R; a postdoc in physics maintaining a Fortran solver inherited from a predecessor; an economist whose replication package has to satisfy a journal's data editor; or a principal investigator who suspects the group's code is fragile but does not know what to ask for. You can write a loop and a function. You may have used Git, perhaps by copying commands from a colleague. You have probably never written a test on purpose. You do not need to become a professional software engineer. The research software engineering movement — which took its name at a workshop in Oxford in 2012, gained a UK society in 2019, and now has sister organisations in Germany, the Netherlands, the United States, Australia and elsewhere — exists partly so that universities can employ specialists for the hardest problems. But specialists cannot be everywhere, and most research code will always be written by researchers. The goal here is the level of craft that lets your code do what you think it does, lets you prove it, and lets someone else run it in five years. What the booklet covers, and what it leaves out The booklet follows the life of a piece of research code from its first commit to its eventual archiving. The first chapter sets out why research software fails and why those failures are hard to see. Chapters 2 and 3 cover version control with Git: first as a personal laboratory notebook, then as the basis for collaboration through branches, pull requests and review. Chapter 4 turns to the structure of the code itself — how to break a sprawling analysis script into functions and modules that can be understood and tested one piece at a time. Chapter 5 addresses unit testing, with particular attention to the problem that makes scientific testing distinctive: you often do not know the right answer in advance. Chapter 6 covers continuous integration, the practice of having a machine run your tests on every change. Chapter 7 deals with computational environments and containers, including Docker and the HPC-friendly Apptainer. Chapter 8 addresses the long view: documentation, licensing, citation, versioning, and what happens to code when its author moves on. Some things are deliberately absent. There is no tutorial in a specific language; the examples lean on Python and R because those dominate academic computing, but the principles transfer to Julia, MATLAB, C++ or Fortran. There is nothing on high-performance optimisation, GPU programming or parallel computing, each of which deserves its own treatment and none of which matters if the serial code is wrong. Workflow managers such as Snakemake and Nextflow are mentioned where they bear on reproducibility but not taught. And the booklet does not attempt to cover the governance of large community software projects with dozens of contributors; it is aimed at the individual and the small group, where most research code lives. A note on proportion The most common objection to everything in this booklet is that it is overkill. For a thirty-line script that will run once and be discarded, it is. Research code exists on a spectrum: exploratory notebooks, analysis pipelines behind a paper, tools shared within a lab, and libraries used by a community. Each position on that spectrum warrants a different level of engineering, and one of the aims of the chapters that follow is to help you recognise where your code sits and what it needs. A throwaway exploration needs version control and nothing else. An analysis that supports a published claim needs tests for its core calculations and a recorded environment. A tool used by other people needs continuous integration, documentation, versioned releases and a licence. The mistake is not to under-engineer throwaway code. It is to fail to notice when throwaway code has quietly become the foundation of a thesis. That transition happens without announcement, usually around the time the code has grown too large to be rewritten comfortably, and the practices in this booklet are cheapest to adopt just before it. It helps to think of the practices as layered, each one making the next possible. Version control gives you a history you can trust, which makes it safe to restructure code into modules. Modular code can be tested one piece at a time. Tests are what a continuous integration service runs. A recorded environment is what lets those tests, and your results, mean the same thing on another machine. And all of it together is what makes code maintainable by someone other than its author. You can start at the bottom of that stack this afternoon, and each layer you add pays for itself before you need the next. The chapters are written to be read in order for that reason, though each also stands on its own for the reader who arrives with a specific problem. Chapter 1. The Instrument Nobody Calibrated Every experimental scientist learns to calibrate instruments. A mass spectrometer is run against a reference standard before samples go in; a thermometer is checked in ice water; a new antibody is validated against a known positive and a known negative. Nobody regards this as bureaucratic overhead. It is simply what it means to take a measurement seriously. Yet the same researchers routinely pass their data through code that has never been checked against anything, written in a hurry, modified dozens of times, and run on a machine whose configuration nobody could reproduce. Software is an instrument too, and the argument of this chapter is that it fails in ways that are unusually hard to see — which is precisely why it needs the kind of deliberate checking that other instruments receive as a matter of course. How research code goes wrong The failures that make headlines are rarely exotic. They are mundane errors that survived because nothing was in place to catch them. Consider the case of the "Willoughby–Hoye" scripts. In 2014 Patrick Willoughby, Matthew Jansma and Thomas Hoye published a protocol in Nature Protocols for assigning the structures of small molecules by computing NMR chemical shifts, together with a set of Python scripts that automated part of the calculation. The protocol was widely used. In 2019 a graduate student at the University of Hawaii, Yuheng Luo, working with the chemist Rui Sun on the characterisation of compounds from a cyanobacterium, noticed that the scripts gave different answers on different computers. On one machine a computed value came out around 172.4; on another, around 173.2 — enough to change which candidate structure looked correct. The cause, reported in Organic Letters by Jayanti Bhandari Neupane and colleagues, was a single call to Python's glob function, which lists files matching a pattern. The Python documentation states that glob returns results in arbitrary order. On some operating systems the order happened to be alphabetical; on others it was not. The scripts silently assumed it was, and so matched the wrong output files to the wrong inputs. The fix was one line: sort the list. The authors estimated that well over a hundred published studies might have used the scripts. Nothing about this bug was unusual. The code was not badly written by the standards of academic software. It worked, repeatedly and convincingly, on the machines of the people who wrote it. It failed only when run somewhere else, and even then it did not crash — it produced plausible numbers that were wrong. That combination, plausible output from a quiet error, is the signature of most serious research software failures. The same pattern appears in the case with which this booklet opened. Geoffrey Chang's group did not publish nonsense; they published protein structures that were entirely credible to reviewers and to the field until other groups produced structures of related proteins that could not be reconciled with them. The in-house program that flipped two columns had presumably been run many times without complaint. There was no oracle — no case with a known answer — against which its output was ever compared. Spreadsheets deserve mention here, because much research computation still happens in them and they illustrate the same failure mode with particular clarity. In 2010 the economists Carmen Reinhart and Kenneth Rogoff published an influential paper associating high public debt with sharply lower growth. In 2013 Thomas Herndon, Michael Ash and Robert Pollin, working from the spreadsheet the authors had kindly shared, found that a formula averaging across countries omitted several rows, among other issues concerning data exclusion and weighting. The coding error was only one of several disputed choices, and its effect on the headline conclusion has been argued over ever since, but it was the error that captured public attention because it was so easy to understand. A range in a formula stopped a few cells short. Nothing warned anyone. Genomics offers a still more pervasive example. Microsoft Excel, by default, converts certain gene symbols into dates: SEPT2 becomes 2-Sep, MARCH1 becomes 1-Mar. In 2016 Mark Ziemann, Yotam Eren and Assam El-Osta screened supplementary files attached to papers in leading genomics journals and found that roughly a fifth of the papers with Excel gene lists contained such corrupted names. The problem was persistent enough that in 2020 the HUGO Gene Nomenclature Committee renamed the affected genes — SEPT1 became SEPTIN1, MARCH1 became MARCHF1 — partly to stop software mangling them. When a scientific naming authority changes the names of human genes to accommodate a spreadsheet's behaviour, the scale of the problem is hard to dispute. Why the errors stay hidden It would be comforting to think that these are the failures of careless people. They are not. They are the predictable result of three features of research software that distinguish it from most other code. The first is that research code often has no independent specification. A payroll system can be checked against the law and the contract: if an employee is paid the wrong amount, someone notices. Research code frequently computes something that nobody has computed before, which is the point of the research. If the output looks surprising, that may be a discovery or a bug, and the researcher has strong incentives to hope it is the former. If the output looks unsurprising, nobody checks it at all. The absence of a known right answer is what makes scientific testing distinctive, and Chapter 5 is largely about techniques for testing when the answer is unknown. The second is that research code evolves by accretion. A typical analysis starts as a short script to load some data and make a plot. It gains a cleaning step, then a special case for a malformed file, then an extra model, then a flag to switch between two versions of a preprocessing step because a reviewer asked. Each change is small and sensible. After two years the script is two thousand lines long, contains several abandoned approaches that are commented out or still run but whose output is ignored, and relies on global variables set at the top that are silently changed halfway down. Nobody designed it. Its structure is the fossil record of the project's history, and it is almost impossible to reason about any one part of it in isolation. Chapter 4 addresses how to prevent and undo this. The third is that research code runs in an environment nobody records. The result of an analysis depends not only on the code but on the versions of the language, the libraries, the operating system and occasionally the hardware. Default arguments change between library versions. Random number generators change their algorithms. A function deprecated in one release is removed in the next. The glob bug was an environmental failure in exactly this sense: the code was identical everywhere, and the environment determined the answer. Chapter 7 deals with capturing environments so that they can be reconstructed. These three features compound. Code with no specification is hard to test; code that grew by accretion is harder still; and code whose behaviour depends on an unrecorded environment may pass a test on one machine and fail it on another. The result is software whose correctness rests almost entirely on the confidence of its author. What the evidence says about reproducibility Anecdotes can mislead, so it is worth asking how common these problems are in the aggregate. Several large studies have tried to find out, and their results are sobering. In 2022 Ana Trisovic, Matthew Lau, Thomas Pasquier and Mercè Crosas reported in Scientific Data on an attempt to re-execute research code deposited in the Harvard Dataverse repository. They collected more than 9,000 R files from over 2,000 replication datasets published between 2010 and 2020 and ran them in a clean environment. Seventy-four per cent of the files failed to complete without error on the first attempt. After the authors applied automated cleaning — fixing common problems such as hard-coded absolute file paths and missing library calls — 56 per cent still failed. These were not random scripts found on the internet; they were code that researchers had deliberately packaged and deposited so that others could reproduce their work, often to satisfy a journal's policy. Earlier, Christian Collberg and Todd Proebsting at the University of Arizona attempted to build the software described in several hundred papers from computer systems conferences and journals — a field whose researchers are, if anyone is, professional programmers. They reported in Communications of the ACM in 2016 that for a substantial fraction of papers the code could not be obtained, and that of the code they did obtain, a large share could not be built without significant effort. Their project became a widely cited illustration that sharing code is not the same as sharing working code. Studies of Jupyter notebooks tell the same story. In 2019 João Felipe Pimentel and colleagues analysed over a million notebooks on GitHub and found that only a small minority could be re-executed to produce the same results, with failures caused by missing dependencies, cells run out of order, and references to files that did not exist in the repository. The notebook format encourages running cells in whatever order is convenient during exploration, which means that the state of the notebook at the moment its author saved it may be unreachable by running it top to bottom. Pimentel and his co-authors put numbers on this: of the notebooks they could attempt to run, only about a quarter executed without error, and only around four per cent reproduced the outputs stored in them. The common thread is that code which works on its author's machine, today, frequently does not work anywhere else or at any later date. This is not a moral failing. It is the natural state of software that has never been run anywhere but where it was written. The case for treating code as a method There is a useful reframing that runs through the rest of this booklet. When a paper describes an experimental procedure, the methods section is supposed to contain enough detail for a competent colleague to repeat it. Journals, funders and reviewers take this seriously, at least in principle. The code that transforms raw data into reported results is part of the method, often the most consequential part, and for most of the history of computational science it was described, if at all, in a sentence: "analyses were performed in R version 3.4 using custom scripts." That sentence is not a method. It is a promissory note. The practices in the chapters that follow are, collectively, how to turn it into something a colleague could actually use. Version control records exactly which code produced which result. Tests are the calibration record — the evidence that the instrument reads correctly against known standards. Modular structure makes the procedure legible, so that a reviewer can find and check the step that matters. Continuous integration shows that the tests pass on a machine other than the author's. A recorded environment specifies the conditions under which the procedure was carried out. Documentation and a licence make it possible for someone else to use it lawfully and correctly. The same reframing explains why these practices are not merely defensive. A laboratory with calibrated instruments runs experiments faster, not slower, because it does not have to rerun them when a result looks wrong. A researcher with tested, versioned, modular code can respond to a reviewer's request by changing one parameter and rerunning a pipeline, confident that nothing else has moved. The cost of engineering is paid once; the cost of its absence is paid every time the code must change. When the code is the public argument The cases above concern errors. A different kind of episode shows what happens when research code, even correct code, is exposed to scrutiny it was never built to withstand. In March 2020 the epidemic model developed by Neil Ferguson's group at Imperial College London informed the United Kingdom's decision to move towards a national lockdown. The model's code had been developed over many years, originally in C, as a single large file of many thousands of lines. When the group released a cleaned-up version on GitHub in the spring of 2020, working with software engineers from Microsoft and GitHub to restructure it, the code attracted intense public criticism. Commentators — some of them professional programmers, some with political motives — pointed to its size, its lack of tests, and reports that it could produce different results from the same inputs under some configurations. The debate was heated and often ill-informed about how stochastic simulations work; run-to-run variation is expected in such models, and the question is whether it is controlled and understood. The more careful response came from the reproducibility community. Stephen Eglen of the University of Cambridge, through the CODECHECK initiative, independently ran the released code and in June 2020 reported that he could reproduce the key results in the relevant Imperial report, within the variation expected from a stochastic model. The code, in other words, did what its authors said it did. But the episode showed how much credibility depends on things other than correctness. A model that has tests, a documented history, a declared environment and a structure that outsiders can follow is far easier to defend than one whose correctness must be established after the fact, under public pressure, by a volunteer. The Imperial team's code was vindicated; it would have been better for everyone had it never needed to be. The lesson for an ordinary researcher is not that their thesis code will be debated in newspapers. It is that code which cannot be inspected easily will be trusted, or distrusted, on grounds that have nothing to do with whether it is right. Engineering practices are, among other things, a way of making the evidence of correctness available before anyone asks for it. The costs researchers already pay That last claim deserves defending, because it is the heart of the booklet's argument and because it cuts against a strong intuition. Researchers tend to believe that engineering practices are expensive and their absence is free. In fact the absence has costs that are real but diffuse, and so go unaccounted. There is the cost of archaeology: the hours spent working out which of analysis_final.R, analysis_final_v2.R and analysis_final_REALLY.R produced the figures in the submitted manuscript, and whether the version on the laptop matches the version on the cluster. There is the cost of fear: the reluctance to improve code because any change might break something in a way that will not be noticed, which causes code to ossify around its earliest and worst decisions. There is the cost of revision: when a reviewer asks for an analysis with a different exclusion criterion, the researcher with a monolithic script must trace every place the criterion is used, while the researcher with a modular pipeline changes a configuration value. There is the cost of succession: when a doctoral student graduates, their code often becomes unusable within months, and the next student starts again from nothing, as if the previous years of work had been lost in a fire. And there is the tail risk: the small but real chance of an error that reaches publication, with consequences ranging from an embarrassing correction to a retraction and years of reputational damage. None of these costs appears on a budget line, which is why they are so easy to ignore. But anyone who has spent a week reconstructing an analysis to answer a reviewer, or inherited a predecessor's code and given up on it, has paid them. The chapters that follow are, in effect, an argument that the same effort spent up front buys a good deal more. Where to begin If this chapter has been persuasive, the natural temptation is to try to adopt everything at once. That rarely works. Engineering practices stick when each one solves a problem the researcher already feels, and when it is introduced at a moment when the cost of adoption is low. The order of the chapters reflects a sensible order of adoption. Version control comes first because it is cheap, immediately useful, and makes every subsequent change safe. Modular structure and tests come next, because they are what make the code trustworthy. Automation and environments follow, because they extend that trust beyond the author's own machine. Long-term maintainability comes last, because it matters most once the code has other users. The next chapter starts where every research project should: with a history of the code that you can trust. Chapter 2. Version Control as a Laboratory Notebook Every researcher has seen the folder. It contains model.py, model_old.py, model_new.py, model_new_fixed.py, model_thesis.py and model_thesis_JB_comments.py, along with a directory called backup_march whose relationship to the others is unclear. The folder is a version control system: an improvised, manual, unreliable one. Its owner knows that the history of the code matters and has tried to preserve it. What the folder cannot do is say what changed between any two files, why, or which one produced the result in the paper. Version control software does exactly those things, and the dominant tool, Git, is now so widely used that learning it is less a choice than a basic literacy. This chapter treats Git not as a tool for software teams but as what it can be for an individual researcher: a laboratory notebook for code, recording what was done, when and why, in a form that cannot be quietly altered and can always be returned to. What Git actually records Git was written by Linus Torvalds in 2005 to manage the development of the Linux kernel, after the project lost access to the proprietary tool it had been using. Its design reflects that origin: it is distributed, meaning every copy of a repository contains the full history, and it is built around content rather than files. Understanding a small amount of how it works makes the rest of it far less mysterious. A Git repository is a directory whose history Git tracks. The unit of history is the commit: a snapshot of the state of every tracked file at a moment in time, together with the name of the person who made it, a timestamp, a message describing it, and a pointer to the commit or commits that came before. Each commit is identified by a hash — a long string of hexadecimal characters computed from its contents and its ancestry. Because the hash depends on everything that came before, altering any past commit changes its hash and the hash of every commit after it. That property is what makes the history trustworthy: a commit hash cited in a paper identifies exactly one state of the code, and nobody can change that state without the change being detectable. Between the files on disk and the committed history sits the staging area, sometimes called the index. When you run git add on a file, you are not saving it to history; you are placing its current contents into the next commit you are assembling. git commit then records the staged snapshot. Newcomers often find the extra step irritating, but it has a real purpose for researchers: it lets you commit one logical change even when you have made several unrelated edits. If you fixed a bug in the data loader and also started experimenting with a new plotting style, you can stage and commit the bug fix alone, with a message that describes it, and leave the experiment for later. The last concept needed for solo work is the working tree — the files as they currently sit on your disk. git status compares the working tree and the staging area against the last commit and tells you what has changed. git diff shows the changes line by line. git log shows the history. With these five commands — status, add, commit, diff and log — a researcher already has most of the value of version control. Commits as notebook entries A laboratory notebook is useful to the extent that its entries are made at the right moments and say the right things. The same is true of commits. The right size for a commit is one coherent change: a bug fixed, a function added, a parameter changed, a figure restyled. A commit that does one thing can be understood, reviewed, and if necessary reversed on its own. A commit that bundles a week of work — "lots of changes" — is almost as unhelpful as no commit at all, because when something breaks it offers no way to find which of the many changes was responsible. Committing little and often is the single most important habit. Most experienced users commit several times an hour when actively working. The message is the notebook entry proper. Its first line should be a short summary, conventionally under about fifty characters, written as an instruction: "Fix off-by-one error in sliding window," "Add bootstrap confidence intervals to summary table." After a blank line, a longer body can explain why the change was made, which is the part future readers most need and the code itself cannot tell them. The code shows that the exclusion threshold changed from 0.05 to 0.01; only the message can record that this followed a discussion with a co-author about the reviewer's second comment. The diff is the what; the message is the why. There is a particular kind of commit that researchers should learn to make deliberately: the commit that corresponds to a result. When you generate the figures for a manuscript submission, commit first, and then record the commit hash alongside the output — in the figure's metadata, in a log file, or in the lab's own notes. Git's tags exist for this purpose. A tag is a human-readable name attached permanently to a commit: submitted-2026-03, revision-1, thesis-final. Six months later, when the reviewer's report arrives, git checkout submitted-2026-03 restores the exact code that produced the submitted results, and git diff submitted-2026-03 shows everything that has changed since. Making provenance automatic Recording hashes by hand works until the day it is forgotten, which is usually the day it matters. A more robust habit is to have the analysis record its own provenance. Both Python and R can ask Git for the current commit hash when a script starts — in Python by calling git rev-parse HEAD through the subprocess module, in R through system() or the gert package — and write it into every output: a line in a log file, a field in the metadata of a saved results table, or a small text label in the corner of a draft figure. The same call can check whether the working tree is clean. If there are uncommitted changes, the script can print a loud warning or refuse to run in "production" mode, because an output generated from uncommitted code cannot be traced to any state in the history. This small piece of machinery closes a gap that otherwise stays open for the life of a project. Every figure and table can be traced to exactly one commit, and every commit can be restored. When a co-author asks, eighteen months later, whether the numbers in Table 2 of the thesis were computed before or after the fix to the outlier filter, the answer is a lookup rather than an argument. A related discipline concerns rewriting history. Git allows local commits to be amended or reorganised before they are shared — git commit --amend fixes the message or contents of the most recent commit, and an interactive rebase can squash several work-in-progress commits into one. This is harmless and often tidy as long as the commits exist only on your machine. Once commits have been pushed to a shared remote, and especially once a hash has been cited anywhere, they should be treated as permanent. Rewriting published history breaks the guarantee that a hash identifies one state of the code, which is the whole point of recording it. What belongs in the repository A question that trips up many researchers is what to put under version control. The short answer is: everything a human writes, and nothing a machine generates or that is too large or sensitive to share. Source code belongs, obviously. So do configuration files, documentation, the text of the paper if it is written in LaTeX or Markdown, small hand-curated reference files, and the scripts that download or generate everything else. The file that records the software environment — requirements.txt, environment.yml, renv.lock or equivalent — belongs, for reasons Chapter 7 explains. Generated outputs generally do not belong. Figures, intermediate data, compiled binaries and cached results can be regenerated from the code, and committing them clutters the history and invites conflicts. There are sensible exceptions — a small table of final results kept under version control makes it easy to see when a code change alters a number — but the default should be to exclude them. Large raw data does not belong in a Git repository. Git stores every version of every file forever, and a repository containing a few gigabytes of binary data becomes slow to clone and painful to use. Data should live in an appropriate data repository or institutional store, with the repository containing a script that fetches it and, ideally, a checksum that verifies it. Tools such as Git LFS (Large File Storage), git-annex and DVC (Data Version Control) exist to bridge this gap for projects where data and code must be versioned together; they store large files elsewhere and keep lightweight pointers in Git. Secrets must never enter the repository: passwords, API keys, access tokens, and any personal or confidential data. Because Git preserves history, a secret committed once and deleted in the next commit is still present in the repository, and anyone with a copy can retrieve it. Removing it properly requires rewriting history, which is disruptive and, once the repository has been pushed to a shared service, may already be too late. Credentials should live in environment variables or local configuration files excluded from version control. The same caution applies with extra force to research data governed by ethics approvals or data protection law: participant data committed to a public repository is a reportable breach, not a technical inconvenience. The mechanism for exclusion is a .gitignore file in the repository's root, listing patterns for files Git should never track: data/raw/, .env, *.pdf in an output directory, the pycache folders Python creates, the .Rhistory file R leaves behind. Setting it up at the start of a project prevents a great deal of trouble later. Templates for common languages are widely available. Remotes, backups and sharing Git on a single laptop is a notebook that can be lost with the laptop. Its value multiplies when the repository is pushed to a remote — a copy hosted elsewhere, most often on GitHub, GitLab, Bitbucket, or an institution's own GitLab server. git push sends local commits to the remote; git pull fetches and merges commits from it. For an individual researcher, the remote is first of all a backup, and a better one than most, because it preserves not just the latest state but the entire history. It is also how code moves between machines: write on the laptop, push, pull on the cluster, run. This alone eliminates a class of errors in which the version on the cluster and the version on the laptop diverged without anyone noticing. The choice of host has practical consequences. GitHub is the largest and has the richest ecosystem of integrations, including the continuous integration service discussed in Chapter 6 and the Zenodo integration discussed in Chapter 8. GitLab can be self-hosted, which many universities do, keeping code on institutional infrastructure — sometimes a requirement for sensitive projects. Repositories can be private until publication and made public afterwards; there is no need to expose work in progress. What matters most is that the remote exists and that pushing to it becomes habitual. Using the history A history is only valuable if you use it, and Git offers tools that make the history an active aid to research rather than an archive. The simplest is reading it. git log with a file name shows every commit that touched that file; git log -p shows the changes themselves. When a number in the output changes unexpectedly, the log of the relevant files is the first place to look. git blame, despite its name, is not about assigning fault. It annotates each line of a file with the commit that last changed it. When you encounter a strange line — a magic constant, an unexplained special case — blame leads you to the commit that introduced it, and a well-written commit message explains why it is there. In inherited code this is often the only surviving explanation of a decision. The most powerful tool for researchers is git bisect. Suppose a result that was correct in January is wrong in June, and there are two hundred commits in between. bisect performs a binary search over the history: you mark a known good commit and a known bad one, Git checks out the commit halfway between, you test it and report good or bad, and Git halves the range again. After about eight steps — two to the eighth is 256 — it identifies the exact commit that introduced the problem. If the test can be scripted, git bisect run automates the whole search. This is only possible if commits are small and each one leaves the code in a runnable state, which is another reason to commit carefully. Finally, the history makes undoing things safe. git restore discards uncommitted changes to a file. git revert creates a new commit that undoes an earlier one, preserving the record that both happened. git checkout or git switch moves the working tree to any past commit or branch. The existence of these commands changes how people work: when every state can be recovered, experimentation stops being risky, and the collection of old and backup files becomes unnecessary. Notebooks, binary files and the limits of Git Git works best with plain text, because its diffs are line-based. Several formats common in research sit awkwardly with it. Jupyter notebooks are stored as JSON files that mix code with outputs, including embedded images and execution counters. A notebook whose code is unchanged but which has been rerun produces a large, meaningless diff. Merging two people's changes to the same notebook is often impractical. Several remedies exist. Tools such as nbstripout remove outputs before committing, so that only the code and text are tracked. Jupytext pairs each notebook with a plain-text script version — a Python file or an R Markdown file — that diffs cleanly. Increasingly, researchers use notebook formats that are plain text from the start, such as Quarto documents or R Markdown. More fundamentally, the analytical core of a project should not live in a notebook at all; notebooks are excellent for exploration and presentation, but the functions they call belong in modules that can be tested, as Chapter 4 argues. Word documents, Excel files, images and compiled binaries can be committed, but Git can only record that they changed, not how. For manuscripts written in Word, Git adds little beyond backup. For spreadsheets used as data entry, exporting to CSV alongside the original makes changes visible. Starting today Version control is the rare practice whose benefits begin immediately and whose cost is almost entirely a matter of habit. For an existing project, the steps are: initialise a repository in the project folder, write a .gitignore that excludes data, outputs and secrets, make a first commit of the current state, and push it to a private remote. From that moment, every change is recorded. The _old files can be deleted once they are safely committed, because the history now does their job better. The discipline that follows is small: commit whenever a coherent piece of work is done, write a message that says why, tag the commits that produce results, and push at the end of every session. Within a few weeks this becomes automatic, and it becomes difficult to remember how research was done without it. The next chapter builds on this foundation to address what happens when a project has more than one line of development — and more than one person. Chapter 3. Branches, Merges and the Shape of Collaboration A single line of history is enough for a researcher working alone on one thing at a time. Research is rarely like that. A doctoral student is halfway through restructuring the data pipeline when a supervisor asks for a quick variant of last month's figure. A postdoc wants to try a new estimation method without disturbing the version that the rest of the group depends on. Two students need to modify the same simulation code for different projects. A collaborator in another institution sends a fix. Each of these situations involves more than one line of development at once, and Git's answer to all of them is the branch. This chapter explains what branches are, how the main branching strategies used in industry differ, and which of them suits the scale and rhythm of academic work. Its argument is that most research groups should adopt a deliberately simple strategy — short-lived branches merged into a single main line through review — and resist the more elaborate models that were designed for problems researchers do not have. What a branch is In Git, a branch is nothing more than a movable label pointing at a commit. When you make a new commit on a branch, the label moves forward to point at it. The default branch in a new repository has historically been called master; since 2020, GitHub, GitLab and Git itself have moved to main as the default name, and this booklet uses main throughout. Creating a branch — git switch -c new-estimator — makes a new label pointing at the current commit. Commits made from then on advance the new label while leaving main where it was. You can switch back to main at any time, and the working tree returns to the state it was in before the experiment began. The two lines of development coexist, each with its own history, sharing everything up to the point where they diverged. Because branches are just labels, they are cheap. Creating one takes no time and no disk space. This matters because it changes behaviour: when branching is free, it becomes natural to start every piece of non-trivial work on its own branch, keeping main in a known, working state at all times. Merging, and what conflicts mean Eventually a branch's work either gets abandoned — in which case the branch is simply deleted — or brought back into main. Bringing it back is a merge. If main has not moved since the branch was created, Git can simply advance the main label to the branch's latest commit, which is called a fast-forward merge. If both have moved, Git creates a new merge commit with two parents, combining the changes from each. Git merges changes line by line. When the two branches modified different files, or different parts of the same file, the merge is automatic. When both changed the same lines in different ways, Git cannot know which version is right, and it reports a conflict: it marks the disputed region in the file with both versions and stops, asking a human to decide. Resolving a conflict means editing the file to the correct combined state, staging it, and completing the merge. Conflicts intimidate newcomers, but they are information, not failure. A conflict says that two lines of work made incompatible assumptions about the same piece of code, and a person needs to reconcile them. The practical lesson is that conflicts grow with the length of time branches live apart. A branch merged after a day rarely conflicts; a branch merged after three months almost always does, and the conflicts may be difficult to resolve because nobody remembers the reasoning behind either side. This observation, more than any other, shapes the choice of branching strategy. An alternative to merging is rebasing, which replays a branch's commits on top of the latest main as if the work had been started from there, producing a linear history without merge commits. Rebasing produces tidier histories and is popular in some teams. Its cost is that it rewrites the branch's commits, giving them new hashes, which causes trouble if anyone else has based work on them. For research groups the simple rule is: rebase your own unpublished branches if you like tidiness; never rebase anything others are using. Three strategies compared A branching strategy is a convention about which branches exist, how long they live, and how work flows between them. Three strategies dominate discussion, and they differ chiefly in how many long-lived branches they maintain and how quickly work returns to the main line, as Table 1 summarises. Table 1. Three common branching strategies compared. Strategy Long-lived branches Typical branch lifetime Designed for Fit for research groups Git Flow main, develop, plus release and hotfix branches Days to weeks Versioned releases supporting several versions at once Poor for most; useful only for large libraries with formal releases GitHub Flow main only Hours to days Continuously deployed services Good default for groups and shared tools Trunk-based development main only (the trunk) Hours, or direct commits Large teams with strong automated testing Good for solo work and mature, well-tested codebases Git Flow was described by Vincent Driessen in a 2010 blog post, "A successful Git branching model," which became one of the most widely copied pieces of writing about Git. It uses two permanent branches: main, which holds only released versions, and develop, where integration happens. Features are built on branches taken from develop; when a release is due, a release branch is cut, stabilised and merged into main with a version tag; urgent fixes to released versions go on hotfix branches taken directly from main. The model is thorough and handles the case of shipping numbered versions of software while maintaining older ones. It is also heavy. In 2020 Driessen added a note to the original post observing that for software delivered continuously, such as web applications, a simpler workflow like GitHub Flow was often more appropriate, and that Git Flow should not be treated as dogma. For a research group writing analysis code, the ceremony of release branches and a separate develop branch adds overhead with little benefit. GitHub Flow, described by Scott Chacon of GitHub in 2011, strips the model down to one rule: main is always in a working state. Any change, however small, is made on a descriptively named branch taken from main. When it is ready, the author opens a pull request, the change is discussed and reviewed, automated tests run, and the branch is merged into main and deleted. There are no develop or release branches. When a released version needs marking, it is tagged on main. Trunk-based development goes further still: developers commit to the main branch (the "trunk") directly or through branches that live only hours. It depends on strong automated testing, so that a broken commit is detected within minutes, and often on techniques such as feature flags that let incomplete work be merged without being switched on. Large software organisations use it at scale. For a single researcher working alone, committing directly to main is effectively trunk-based development, and it is perfectly reasonable as long as tests exist. A strategy fitted to research For most research groups, a lightly adapted GitHub Flow is the right default. It works like this. The main branch always contains code that runs and whose tests pass. Nobody commits broken work to it. Every piece of work — a new analysis, a bug fix, a refactoring — happens on a short-lived branch with a name that says what it is for: fix-date-parsing, add-mixed-model, reviewer2-sensitivity. When the work is done, it goes back into main through a pull request, ideally reviewed by someone else, with the automated tests described in Chapter 6 running first. The branch is then deleted. Results intended for publication are generated from main and tagged. Research does add one wrinkle that software teams rarely face: exploratory branches that may never be merged. A student may try three approaches to a modelling problem, each on its own branch, and adopt only one. This is a legitimate use of branches, and the abandoned ones need not be deleted immediately; they are a record of what was tried. But they should be recognisable as experiments, perhaps with a prefix such as explore/, and their conclusions should be written down somewhere more durable than a branch name before they are removed. A branch is not a lab notebook entry; a commit message or a short note in the project's documentation is. The other research-specific hazard is the branch that lives for a whole thesis. A student takes a copy of the group's shared model, makes changes for their own project over three years, and never merges anything back. By the time they graduate, their version and the group's have diverged so far that reconciling them is impossible, and the student's improvements are lost. The remedy is the same as for any long-lived branch: merge small, generally useful changes back to main promptly, and keep project-specific changes in project-specific code that uses the shared model rather than modifying it. Chapter 4's discussion of modularity is largely about making that separation possible. Pull requests and code review A pull request (GitLab calls it a merge request) is a proposal to merge one branch into another, presented on the hosting platform as a page showing every change, a thread for discussion, and the results of any automated checks. It is the natural point at which a second person looks at the code. Code review has a long pedigree in software engineering. Studies of practice at large companies, including Microsoft and Google, have found that its main benefits are not only catching defects but spreading knowledge of the codebase, maintaining consistency, and surfacing alternative approaches. For research groups, the knowledge-spreading benefit may be the most valuable. A group in which every change to the shared analysis code is seen by at least one other person is a group in which more than one person understands that code, which matters enormously when someone leaves. Review in a research setting should be proportionate and kind. It is not an examination. A reviewer might check that the change does what its description says, that there is a test for the new behaviour, that the code is readable, and that nothing obviously unsafe — a hard-coded path to someone's home directory, a password — has slipped in. Scientific review is also appropriate: does this preprocessing step make sense, is this the right statistical test? Two eyes on a statistical choice are worth a great deal. A reviewer should ask questions rather than issue verdicts, and authors should keep pull requests small enough to be reviewed in twenty minutes. A pull request of two thousand lines will not be reviewed; it will be approved. Small groups may worry that they lack anyone to review. Even a lone researcher benefits from the pull request as a discipline: opening one, reading your own diff on the web interface, and waiting for automated tests to pass before merging catches a surprising number of mistakes. And review need not come from a domain expert. A student in a neighbouring group, a research software engineer from a central team, or a co-author can review for clarity even if they cannot review the science. Protecting main Hosting platforms let you enforce the convention that main is always working. Branch protection rules can require that changes to main arrive only through pull requests, that automated tests pass before merging, and optionally that at least one other person approves. For a shared group repository these settings are worth turning on; they convert a convention that people forget under deadline pressure into a guard rail. For a solo project, requiring tests to pass is usually enough. Protection also reduces a specific academic hazard: the late-night fix pushed directly to the shared repository an hour before a co-author runs the final analysis for a submission. With protection in place, that fix goes through a pull request, its tests run, and the co-author sees it arrive. Collaboration across institutions Research collaborations frequently span groups that do not share infrastructure. Git's distributed design handles this well. The usual model on public platforms is the fork: an external collaborator makes their own copy of the repository under their account, works on branches there, and opens pull requests back to the original. The maintainers of the original retain control over what enters main, while outsiders can contribute without being granted write access. This is also how researchers contribute fixes to the open-source libraries they depend on — a practice worth encouraging, since a bug fixed upstream is fixed for everyone. Collaborations with sensitive code or data may need a private repository with invited members, or an institutional GitLab instance. The branching strategy remains the same; only the access controls change. Papers, revisions and frozen results One situation in research does call for something closer to a long-lived branch. A paper is submitted from a tagged commit on main. Work continues: the code is refactored, a dependency is upgraded, a new analysis is added for the next paper. Four months later the reviews arrive, asking for a sensitivity analysis on the submitted results. The researcher now needs to modify the code as it was at submission, not as it is today, because today's code may produce slightly different numbers for reasons unrelated to the review. The clean way to handle this is a revision branch taken from the submission tag: git switch -c revision-1 submitted-2026-03. The sensitivity analysis is added there, the revised results are generated and tagged, and any change that is generally useful — a bug fix discovered along the way, say — is merged or cherry-picked back into main. Git's cherry-pick command copies a single commit from one branch to another, which is exactly the operation needed when one fix should reach two lines of development. This is, in miniature, the problem that Git Flow's hotfix branches were designed to solve, and it is the one part of that model worth borrowing. The same logic applies to software released for others to use. A lab that publishes version 1.2 of a tool, then begins substantial work towards 2.0, may need to issue a bug-fix release 1.2.1 for users who cannot upgrade yet. A maintenance branch taken from the 1.2 tag handles this. Most research projects will never need it, and should not create such branches in anticipation; the point is that the simple strategy extends naturally when the need arises. A worked example Consider a group of four — a principal investigator, a postdoc and two doctoral students — sharing a repository that contains a library of preprocessing and modelling functions used by everyone's projects. Under the approach described here, the postdoc notices that a function for detrending time series mishandles missing values. She creates a branch named fix-detrend-missing, writes a test that reproduces the problem, fixes it, and opens a pull request. The automated tests run in a few minutes and pass. One of the students, who uses the function heavily, reviews it that afternoon, asks one question about how the fix treats a series that is entirely missing, and approves after a short exchange. The branch is merged and deleted. The next morning both students pull main and have the fix. Nobody emailed a zip file; nobody's thesis now depends on a private copy of the function; and the history records not only the fix but the discussion that shaped it. The whole episode took perhaps two hours of combined effort, most of it the fix itself. The underlying principle Underneath the details, every successful branching strategy rests on one idea: integrate often. Work that stays separate accumulates divergence, and divergence becomes conflict. GitHub Flow and trunk-based development are both, at heart, ways of shortening the time between writing code and merging it. Research groups should adopt whichever version of this idea they can sustain, and they should add process only when a specific problem demands it. The next chapter turns from the history of the code to its structure — because how easily code can be branched, merged and reviewed depends heavily on how it is organised. Hashtags: #ResearchSoftwareEngineering #ResearchSoftware #VersionControl #Git #GitHub #SoftwareReproducibility #UnitTesting #ModularCode #CodeQuality #ScientificSoftware #ResearchCode #ContinuousIntegration #SoftwareTesting #CodeReview #BranchingStrategies #PullRequests #SoftwareProvenance #ReproducibleResearch #ComputationalReproducibility #Containerization #Docker #Apptainer #SoftwareDocumentation #ResearchCodeManagement #FutureOfResearchSoftwareEngineering

Download the Book (PDF): Introduction In July 1972 the Associated Press reporter Jean Heller published a story that began with a sentence no official could explain away. For forty years, she wrote, the United States Public Health Service had been conducting a study in which Black men with syphilis went without treatment so that doctors could observe what the disease did to the human body. The study had begun in Macon County, Alabama, in 1932. Penicillin had become the standard cure in the late 1940s. The men had not been given it. Many had been told they were being treated for "bad blood." Some had died of the disease, some had infected their wives, and some of their children had been born with congenital syphilis. The study had been discussed at professional meetings and published in medical journals across its entire life. It was not hidden. It was simply not considered wrong by the people in a position to stop it. That last point is the one worth holding on to. The history of research ethics is often told as a story of villains exposed and rules adopted to stop them. There were villains, and there were rules. But the more troubling pattern is how often harmful research was carried out by respected investigators at respected institutions, reviewed by colleagues, funded by governments, and published without objection. The failures were not usually secret. They were ordinary. And the system of protections that followed was designed, above all, to interrupt ordinary professional judgement before it could go wrong again. This book traces how that system came to be, how it works now, and where it is straining. It begins with the Nazi medical experiments and the Nuremberg Code of 1947, a document that stated the principle of voluntary consent with a clarity that has never been bettered and was then largely ignored by the countries that wrote it. It moves through the scandals that finally forced change in the United States in the 1960s and 1970s, including Tuskegee, the hepatitis studies at the Willowbrook State School and the injection of live cancer cells into elderly patients at the Jewish Chronic Disease Hospital in Brooklyn. It follows the construction of the modern framework: the National Research Act of 1974, the Belmont Report of 1979, the federal regulations that became the Common Rule, the successive revisions of the World Medical Association's Declaration of Helsinki, and the international guidelines that now govern clinical trials run across many countries at once. The later chapters turn to how that framework operates in practice and where it falls short. They examine what institutional review boards actually do, and what they cannot do. They look closely at informed consent, the central ritual of the whole system, and at the stubborn evidence that participants often do not understand what they have agreed to. They consider how the rise of biobanks, genomic databases and digital health records has broken the assumption that consent is a single event with a clear beginning and end, and they assess the models, including broad, tiered and dynamic consent, that have been proposed to replace it. They examine the idea of the "vulnerable population," which began as a protective category and has sometimes turned into a reason to exclude whole groups of people from the research that might have helped them. And they follow clinical research as it has moved across borders, to countries where oversight may be weaker and where the gap between what participants need and what the trial offers may be wide. The argument of this book The argument running through these chapters is simple to state. The protections built after the great scandals were designed as gates. An investigator submits a protocol, a committee reviews it before anything happens, a participant signs a form before enrolment, and the ethical work is, in most practical respects, done. The gate model made sense for the abuses it was built to prevent. Tuskegee, Willowbrook and the Nazi experiments were failures of permission: people were enrolled without knowing what was being done to them, or without any real choice, in studies no independent body had ever questioned. Requiring prior review and prior consent was the right answer to those failures. But the risks of research have been moving. Increasingly they arise after the gate has closed. A blood sample given for a diabetes study is later used, without the donor's knowledge, to study schizophrenia and ancestral migration. A genome deposited in a research database under a promise of anonymity turns out to be re-identifiable from public genealogy records. A trial approved in one country is run in another, where the local committee has fewer resources and where participants have no access to the treatment once the trial ends. A participant consents in good faith at enrolment, and the study changes shape over the following decade in ways no one could have described to her at the start. None of these problems is solved by a better form or a stricter committee at the entrance. They are problems of stewardship over time. The claim of this book, then, is that human research protection is in the middle of a transition from gatekeeping to stewardship: from ethics as a checkpoint passed once, to ethics as a continuing relationship between researchers, institutions and the people whose bodies and data make research possible. The transition is incomplete and uneven. Some of the most important recent changes, such as the revised Common Rule that took effect in the United States in 2019, the European Union's Clinical Trials Regulation, the 2024 revision of the Declaration of Helsinki and the 2025 revision of the international Good Clinical Practice guideline, move in this direction without quite committing to it. Understanding why the old model was built the way it was is the best way to see what a better one would require. What the reader will find here The book is written for readers who need to understand research ethics without necessarily being specialists in it: students entering the health and social sciences, researchers preparing their first protocols, members of ethics committees, research coordinators, journalists, policy staff and participants who want to know what protections they are actually owed. It assumes no background in law or philosophy. Where it describes regulations it does so in plain terms, naming the actual provisions where that helps a reader find them. It concentrates on biomedical and clinical research, because that is where most of the formal machinery was built and where the stakes of failure are most visible, but it also draws on social and behavioural research where the lessons carry over. The famous obedience experiments of Stanley Milgram, the covert observation in Laud Humphreys's Tearoom Trade, and the 2014 Facebook experiment that altered the news feeds of nearly 700,000 users to study emotional contagion each show a different side of the same problem: what happens when people become the material of research without meaningfully agreeing to it. The United States features prominently, because its scandals and its regulatory responses shaped the international field and because its regulations are still the template many institutions around the world follow. But the book also describes the European framework, the Council for International Organizations of Medical Sciences guidelines, the ethics review systems of countries that now host large numbers of trials, and the international Good Clinical Practice standard that sponsors use to run studies across dozens of jurisdictions at once. A word about tone. Research ethics is sometimes written as a catalogue of horrors, and sometimes as a bureaucratic manual. Neither serves the reader well. The horrors matter because they reveal how decent people rationalise harm; the procedures matter because they are the mechanism by which a society tries not to repeat it. The aim here is to keep both in view, and to treat the people who were harmed not as illustrations but as the reason the subject exists at all. The men in Macon County were not asked. The families of the children at Willowbrook were offered a place in a crowded institution if they agreed to the research. The Havasupai who gave blood in the Grand Canyon believed they were helping to fight diabetes. Every rule described in the chapters that follow is, at bottom, an attempt to make sure that the next person in their position is asked, is told the truth, is free to say no, and is not forgotten once the form is signed. Whether the rules succeed at that last part is the question this book is most concerned to answer. Chapter 1. The Doctors' Trial and the Code Nobody Adopted On 9 December 1946, in the same Nuremberg courthouse where the leading officials of the Third Reich had just been tried, an American military tribunal opened proceedings against twenty-three defendants, twenty of them physicians. The case was formally United States of America v. Karl Brandt et al., and it became known as the Doctors' Trial. Brandt had been Hitler's personal physician and a senior administrator of the programme that killed disabled people in institutions across Germany. His co-defendants included professors, senior officers of the medical services and doctors who had worked in the concentration camps. The charges concerned both the so-called euthanasia killings and a long series of experiments carried out on prisoners without their consent. The trial ran until August 1947. Sixteen defendants were convicted, and seven were sentenced to death and hanged the following year. The verdict is remembered less for its sentences than for a passage in the judgment in which the judges set out ten principles for "permissible medical experiments." That passage, never given a formal title by the court, came to be called the Nuremberg Code. Its first sentence is the most quoted line in the history of research ethics: "The voluntary consent of the human subject is absolutely essential." This chapter asks two questions about the Code. What exactly did it respond to? And why, having stated the principle of consent so plainly, did it have so little effect for the next twenty years on the countries whose judges had written it? What the experiments were It is tempting to treat the Nazi experiments as simple sadism dressed in laboratory coats, and some of them were close to that. But most were designed to answer questions that the German military or health authorities regarded as urgent, and they were run by people with genuine scientific training. That is precisely what made them instructive to the tribunal. At Dachau, the Luftwaffe physician Sigmund Rascher conducted high-altitude experiments in which prisoners were placed in low-pressure chambers to simulate the conditions a pilot might face after bailing out at great height. Many died. Rascher also ran freezing experiments, immersing prisoners in tanks of ice water for hours to study hypothermia and methods of rewarming, because German pilots were ditching in the North Sea. At Ravensbrück, women prisoners were deliberately wounded, and the wounds contaminated with bacteria, wood shavings and glass, to test whether sulfonamide drugs could prevent the infections that were killing soldiers on the Eastern Front. Other Ravensbrück prisoners had bones, muscles and nerves removed in experiments on regeneration and transplantation. At Buchenwald and elsewhere, prisoners were infected with typhus to test vaccines. At Dachau, the tropical medicine specialist Claus Schilling infected more than a thousand prisoners with malaria. Experiments on mass sterilisation, by X-ray and by chemical injection, were carried out at Auschwitz and Ravensbrück as part of the regime's racial programme. Josef Mengele's experiments on twins at Auschwitz, perhaps the most infamous of all, did not feature in the trial because Mengele had escaped. Several features of these experiments recur, in milder forms, throughout the rest of this book. The subjects were drawn from populations already stripped of rights: prisoners, Jews, Roma, Poles, Soviet prisoners of war. Their availability was treated as an opportunity. The research questions were framed in terms of the needs of others, soldiers and airmen above all, so that the suffering of the subjects was weighed against a large and diffuse benefit to the nation. And the investigators were embedded in institutions that approved, funded and published their work. Some of the hypothermia data were later discussed at a German medical conference in 1942, to an audience of physicians who raised no public objection. The argument of the defence The defendants did not, for the most part, deny what they had done. Their lawyers argued instead that there was no settled international standard against which their conduct could be judged, and that doctors in the Allied countries had done similar things. The defence pointed to experiments conducted in the United States during the war, including the malaria studies on prisoners at the Stateville Penitentiary in Illinois, in which inmates were infected in the course of testing antimalarial drugs for troops in the Pacific. They cited older examples too: Walter Reed's yellow fever experiments in Cuba in 1900, in which volunteers were deliberately exposed to infected mosquitoes, and the work of researchers who had tested treatments on prisoners in the Philippines early in the century. The point was not that these experiments were identical to Dachau, but that the line between acceptable and unacceptable research was nowhere written down. The Reed example cut in more than one direction. The yellow fever commission had asked its volunteers, many of them recently arrived Spanish immigrants, to sign written agreements, in Spanish as well as English, which acknowledged that the disease could be fatal and offered payment in gold, with a further sum for those who fell ill. These documents are often described as among the earliest written consent forms in medical research. They show that the idea of telling people about the risk and securing their agreement in writing was familiar to at least some American investigators at the turn of the century. They also show, uncomfortably, how early the tension between payment and free choice appeared: a large sum offered to poor men for accepting the chance of a deadly infection raises exactly the question of undue inducement that ethics committees still debate today. Several volunteers did contract yellow fever, and Reed's colleague Jesse Lazear died of it after being bitten, whether deliberately or accidentally remains disputed. The prosecution relied heavily on two American medical experts. Andrew Ivy, a physiologist from Illinois, had been sent by the American Medical Association, and Leo Alexander, a Boston neuropsychiatrist of Austrian origin, served as a consultant to the prosecution. Both argued that there were in fact well-understood principles of medical ethics, including the requirement of consent, and that the defendants had violated them. Ivy's position was complicated by the Stateville studies, which had been conducted in his own state. While the trial was under way, a committee convened by the governor of Illinois examined the ethics of prison research and concluded, in terms that supported the prosecution's case, that prisoners could volunteer for research provided their consent was free. Historians have since noted how conveniently timed that conclusion was. The defence argument had one embarrassing weakness. Germany itself had issued detailed guidelines on human experimentation in 1931. The Reich Circular on new therapies and human experimentation required consent, prohibited experiments on dying patients and required special caution with children. By the standards of the time it was one of the most demanding research codes anywhere. It had never been repealed. Germany had not lacked rules; it had created conditions in which rules ceased to matter. This is an early version of a lesson the rest of this book repeats: written standards protect people only when institutions are willing to enforce them against their own members. There was an even older German precedent. In the late 1890s the Breslau dermatologist Albert Neisser injected serum from syphilis patients into women, several of them sex workers, without telling them, in an attempt to develop a vaccine. Some later developed syphilis. The resulting public controversy led the Prussian government in 1900 to issue a directive requiring unambiguous consent after proper explanation for any non-therapeutic intervention. Consent as a principle, in other words, was not invented at Nuremberg. It had been stated, and set aside, before. The ten points The Code the judges wrote contains ten principles. The first, on consent, is by far the longest. It requires that the person involved have legal capacity to consent, be able to exercise free power of choice without "any element of force, fraud, deceit, duress, over-reaching, or other ulterior form of constraint or coercion," and have sufficient knowledge and comprehension of the elements of the experiment to make an understanding and enlightened decision. It lists what the person must be told: the nature, duration and purpose of the experiment, the method and means, the inconveniences and hazards reasonably to be expected, and the effects on health that may follow. And it places the duty to ensure the quality of consent on each individual who initiates, directs or engages in the experiment, a duty that "may not be delegated to another with impunity." The remaining nine principles concern the design and conduct of the research. The experiment should be such as to yield fruitful results for the good of society, unprocurable by other means. It should be based on the results of animal experimentation and knowledge of the natural history of the disease. It should avoid all unnecessary physical and mental suffering. No experiment should be conducted where there is reason to believe that death or disabling injury will occur, except perhaps where the experimental physicians also serve as subjects. The degree of risk should never exceed the humanitarian importance of the problem. Proper preparations and facilities should protect the subject against even remote possibilities of injury. The experiment should be conducted only by scientifically qualified persons. The subject should be at liberty to bring the experiment to an end. And the scientist in charge must be prepared to terminate the experiment if continuation is likely to result in injury, disability or death. Read today, the Code is striking both for what it contains and for what it omits. It has no provision for anyone who cannot consent: children, people with serious mental illness, the unconscious. Taken literally, it forbids all research on them, including research designed to benefit them. It makes no distinction between research on healthy volunteers and research combined with medical care, a distinction that would later become central to the Declaration of Helsinki. It places the entire burden of protection on the individual investigator and says nothing about independent review. There is no committee in the Nuremberg Code. The judges assumed that a properly conscientious scientist, bound by these principles, would be enough. That assumption turned out to be the Code's deepest flaw, and the history of the next thirty years is largely the history of discovering it. A code for barbarians The Nuremberg Code had no formal legal force in any country. It was part of a judgment by a military tribunal applying international law to defendants accused of crimes against humanity. Outside that context, it was a statement of principles that anyone could choose to ignore, and in the United States, Britain and elsewhere, the medical profession largely did. The reasons were partly professional and partly psychological. American physicians could read the Code as a response to atrocities committed by people fundamentally unlike themselves. The legal scholar and psychiatrist Jay Katz, who spent much of his career studying the gap between the Code and practice, argued that many American researchers regarded it as a good code for barbarians but an unnecessary one for ordinary physicians. The ideal of the ethical physician, guided by conscience and professional tradition, seemed an adequate safeguard. Formal consent requirements were seen as a legalistic intrusion into the doctor's judgement about what was good for a patient, and the line between patient and research subject was, in the clinics of the 1950s, often not drawn at all. The scientific context also discouraged reflection. The postwar decades were a period of extraordinary expansion in medical research, driven in the United States by the growth of the National Institutes of Health and by the conviction, forged by penicillin, radar and the atomic bomb, that organised science could solve almost any problem. The federal research budget grew rapidly through the 1950s and 1960s. Clinical research became a career in its own right, with its own incentives to publish and to secure further funding. Subjects were needed in large numbers, and they were found in the places where they had always been found: charity wards, prisons, institutions for the disabled, and military bases. The attitude of governments was no better. In the early 1950s the United States Department of Defense adopted a version of the Nuremberg principles for its own research on atomic, biological and chemical warfare, in a memorandum issued by Secretary of Defense Charles Wilson in 1953. The memorandum was classified, and its requirements were unevenly followed. The later investigations of the Advisory Committee on Human Radiation Experiments, established by President Clinton in 1994, documented a long series of Cold War studies in which people were exposed to radiation without adequate information, including the injection of plutonium into hospital patients in the 1940s and the feeding of radioactive tracers to children at the Fernald State School in Massachusetts in the late 1940s and early 1950s. The committee found that many officials had been aware of the ethical principles and had not applied them. There is a further, less often discussed, case. Japan's Unit 731, a biological warfare research unit in occupied Manchuria led by the army physician Shiro Ishii, conducted lethal experiments on thousands of prisoners, mostly Chinese, during the 1930s and 1940s, including deliberate infection with plague and anthrax, vivisection and frostbite experiments. Its leaders were never tried by the Tokyo tribunal. Declassified documents have since shown that American officials, interested in the unit's biological warfare data, granted its members immunity in exchange for access to their findings. The Nuremberg Code was announced at almost the same moment that the Allied powers chose to treat another set of human experiments as a source of useful knowledge rather than a crime. What Nuremberg settled and what it left open It would be unfair to say that the Code achieved nothing. It gave the principle of voluntary consent a place in international law and a canonical wording that every later framework has had to address. It established that research ethics is not simply a private matter between doctor and patient but a subject on which courts and societies can pass judgement. And it provided a benchmark against which later abuses could be measured; the critics of the 1960s were able to say, with force, that American researchers were doing things the Code forbade. What Nuremberg did not settle was how such principles could be made to govern the everyday conduct of respectable science. Its model of protection rested on the conscience of the individual investigator. It assumed that the danger lay in bad people, and that good people, once reminded of the rules, would follow them. The following two decades demonstrated that this assumption was false. The researchers who ran the studies described in the next chapter were not Nazis. They were, by the standards of their profession, conscientious, accomplished and in many cases admired. Some of them believed sincerely that what they were doing was good for their subjects. It was precisely because conscience failed in such people that the answer, when it came, took the form of external review: a committee that would look at a protocol before any subject was enrolled and ask whether it should go ahead at all. That answer, the gate, is the institution whose strengths and limits run through the rest of this book. It was invented not because the Nuremberg principles were wrong, but because stating a principle had turned out to be the easy part. The difficulty was building a system that would apply the principle to people who were sure they did not need it. Chapter 2. Scandals at Home: Tuskegee, Willowbrook and the Beecher Exposé In June 1966 the New England Journal of Medicine published an article by Henry K. Beecher, a professor of anaesthesia research at Harvard Medical School, under the modest title "Ethics and Clinical Research." Beecher was not an outsider or a campaigner. He was one of the most eminent clinical investigators in the United States, and he had spent much of his career running studies on human beings, including research on pain and on the placebo effect. That was what gave his article its force. He described twenty-two studies, all published in reputable journals by researchers at reputable institutions, in which he believed subjects had been placed at serious risk without their informed consent. He did not name the investigators. He did not need to. Anyone in the field could look up the references. The examples included the withholding of known effective treatment from patients, among them servicemen with streptococcal infections who were denied penicillin in a study of rheumatic fever prevention, and charity patients who were given no treatment for typhoid in order to compare outcomes. They included studies in which liver damage was deliberately induced, in which cancer cells were injected into patients who did not know what they were receiving, and in which children with intellectual disabilities were deliberately infected with hepatitis. Beecher's central claim was that such research was not the work of a few rogues. It was, as he put it, a matter of widespread practice, and it arose mostly from thoughtlessness and carelessness rather than wilful disregard of the patient's rights. Beecher's article is the natural hinge of this chapter because it named the problem that Nuremberg had not. The trouble was not that American medicine contained monsters. The trouble was that the culture of clinical research allowed conscientious people to do things to their subjects that, looked at plainly, they could not defend. The Jewish Chronic Disease Hospital and Willowbrook Two of the cases Beecher alluded to became the subject of public controversy in their own right, and together they illustrate how professional prestige shielded questionable research. In July 1963 Chester Southam, a respected cancer researcher at the Sloan Kettering Institute in New York, arranged for live cultured cancer cells to be injected under the skin of twenty-two elderly, chronically ill patients at the Jewish Chronic Disease Hospital in Brooklyn. Southam was studying how the immune system rejected cancer cells, and his hypothesis was that debilitated patients would reject them more slowly than healthy people. He had previously injected cancer cells into prisoners at the Ohio State Penitentiary, who had volunteered. At the hospital, the patients were told they were receiving an injection to test their immune response. They were not told that the cells were cancerous. Southam's view was that the cells would be rejected and posed no real risk, and that telling patients the word "cancer" would cause them needless distress. Three young physicians at the hospital refused to take part and resigned. A member of the hospital's board, the lawyer William Hyman, went to court to obtain the patients' records. In 1965 the Board of Regents of the University of the State of New York found Southam and the hospital's medical director, Emanuel Mandel, guilty of fraud, deceit and unprofessional conduct. Their licences were suspended for a year, but the suspensions were stayed and they were placed on probation. The penalty was mild, and the professional consequences milder still: a few years later Southam was elected president of the American Association for Cancer Research. The case shows the profession's reluctance, even in the face of a formal finding of deceit, to treat such conduct as seriously wrong. The Willowbrook State School on Staten Island was an institution for children with intellectual disabilities, chronically overcrowded and understaffed. Hepatitis was endemic there. From the mid-1950s into the early 1970s, the paediatrician Saul Krugman of New York University led a research programme in which newly admitted children were deliberately infected with hepatitis, at first by being fed extracts made from the faeces of infected patients, in order to study the course of the disease and to test whether gamma globulin could protect against it. The research was scientifically important. It helped to distinguish the two forms of viral hepatitis now known as hepatitis A and hepatitis B and contributed to the path toward a hepatitis B vaccine. Krugman and his defenders offered a set of justifications that are worth examining because they recur in later debates. The children were likely to contract hepatitis anyway, given conditions in the institution. The infections were induced under controlled conditions in a special unit with better care than the rest of the school. The strain used was believed to produce a relatively mild illness in children. And the parents had given consent. Critics replied that the "they would have caught it anyway" argument made the institution's own failings a licence for research; that the proper response to endemic hepatitis was to improve conditions, not to study them; and that the consent was compromised. In the mid-1960s, when overcrowding led Willowbrook to close its general admissions, places remained available in the hepatitis research unit. Parents seeking a place for their child were, in effect, told that there was room if they agreed to the study. A choice offered to a family with no other option for care is not the free choice the Nuremberg Code had described. The Willowbrook case has a lasting place in teaching because the arguments on both sides are serious. Krugman was a humane and accomplished physician who went on to receive major honours, including a Lasker award in 1983. The research produced real knowledge. Yet it depended on a population that had been placed in an institution by a society that had largely stopped looking at it, and on a consent obtained under circumstances that made refusal costly. The case is the clearest early example of what later ethicists would call structural vulnerability: harm that arises less from any single decision than from the conditions under which decisions are made. Tuskegee No case shaped American research ethics more than the Tuskegee Study of Untreated Syphilis in the Negro Male, to give it the name used by the Public Health Service itself. It began in 1932 in Macon County, Alabama, one of the poorest counties in the country, in collaboration with the Tuskegee Institute. The original plan was a short observational study of the effects of untreated syphilis in Black men, justified in part by a belief that the disease behaved differently in Black and white patients. It lasted forty years. About six hundred men were enrolled, roughly four hundred with latent syphilis and two hundred without the disease as controls. They were not told they had syphilis. They were told they had "bad blood," a local term covering a range of ailments, and were offered free medical examinations, meals on examination days, transport and a burial stipend in exchange for allowing an autopsy after death. Some of the procedures presented to them as treatment, including painful spinal taps, were purely diagnostic. A Black public health nurse, Eunice Rivers, served as the men's point of contact throughout the study and was central to keeping them enrolled. The ethics of the study were questionable from the start, since treatments for syphilis in the 1930s, although toxic and imperfect, did exist. They became indefensible after penicillin was established as an effective cure in the 1940s. The men were not offered it. The Public Health Service took active steps to keep them from treatment elsewhere, including providing lists of study participants to local physicians and, during the Second World War, to draft boards, so that men called up for military service would not be treated as other syphilitic draftees were. The study continued through the Nuremberg trial, through the adoption of the Nuremberg Code, through the Declaration of Helsinki in 1964, and through the publication of Beecher's article. It ended because of one persistent employee and one reporter. Peter Buxtun, a venereal disease interviewer with the Public Health Service in San Francisco, learned of the study in the mid-1960s and began raising objections internally from 1966. In 1969 a panel convened by the Centers for Disease Control reviewed the study and recommended that it continue. Buxtun eventually took the story to the press. Jean Heller's Associated Press report appeared in July 1972, and the study was halted later that year. Hearings in the Senate, led by Edward Kennedy, followed in 1973. A class-action lawsuit brought on behalf of the men by the civil rights lawyer Fred Gray was settled in 1974 for about ten million dollars, and the government agreed to provide lifetime medical care to the survivors and, later, to their affected wives and children. In May 1997 President Clinton issued a formal apology at the White House, in the presence of several of the surviving men. The harm was not confined to the men and their families. Tuskegee became a symbol, in Black communities across the United States, of the willingness of medical institutions to treat Black lives as expendable. Its effects are measurable. In a study published in the Quarterly Journal of Economics in 2018, the economists Marcella Alsan and Marianne Wanamaker found that the 1972 disclosure was followed by increased medical mistrust and reduced use of health care among older Black men, and estimated that these effects were associated with a meaningful reduction in their life expectancy. Research abuse, in other words, damages not only its direct victims but the relationship between whole communities and the institutions meant to serve them. That damage makes it harder to recruit the very populations whose underrepresentation in research later chapters of this book will discuss. Guatemala: the scandal found later The same agency produced a scandal that remained hidden for more than sixty years. In 2010 the historian Susan Reverby, working in the papers of the Public Health Service physician John Cutler at the University of Pittsburgh, found records of experiments conducted in Guatemala between 1946 and 1948. Cutler, who later also worked on the Tuskegee study, had led a programme in which prisoners, soldiers, psychiatric patients and sex workers in Guatemala were deliberately exposed to syphilis, gonorrhoea and chancroid in order to test prophylactic methods. In some cases infected sex workers were brought into prisons; in others, bacteria were introduced through abrasions. The research was funded by the United States government and conducted with the cooperation of Guatemalan officials. Following Reverby's disclosure, the American government apologised to Guatemala, and President Obama asked the Presidential Commission for the Study of Bioethical Issues to investigate. Its 2011 report, titled "Ethically Impossible": STD Research in Guatemala from 1946 to 1948, found that more than 1,300 people had been deliberately exposed to these diseases, that far fewer were documented as receiving adequate treatment, and that the researchers had understood at the time that what they were doing would not have been accepted at home. The Commission noted that the experiments were being carried out in the same years that American judges were sitting in judgement at Nuremberg. Guatemala foreshadows a theme of the final chapter of this book: research that would not pass at home has repeatedly been conducted abroad, on populations with less power to object. Social and behavioural research The crisis was not limited to medicine. In the early 1960s the Yale psychologist Stanley Milgram ran his obedience experiments, in which participants were told to administer what they believed were increasingly severe electric shocks to another person, in fact an actor, as part of a supposed study of learning. Many continued to the highest levels on the instructions of the experimenter. The findings were profound, and the participants were debriefed, but many had experienced considerable distress, and the study relied on deception as its central method. In 1970 the sociologist Laud Humphreys published Tearoom Trade, a study of anonymous sexual encounters between men in public toilets. Humphreys had acted as a lookout to observe the encounters, noted the licence plates of the men involved, traced their addresses, and later interviewed them in their homes under a false pretext as part of a separate health survey. In 1971 the Stanford Prison Experiment, led by Philip Zimbardo, was stopped after six days when the simulated prison environment produced serious distress among participants. These studies raised different questions from the medical scandals. The risks were psychological and social rather than physical: distress, humiliation, loss of privacy, the threat of exposure in a period when homosexual conduct was criminal in most American states. The methods, including deception and covert observation, were thought by many researchers to be indispensable to answering important questions. They showed that any framework of protections would have to cover research in which harm was not a matter of needles and drugs, and that consent could be compromised by the design of a study as much as by the neglect of an investigator. From exposure to regulation By the late 1960s the pieces of a response were already forming, though slowly. The thalidomide disaster, in which a sedative prescribed to pregnant women in Europe and elsewhere caused severe birth defects in thousands of children, prompted the United States Congress to pass the Kefauver-Harris Amendments to the Food, Drug, and Cosmetic Act in 1962. Among their provisions was a requirement that investigators obtain the consent of people receiving experimental drugs, though with exceptions where the investigator judged consent not feasible or contrary to the patient's best interests. The Food and Drug Administration strengthened the requirement over the following years. More important for the long run was a policy issued by the Surgeon General, William Stewart, in February 1966, a few months before Beecher's article appeared. It required institutions receiving Public Health Service research funds to provide for prior review of research involving human subjects by a committee of the investigator's "institutional associates," which would assess the rights and welfare of the subjects, the appropriateness of the methods of obtaining consent, and the risks and potential benefits. This was the germ of the institutional review board. For the first time, the judgement of the individual investigator would be checked, before the research began, by someone else. The policy was an administrative condition of funding rather than a law, and its implementation was uneven. Some institutions set up serious committees; others treated review as a formality. What finally turned it into a national system was Tuskegee. The public outrage of 1972 and the Kennedy hearings that followed made it impossible to leave the matter to the discretion of the profession and its funders. The next chapter describes the statute that followed, the national commission it created, and the report that commission produced, which remains the moral foundation of human research regulation in the United States. Taken together, the cases in this chapter reveal a consistent pattern. The subjects were almost always people with limited power: the poor, the institutionalised, the imprisoned, the elderly, the colonised, members of racial minorities. The research was almost always conducted in the open and justified by appeal to scientific value or to the argument that subjects were no worse off than they would otherwise have been. And the people best placed to object, the investigators' own colleagues, usually did not. A system of protection built on the conscience of individual researchers had been tried for a generation after Nuremberg. It had failed, not dramatically but routinely, and that routine failure is what the gatekeeping model was designed to prevent. Chapter 3. From Outrage to Architecture: Belmont, the Common Rule and Helsinki Scandals produce anger, and anger produces demands that something be done. What gets done depends on who is in the room when the response is designed. In the United States after Tuskegee, the room contained legislators, lawyers, physicians, philosophers and theologians, and what they built was a combination of three things: a statute that required review, a set of regulations that specified how the review would work, and a short philosophical document that explained what the review was for. Outside the United States, the medical profession had already begun to build its own framework, the Declaration of Helsinki, which would evolve over sixty years into something quite different from where it began. And alongside both, the pharmaceutical industry and its regulators developed a technical standard, Good Clinical Practice, that now governs most drug trials in the world. This chapter describes those three layers, professional, governmental and industrial, and the way they fit together, often awkwardly. Understanding the architecture matters because nearly every practical question in research ethics, from what must go into a consent form to whether a study in another country is acceptable, is answered by reference to one or more of these documents. They do not always give the same answer. The National Research Act and the Belmont Report The National Research Act was signed into law in July 1974. It did two main things. It gave statutory backing to the requirement that institutions receiving federal research funds establish review boards to protect the rights of human subjects. And it created the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, an eleven-member body charged with identifying the basic ethical principles that should underlie research and with recommending how they should be applied. The Commission met from 1974 to 1978 and produced a series of reports on specific topics, including research on the fetus, on prisoners, on children, on people institutionalised as mentally infirm, and on institutional review boards. Its best-known work, however, was a short document that emerged from a period of intensive deliberation beginning with a meeting at the Smithsonian Institution's Belmont Conference Center in Maryland in 1976. The Belmont Report, formally titled Ethical Principles and Guidelines for the Protection of Human Subjects of Research, was published in the Federal Register in April 1979. The Report does three things in fewer than ten pages. First, it distinguishes research from practice. Medical practice consists of interventions designed solely to enhance the wellbeing of an individual patient with a reasonable expectation of success; research is an activity designed to test a hypothesis and contribute to generalisable knowledge. The distinction matters because research, by its nature, serves purposes beyond the individual, and therefore requires a different kind of scrutiny. Many of the scandals of the previous chapter had happened in the fog between the two, where physicians experimented on patients while telling themselves, and the patients, that they were providing care. Second, the Report identifies three basic ethical principles. Respect for persons incorporates two convictions: that individuals should be treated as autonomous agents, and that persons with diminished autonomy are entitled to protection. Beneficence is understood not as kindness but as an obligation: do not harm, and maximise possible benefits while minimising possible harms. Justice concerns the distribution of the burdens and benefits of research, and the Report makes the point with direct reference to the history: the burdens of research had fallen disproportionately on the poor and on ward patients, while the benefits of improved medical care flowed mainly to private patients, and the Tuskegee study had used disadvantaged rural Black men to study a disease that was by no means confined to that population. Third, the Report connects each principle to a practical requirement. Respect for persons is applied through informed consent, which the Report analyses as having three elements: information, comprehension and voluntariness. Beneficence is applied through the systematic assessment of risks and benefits. Justice is applied through the fair selection of subjects, at both the individual and the social level. The genius of the Belmont Report was its economy. Three principles, three applications, a clear distinction between research and practice. It gave review boards a vocabulary for their deliberations and gave the regulations a rationale. Its limitations follow from the same economy. It does not tell anyone what to do when the principles conflict, as they constantly do: when respecting a person's choice to enter a risky trial collides with the duty to protect them, or when a fair distribution of research burdens requires recruiting groups whom other rules treat as vulnerable. It was written with the individual subject in mind and has little to say about communities, about data, or about research that crosses borders. And its account of justice is almost entirely about burdens. It says much less about the injustice of being left out of research altogether, which later became a central concern. The regulations and the Common Rule The Department of Health, Education, and Welfare had issued its first regulations on the protection of human subjects in May 1974, codified at Title 45 of the Code of Federal Regulations, Part 46. Following the Commission's work, the Department of Health and Human Services substantially revised them in 1981, and the Food and Drug Administration issued parallel regulations on informed consent and institutional review boards for research on products it regulates, codified in Title 21, Parts 50 and 56. Additional subparts were added to 45 CFR 46 over the following years to govern research involving pregnant women, fetuses and neonates, prisoners, and children. The core of the regulations, known as Subpart A, was adopted in 1991 by more than a dozen federal departments and agencies as a single shared policy. This became the Federal Policy for the Protection of Human Subjects, universally called the Common Rule. Its reach is defined by funding: it applies to research conducted or supported by the signatory agencies. Research funded privately and not involving FDA-regulated products falls outside it, unless an institution chooses to apply it voluntarily, which many universities and hospitals historically did through their assurances with the federal government. The Common Rule translated the Belmont principles into procedures. It set out the membership requirements for review boards, the criteria a board must find satisfied before approving research, the elements that must be disclosed in informed consent, the circumstances in which consent may be waived or altered, the categories of research that are exempt from its requirements, and the requirement for continuing review of approved research. Chapter 4 describes how these procedures work in practice. The Rule remained substantially unchanged for a quarter of a century. By the 2010s it was widely regarded as out of date: too burdensome for low-risk social science, too permissive about the use of stored specimens and data, and poorly suited to multi-site research, in which the same protocol might be reviewed separately by dozens of boards, each demanding its own changes. After a lengthy consultation, a revised Common Rule was published in January 2017. After two delays, its main provisions took effect in January 2019, and a requirement that most federally funded cooperative research in the United States use a single review board took effect in January 2020. The revisions were significant. They introduced a requirement that consent forms begin with a concise presentation of the key information most likely to help a prospective participant decide, in response to evidence that participants were being overwhelmed by long forms. They created a mechanism for "broad consent" to the storage and future use of identifiable data and specimens. They eliminated routine continuing review for many minimal-risk studies. They expanded the categories of exempt research. And they required the posting of a consent form for each federally funded clinical trial on a public website. The revised rule also changed its language on vulnerability, replacing references to "mentally disabled persons" and removing pregnant women and "handicapped" persons from its list of examples of subjects vulnerable to coercion, in favour of "individuals with impaired decision-making capacity" and "economically or educationally disadvantaged persons." Some of the most ambitious proposals, including a plan to require consent for research on all non-identified biospecimens, were dropped after objections from researchers and institutions. The Declaration of Helsinki While the United States was building a regulatory system, the World Medical Association, an international federation of national medical associations, was writing its own. The Declaration of Helsinki was adopted at the Association's General Assembly in Helsinki in June 1964. It was, in part, the profession's answer to Nuremberg: a statement written by physicians, for physicians, that would govern medical research without importing the rigid legal framing of a war crimes tribunal. The 1964 Declaration differed from the Nuremberg Code in two important ways. It distinguished between clinical research combined with professional care, which it treated more permissively, and non-therapeutic research on healthy volunteers, where it applied stricter standards. And it provided for consent by a legal guardian where the subject lacked capacity, opening the door to research on children and incapacitated adults that the Nuremberg Code had implicitly closed. Critics have noted that the Declaration's first version was, in some respects, less protective than Nuremberg, and more accommodating of the physician's therapeutic judgement. The Declaration has since been revised many times, and the revisions trace the development of the field. The 1975 revision, adopted in Tokyo, introduced the requirement that research protocols be reviewed by an independent committee, bringing the Declaration into line with the emerging American model. The 2000 revision, adopted in Edinburgh, was a major rewrite that abandoned the therapeutic and non-therapeutic distinction as an organising principle and took positions on two issues that proved highly contentious: the use of placebo controls when a proven treatment exists, and the obligation to provide participants with access to beneficial interventions after a trial ends. Both positions were driven by controversies over trials in low-income countries, discussed in the final chapter of this book. Further revisions followed in 2008 and 2013. In October 2024 the Association adopted a further revision at its General Assembly, again in Helsinki, after a consultation of about two and a half years. The 2024 text consistently refers to research "participants" rather than "subjects," a change of vocabulary that signals a change of attitude. It broadens its intended audience beyond physicians to all those involved in medical research. It places new emphasis on meaningful engagement with participants and communities before, during and after research, on scientific integrity, and on fairness in the distribution of the burdens and benefits of research across populations. And it adopts a more contextual account of vulnerability, warning that excluding groups from research can itself entrench health disparities. The Declaration has no legal force of its own, but it is incorporated by reference into the laws and regulations of many countries, and journals commonly require authors to state that their research conformed to it. Global guidelines and Good Clinical Practice Two further frameworks complete the architecture. The Council for International Organizations of Medical Sciences, founded in 1949 under the auspices of the World Health Organization and UNESCO, has issued guidelines on biomedical research involving humans since the early 1980s, with the explicit aim of showing how the principles of the Declaration of Helsinki could be applied in low- and middle-income countries. Its most recent version, the International Ethical Guidelines for Health-related Research Involving Humans, was published in 2016 in collaboration with the World Health Organization. The 2016 guidelines are notable for grounding research ethics in both scientific and social value, for their detailed treatment of research in low-resource settings, and for an approach to vulnerability that looks at the specific characteristics that make a person or group susceptible to harm rather than simply listing categories. The second is Good Clinical Practice. The International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use, which brings together drug regulators and industry from Europe, Japan, the United States and, increasingly, many other regions, adopted its guideline E6 on Good Clinical Practice in 1996. It set a single standard for the design, conduct, monitoring, recording and reporting of clinical trials, so that data generated under it would be acceptable to regulators in all participating regions. Its ethical provisions are brief and largely derivative of the Declaration of Helsinki, but its practical influence is enormous, because it governs the day-to-day operations of the trials on which drug approvals depend. An addendum in 2016 introduced risk-based approaches to monitoring. A thorough revision, E6(R3), was finalised in January 2025. It restructures the guideline around principles of proportionality and quality by design, recognises decentralised trial designs and digital health technologies, and gives more attention to participants' perspectives and to informed consent. The European Medicines Agency applied the new principles and main annex from July 2025. The main features of these frameworks can be compared directly, as Table 1 sets out. The differences in authority and focus explain much of the tension between them: a physician running an industry-sponsored trial in a middle-income country may be bound simultaneously by the country's law, the sponsor's Good Clinical Practice obligations, the regulations of the country where the drug will be marketed, and a journal's requirement to comply with the Declaration of Helsinki. Table 1. The foundational frameworks of human research ethics compared. Framework and issuer Current version Legal status Distinctive contribution Nuremberg Code (US military tribunal) 1947 Part of a judgment, not binding law Absolute requirement of voluntary consent Declaration of Helsinki (World Medical Association) 2024 Professional code, adopted into many laws Independent review; placebo and post-trial rules Belmont Report (US National Commission) 1979 Ethical basis for US regulations Respect for persons, beneficence, justice Common Rule, 45 CFR 46 (US federal agencies) Revised 2018, in force 2019 Binding on federally funded research Procedures for review boards and consent CIOMS Guidelines (CIOMS with WHO) 2016 Guidance Low-resource settings; social value ICH E6 Good Clinical Practice (regulators and industry) E6(R3), 2025 Adopted by regulators for drug trials Operational standard for multinational trials An architecture built for the gate Looking at this architecture as a whole, its centre of gravity is clear. Every framework after Nuremberg places heavy emphasis on the moment before research begins: the protocol is reviewed, the risks weighed, the consent form approved, the participant informed and enrolled. These are the functions that the scandals of the previous chapter most obviously demanded, and they are the functions that the architecture performs best. What happens afterwards receives much less attention. Continuing review exists, but it has always been the weakest part of the system, a paperwork exercise at many institutions and now, for minimal-risk research in the United States, largely abolished. Monitoring of trial conduct is the domain of sponsors and regulators, whose concern is primarily data quality. The relationship between researchers and participants after enrolment, the use of data and samples after a study ends, the fate of participants after a trial closes, the effects of research on communities rather than individuals: these are the areas where the frameworks are thinnest and have been changing most. The 2024 Declaration of Helsinki, the revised Common Rule's provisions on broad consent, and ICH E6(R3)'s attention to participant perspectives are all efforts to strengthen them. None yet amounts to a fully developed account of what researchers owe the people they study across the life of a project. That gap is the subject of the second half of this book. The first task, however, is to look closely at the institution the architecture put at its centre: the committee at the gate. Hashtags: #HumanParticipantResearchEthics #ResearchEthics #HumanSubjectsResearch #InformedConsent #InstitutionalReviewBoards #IRB #NurembergCode #TuskegeeStudy #WillowbrookStudy #BeecherExposé #BelmontReport #CommonRule #DeclarationOfHelsinki #GoodClinicalPractice #ICHGCP #CIOMSGuidelines #ResearchGovernance #ParticipantProtection #VulnerablePopulations #ResearchMisconduct #ClinicalResearchEthics #ConsentModels #ResearchStewardship #GlobalResearchEthics #FutureOfResearchEthics

Download the Book (PDF): Introduction In the late 1980s residents of West Harlem began organizing against the environmental burdens their neighborhood had been asked to carry: a sewage treatment plant on the Hudson waterfront, and a disproportionate share of Manhattan's diesel bus depots. Northern Manhattan's children were being hospitalized for asthma at rates far above the city average. Residents did not need a study to tell them the air was bad; they could smell it and they could see their children's inhalers. What they needed was evidence that would count in rooms where they had no seat. The organization that grew out of that fight, West Harlem Environmental Action, founded in 1988 and now called WE ACT for Environmental Justice, entered into a long partnership with researchers at Columbia University's school of public health. In one early pilot study, published by Patrick Kinney, Peggy Shepard and colleagues in 2000, community members and scientists together measured fine particles and diesel exhaust on Harlem sidewalks and found that concentrations tracked the traffic of trucks and buses street by street. Measurements of that kind became part of the case that pushed New York's transit authority toward cleaner buses and depot reforms. The story is often told as a success of community-based participatory research, and it was. But the most instructive thing about it is not the air monitors. It is the order of events. The community named the problem before any researcher arrived. The community decided that measurement was the tool it needed. The researchers were invited into a fight that was already under way, and the value of their contribution depended on whether they could bring the technical capacity of a university without taking the fight away from the people who had started it. That order of events is the subject of this book. Community-based participatory research, usually shortened to CBPR, is a family of approaches in which people affected by a problem take part as partners, not subjects, in studying it. In the definition that has shaped the field in public health, it is a collaborative approach that equitably involves community members, organizational representatives and researchers in all aspects of the research process, with each partner contributing unique strengths and shared responsibilities. The key words in that sentence are "equitably" and "all aspects." Many projects involve communities. Far fewer involve them equitably, and fewer still involve them in all aspects, from the choice of question to the ownership of what was learned. The argument The argument of this book is that participation is real only to the extent that it redistributes control over decisions that carry consequences. Research is a sequence of such decisions. Someone decides what question is worth asking and what counts as an answer. Someone decides who collects the data and on what terms. Someone decides what the data mean, and whose interpretation prevails when two readings conflict. Someone's name goes on the paper. Someone owns the samples, the data files and any invention that follows. Someone decides what happens to all of it when the grant ends. At each of these points, control can move toward the community or quietly return to the institution. A project is participatory at the points where it moved, and conventional at the points where it did not, whatever its funding application says. This way of seeing the work has a practical advantage. Much of the literature on participatory research describes principles: trust, respect, mutual benefit, colearning. The principles are sound, but principles are cheap to endorse and hard to audit. Decisions, by contrast, leave traces. One can ask who was in the room when the research question was fixed, who signed the data use agreement, who is listed as first author, and who holds the keys to the database. A partnership that cannot answer those questions clearly has not yet decided how power will be shared, and in the absence of a decision the default is the institution, because the institution holds the money, the ethics approval, the publication channels and the legal personality that contracts require. The argument also explains why CBPR is harder than its admirers sometimes admit. Redistributing control has costs. It takes time that grant cycles do not budget for. It gives community partners the ability to say no, including to questions a researcher cares about and to publications a researcher needs for promotion. It exposes academic credit systems, built around individual authorship and priority, to claims they were not designed to handle. And it forces uncomfortable conversations about ownership in a legal environment that generally assumes the institution owns what its employees produce. A partnership that has never felt any of these costs has probably not shared much control. What the book covers and what it leaves out The chapters follow the life of a research project, because the decisions that matter arrive in roughly that order. The first chapter traces where participatory research came from, since its competing traditions still shape what practitioners mean by it. The second examines power directly: the structures, agreements and habits through which partnerships allocate control, and the ways control leaks back to the institution. The next three chapters take the core research tasks in turn: framing the question, collecting data, and interpreting findings. The sixth and seventh chapters take up the two issues where participatory ideals most often collide with institutional rules: authorship and credit, and ownership of data and intellectual property. The eighth asks what a partnership leaves behind when the funding stops, which is where many communities judge whether the whole exercise was worth their time. The book draws its examples mainly from health and environmental research, because that is where CBPR has been most fully developed, evaluated and argued over, and from research with Indigenous peoples, because Indigenous nations have done more than anyone to turn the ethics of research into enforceable rules. The principles transfer to education, urban planning, criminal justice, disability research and development studies, and examples from those fields appear where they sharpen a point. The book does not attempt a manual of specific data collection techniques, and it does not survey the large literature on citizen science in ecology and astronomy, where volunteers typically contribute observations to projects designed by others. That arrangement has great value, but it answers a different question from the one pursued here. A note on words "Community" is an unstable word, and the instability matters. A community can be a neighborhood, a tribal nation with its own government, a group of people sharing a diagnosis, a workforce, or a population defined by a researcher's sampling frame. These are very different partners. A sovereign nation can pass laws governing research on its lands; a neighborhood association cannot. A patient advocacy group may have paid staff and a policy agenda; a loosely connected group of residents may have neither. When this book speaks of the community partner, it means whichever people and organizations have a legitimate claim to represent those most affected by the research, and it treats the question of who holds that claim as part of the work, not a preliminary to it. Similarly, "researcher" here usually means an academic or institutionally employed researcher, and "institution" means the university, hospital, agency or firm that employs them, holds the grant and signs the contracts. The division is a convenience. Many community members are researchers by training, and many academic researchers come from the communities they study. But the division tracks something real: the unequal distribution of the resources that research requires, and the unequal exposure to the harms it can cause. The book is written for researchers who want to do this work well, for community organizations deciding whether and how to partner with a university, for funders and ethics committees trying to judge whether a proposal's claims to participation are genuine, and for students entering fields where community engagement is now expected. Each will find that the questions are the same from different sides of the table. The answers, when they are good, are agreed between those sides and written down. Chapter 1. Two Lineages of Participation Community-based participatory research did not begin as a method. It began as a set of arguments about who is entitled to produce knowledge, and those arguments came from two directions that have never been fully reconciled. One tradition treats participation as a way to make research work better: more relevant questions, better recruitment, more valid measures, findings that are actually used. The other treats participation as a way to shift power: research as a tool with which oppressed people analyze and change their own situation. The first tradition asks how communities can help research. The second asks how research can help communities, and whether research controlled by outsiders can ever do so. Most contemporary CBPR sits somewhere between them, and many of the disputes inside partnerships are really disputes about which lineage the project belongs to. The northern tradition: action research and the usefulness of involvement The first lineage is usually traced to Kurt Lewin, the German-American social psychologist who coined the phrase "action research" in the 1940s. In a 1946 paper in the Journal of Social Issues, "Action Research and Minority Problems," Lewin argued that research which produced nothing but books would not suffice, and proposed a cycle of planning, action and fact-finding about the results of action. His concern was practical. He worked with community relations bodies trying to reduce intergroup prejudice, and he observed that the people charged with acting on a problem learned more, and acted more effectively, when they were involved in studying it. Lewin's framing proved durable because it made participation an instrument of effectiveness. In organizational development, education and later in health services research, action research came to mean practitioners studying their own practice in iterative cycles, often with an outside facilitator. Teachers investigated their own classrooms; nurses studied their own wards. The approach was participatory in that the people closest to the work did the inquiry, but it did not necessarily challenge who held power in the organization. A hospital could run an action research project that improved discharge planning without asking whether patients should have a say in what counted as a good discharge. This lineage has much to recommend it. It produced a large body of practical knowledge about how to run iterative inquiry, how to combine reflection with change, and how to make findings usable. It also produced one of the central claims of modern CBPR: that involving the people affected by a problem can improve the science itself, by surfacing variables outsiders miss, by designing instruments that people actually understand, and by recruiting participants who would not answer a stranger's knock. When funders today justify community engagement, they usually do so in these terms. The southern tradition: knowledge as power The second lineage emerged in Latin America, Africa and South Asia in the 1960s and 1970s, and its founding texts read very differently. Paulo Freire's Pedagogy of the Oppressed, first published in Portuguese in 1968 and in English in 1970, argued that education could either domesticate people or free them, and that liberating education began with learners naming their own world. Freire's literacy circles in northeastern Brazil started from "generative themes" drawn from peasants' own lives, not from textbooks written elsewhere. The method assumed that poor and illiterate people already possessed knowledge about their circumstances, and that the educator's job was to help them analyze it critically, not to deposit correct information into them. The Colombian sociologist Orlando Fals-Borda carried a similar conviction into research. Working with peasant movements on Colombia's Atlantic coast in the 1970s, he developed what came to be called participatory action research, in which the separation between researcher and researched was deliberately broken down and the results were returned to the communities in forms they could use, including popular publications and illustrated histories. Fals-Borda and Muhammad Anisur Rahman later collected accounts of this work from several continents in Action and Knowledge: Breaking the Monopoly with Participatory Action-Research (1991). The subtitle states the thesis. Academic research held a monopoly on legitimate knowledge, and that monopoly was one of the ways in which the powerful stayed powerful. In Tanzania, Budd Hall and colleagues working in adult education in the early 1970s used the term "participatory research" for similar work, and Hall went on to help build international networks around it. In India, Rajesh Tandon founded the Society for Participatory Research in Asia (PRIA) in 1982. Robert Chambers, at the Institute of Development Studies in Sussex, popularized participatory rural appraisal in the 1980s and 1990s, a family of techniques such as community mapping, seasonal calendars and wealth ranking through which villagers could analyze their own conditions. Chambers's book Whose Reality Counts? Putting the First Last (1997) asked the question in its title of development professionals who had been designing programs from capital cities. This tradition does not regard participation primarily as a route to better data. It regards the conventional research relationship, in which an outsider extracts information and takes it away to be analyzed and published elsewhere, as itself a form of domination. The remedy is not to involve communities more skillfully in the outsider's project but to make the project theirs. Other currents Two further currents fed into contemporary CBPR and gave it some of its sharpest tools. Feminist researchers from the 1970s onward challenged the claim that good research required detachment, argued that the researcher's position shaped what she could see, and developed methods such as collaborative interviewing that treated participants as knowers. Their emphasis on reflexivity, the discipline of examining how one's own identity and interests shape the research, is now standard in participatory work. Indigenous scholars made the most direct challenge. Linda Tuhiwai Smith's Decolonizing Methodologies: Research and Indigenous Peoples, first published in 1999, opened with the observation that "research" is probably one of the dirtiest words in the Indigenous world's vocabulary. Her book documented how research had served colonial projects: classifying, measuring and displaying Indigenous peoples, removing their remains and artifacts, and producing knowledge that justified dispossession. Smith, a Māori scholar, set out an agenda in which Indigenous communities would define research priorities themselves, grounded in their own values and protocols. In Aotearoa New Zealand this became Kaupapa Māori research; in Canada and the United States, tribal nations and First Nations organizations developed their own research codes and review boards. Indigenous research ethics now supplies much of the most concrete guidance available on the questions this book treats as central: ownership, control and benefit. Disability rights activism contributed the slogan that many participatory researchers now use, "Nothing about us without us," which James Charlton took as the title of his 1998 book on disability oppression. The slogan compresses the core claim of the power-sharing tradition into five words. The formation of CBPR in public health The term "community-based participatory research" took hold in North American public health in the 1990s. A key text was a 1998 review in the Annual Review of Public Health by Barbara Israel, Amy Schulz, Edith Parker and Adam Becker, "Review of Community-Based Research: Assessing Partnership Approaches to Improve Public Health." It set out a list of principles that has been quoted ever since: CBPR recognizes community as a unit of identity; builds on strengths and resources within the community; facilitates collaborative, equitable partnership in all phases of research; promotes colearning and capacity building among all partners; integrates knowledge and action for mutual benefit; addresses health from positive and ecological perspectives; disseminates findings and knowledge gained to all partners; and involves a long-term commitment by all partners. Later versions added attention to cyclical and iterative processes and to the social inequalities that shape health. Israel and colleagues were writing from experience. The Detroit Community-Academic Urban Research Center, established in 1995 with funding from the US Centers for Disease Control and Prevention, brought together the University of Michigan schools of public health, community organizations on Detroit's east side and the city's health department. Its board, which brought representatives of community-based organizations together with the university, the health department and a health system, reviewed research proposals, and the partners adopted written CBPR principles to govern their joint work. The center became one of the most studied examples of long-term CBPR infrastructure, and the principles it developed influenced many later partnerships. The field grew quickly. Meredith Minkler and Nina Wallerstein edited Community-Based Participatory Research for Health, first published in 2003 and now in its third edition (2018, with Bonnie Duran and John Oetzel as co-editors), which became the standard reference. Community-Campus Partnerships for Health, a network founded in 1996, promoted partnership principles across universities and community organizations. In 2004 the US Agency for Healthcare Research and Quality commissioned a systematic review, led by Meera Viswanathan, of the evidence on CBPR, which concluded that the approach was promising but that the evidence base was thin and inconsistently reported. The journal Progress in Community Health Partnerships was launched in 2007 to give partnerships a venue for publishing their work. By the 2010s US federal funders, including the National Institutes of Health through its Clinical and Translational Science Awards and the Patient-Centered Outcomes Research Institute, created by the 2010 Affordable Care Act, were requiring or rewarding community and patient engagement. Parallel streams: patient involvement and participatory design The same questions surfaced in fields that rarely cited Freire or Fals-Borda. In the United Kingdom, patient and public involvement in health research became an organized expectation from the mid-1990s. The Department of Health established a body in 1996 that later became INVOLVE, and the National Institute for Health Research, created in 2006, built involvement into its funding requirements, so that grant applications must explain how patients and the public shaped the proposal and will shape the study. The language is different from CBPR; the unit is usually a patient group or a panel of lay advisers rather than a geographically bounded community, and involvement often stops short of shared authority. But the underlying logic, that people with lived experience of a condition know things about it that researchers do not, is the same. In Scandinavia, participatory design emerged in the 1970s from collaborations between computer scientists and trade unions. Projects such as UTOPIA, a Swedish and Danish collaboration with graphic workers in the early 1980s, set out to design workplace technologies with the workers who would use them, on the explicit premise that technology choices were also choices about power in the workplace. The tradition survives in human-computer interaction and in the design of public services, and it has contributed practical techniques for involving non-specialists in technical decisions: prototypes people can handle, scenarios they can argue about, and workshops in which the specialist's role is to make options visible rather than to choose among them. These streams matter here for two reasons. First, they show that the questions CBPR raises are not peculiar to public health or to marginalized neighborhoods; they arise whenever the people who produce knowledge and the people who live with its consequences are different people. Second, they illustrate how the same vocabulary can cover very different allocations of power. A patient panel that comments on a lay summary and a patient panel that holds a veto over the study design are both described as "involvement." Only the second has shifted control. What the evidence shows Because participatory research is often defended by its effects, it is fair to ask what those effects are. The honest answer is that the evidence is substantial but uneven, and that it supports some claims more strongly than others. The 2004 review for the Agency for Healthcare Research and Quality found that community involvement was associated with better recruitment and retention and with interventions better matched to local conditions, but that few studies were designed to test whether participation itself caused these results. Margaret Cargo and Shawna Mercer, reviewing the field in the Annual Review of Public Health in 2008, argued that the value of participatory research lay along three dimensions: translating research into practice, advancing self-determination, and pursuing justice. They noted that the second and third were rarely measured at all. A more useful approach to the evidence came from a realist review led by Justin Jagosh and published in the Milbank Quarterly in 2012. Instead of asking whether participatory research "works," the team asked what mechanisms produce its benefits and under what conditions. Examining a set of long-running partnerships, they identified partnership synergy, the combination of perspectives, skills and resources that no partner could achieve alone, as the central mechanism, and they found that its effects were cumulative: early trust made later co-governance possible, which produced culturally appropriate interventions, which in turn deepened trust. They described "ripple effects" that extended beyond the original project, including new partnerships, community capacity and systemic changes. A follow-up realist evaluation published in BMC Public Health in 2015 elaborated how conflict, when handled well, could strengthen rather than break these partnerships. The Engage for Equity study, led by Nina Wallerstein and John Oetzel with colleagues at the University of New Mexico and elsewhere, took the quantitative route. Surveying academic and community partners in roughly two hundred federally funded community-engaged projects in the United States, the team examined associations between partnership practices, such as shared decision-making and community influence over resources, and outcomes ranging from the quality of the research to changes in policy and community capacity. Their findings, reported in a series of papers including a 2020 special section of Health Education & Behavior, supported the proposition that the practices through which power is shared are associated with better outcomes, not just with partner satisfaction. None of this proves that every participatory project produces better science. Partnerships can fail, and a study can be highly participatory and badly designed. But the weight of the evidence runs in a consistent direction: the benefits of participation come less from the presence of community members than from what they are able to decide. That finding is the empirical counterpart of the argument this book makes on ethical grounds. Why the lineages still matter Institutional adoption brought CBPR money, legitimacy and training programs. It also brought a drift toward the northern lineage. When participation is justified by what it does for research quality and recruitment, it becomes easy to measure success by research outcomes and to treat the community as a stakeholder to be consulted rather than a partner with authority. Nina Wallerstein and Bonnie Duran, writing in the American Journal of Public Health in 2010, described CBPR as sitting on a continuum from a utilitarian pole, concerned with making research more effective, to an emancipatory pole, concerned with social transformation, and argued that the field's contribution to health equity depended on not losing the second pole. The two lineages give different answers to almost every concrete question a partnership faces. Who should decide the research question? The utilitarian answer is that researchers should, with community input to make it relevant; the emancipatory answer is that the community should, with researchers helping to make it answerable. Who owns the data? The utilitarian answer defers to institutional policy; the emancipatory answer treats the community's claim as primary. What counts as success? A publication and a funded follow-up grant, or a change in the conditions people live under? Andrea Cornwall and Rachel Jewkes, in a much-cited 1995 paper in Social Science & Medicine, "What Is Participatory Research?", made the point that the key difference between participatory and conventional research lies not in methods but in the location of power in the research process. A focus group is not participatory because it is a focus group; a survey is not extractive because it is a survey. What matters is who decided to use it, who designed it, who holds the results and who decides what they mean. This is the thread the rest of this book follows. A warning came from the same tradition. In 2001 Bill Cooke and Uma Kothari edited a collection provocatively titled Participation: The New Tyranny?, arguing that participatory methods in development had become a ritual that legitimized decisions already made, gave outsiders access to local knowledge without transferring control, and could override local power dynamics in the name of "the community." The critique was not that participation was bad but that the word had become detached from any redistribution of power. It is still the best test to apply to any project that calls itself participatory: what, specifically, can the community decide that it could not decide before? Most partnerships will never be purely emancipatory. They operate inside universities and funding systems that impose deadlines, require principal investigators, demand ethics approval and reward publication. The realistic goal is not purity but honesty: knowing which decisions are genuinely shared, which are delegated to the community, and which remain with the institution, and saying so openly. The next chapter examines how partnerships make those allocations, and how they go wrong. Chapter 2. Power Is the Method Every research partnership has a governance structure, whether or not anyone designed it. If no one decides how decisions will be made, they will be made by whoever controls the budget, signs the ethics application and answers to the funder. In most projects that is the academic principal investigator. This is not usually a matter of bad faith. It is a matter of defaults: universities are built to route authority through principal investigators, and the path of least resistance runs straight to them. Sharing power therefore requires deliberate design, and the design has to be specific enough to survive the moment when partners disagree. The ladder and its limits The most widely cited account of participation remains Sherry Arnstein's "A Ladder of Citizen Participation," published in the Journal of the American Institute of Planners in 1969. Arnstein, who had worked on federal urban renewal and anti-poverty programs, described eight rungs grouped into three bands. At the bottom were manipulation and therapy, which she called nonparticipation: programs designed to educate or "cure" participants rather than let them influence anything. In the middle were informing, consultation and placation, which she called degrees of tokenism: citizens could hear and be heard but had no assurance their views would be acted on. At the top were partnership, delegated power and citizen control, the degrees of citizen power. Her central point was blunt. Participation without redistribution of power is an empty and frustrating process for the powerless, because it allows those in power to claim that all sides were considered while ensuring that only some benefit. Arnstein's ladder has been criticized as too linear. Real partnerships do not sit on a single rung; a community may control recruitment while the university controls analysis, or hold a veto over publication while having little say in the budget. The International Association for Public Participation's spectrum, which runs from inform through consult, involve and collaborate to empower, is a less judgmental version often used in public agencies. Other writers have argued that the top of the ladder is not always the right goal: some communities do not want to run a research project, only to ensure it is done on their terms. The value of the ladder is not that it tells a partnership where to stand, but that it forces the question of where it actually stands on each decision, as opposed to where it says it stands. A practical way to use it is to break a project into its main decisions and ask of each who proposed, who could object, and who had the final word. The pattern that emerges is usually more informative than any overall characterization. Table 1 sets out how several common forms of community involvement tend to allocate a few of the decisions that matter most. Table 1. How common forms of community involvement allocate key decisions. Form of involvement Research question Budget Data custody Publication Community advisory board Researcher sets; board comments Researcher controls Institution Researcher decides Community-engaged study Researcher sets with input Small subcontract to partner Institution Partner reviews drafts Equitable CBPR partnership Jointly negotiated Shared, written allocation Joint agreement Joint approval and authorship Community-led research Community sets Community holds grant Community Community decides The table simplifies, but it makes one pattern plain. The forms differ less in how often community members attend meetings than in who holds money, data and the right to decide what gets published. A community advisory board that meets monthly and is warmly thanked in the acknowledgments may exercise less power than a partner organization that meets rarely but holds a third of the budget and a veto over data release. Structures that share control Partnerships have developed a set of tools for making power-sharing concrete. None is sufficient alone, and each can be hollowed out, but together they turn principles into obligations. The first is a written partnership agreement, sometimes called a memorandum of understanding or a research agreement. At its best it states the project's goals, the roles and responsibilities of each partner, how decisions will be made and disputes resolved, how money will be allocated, who will own and have access to data, how findings will be reviewed before publication, how authorship will be decided, and what happens if a partner withdraws. Many partnerships derive theirs from published models; the Detroit Community-Academic Urban Research Center's principles and the research agreement templates developed by Indigenous health organizations are frequently adapted. The value of such an agreement lies less in its legal force, which is often limited, than in the conversations required to write it. A partnership that has argued about data ownership before any data exist has a much better chance of resolving the argument when the data arrive. The second is a governance body with real authority. The difference between an advisory board and a steering committee is whether its decisions bind. Some partnerships give a community-majority steering committee formal approval over research questions, instruments and publications; others require consensus of all partner organizations on major decisions. Voting rules matter. A committee where the university holds as many seats as all community organizations combined, and the principal investigator chairs, will reproduce institutional control even if every vote is unanimous, because the agenda and the information flow run through the chair. The third is money. Budgets are statements of priorities, and in CBPR they are also statements of power. The principle is simple: community partners who do research work should be paid for it, and community organizations should receive a share of the grant sufficient to cover their real costs, including staff time, overhead and the unglamorous work of convening residents. The practice is harder. Universities often apply their full indirect cost rate to the entire grant while passing through only direct costs to subcontracted community partners; payment systems can take months to process invoices, which a small nonprofit cannot absorb; and some funders restrict what can be paid to people without formal credentials. Partnerships that take equity seriously negotiate these details before submitting the grant, and some have persuaded funders to name a community organization as co-principal investigator or as the lead applicant. The fourth is time and process: meeting at times and places community partners can attend, providing food and childcare, translating materials, and allowing enough time for community partners to consult the people they represent before a decision is made. These are sometimes treated as courtesies. They are in fact conditions for shared decision-making. A decision taken at a weekday afternoon meeting on campus, on the basis of a document circulated the night before in academic English, has not been shared regardless of who was invited. Positionality and the invisible allocation of authority Formal structures do not capture everything. Power also moves through expertise, language and social position. When a researcher explains why a proposed survey question would compromise validity, community partners may defer even when their objection was sound, because the vocabulary of methodology carries authority. When a community partner describes what residents will and will not tolerate, researchers may treat it as anecdote rather than evidence. The literature calls attention to this through the concept of positionality: the recognition that each partner's race, class, gender, education, institutional role and relationship to the community shape what they see and how they are heard. Michael Muhammad and colleagues, in a 2015 article in Critical Sociology, drew on interviews with academic and community partners to describe how researchers' identities, including their institutional privilege and, for researchers of color, their complicated position as both insiders and outsiders, affected trust, decision-making and outcomes. Their recommendation was not that researchers should confess their privilege and move on, but that partnerships should build regular, structured reflection into their work so that these dynamics could be named and addressed. A practical form of this is the power analysis, conducted periodically, in which partners ask of recent decisions who raised the issue, whose information was decisive, whose preferences prevailed, and whether anyone felt unable to object. Such an exercise is uncomfortable, and it works best when a trusted facilitator not employed by the university runs it. It is also one of the few ways to detect the gradual drift of authority back toward the institution, which rarely happens through any single decision. Who speaks for the community? Power-sharing presupposes a partner with whom to share it, and identifying that partner is itself an exercise of power. Researchers often begin with the organizations they already know, or those with the capacity to manage a subcontract: established nonprofits, health clinics, churches with paid staff. These organizations may be deeply rooted, or they may represent the more organized and resourced part of a community while speaking for the rest. A partnership built entirely with service providers may miss the people those providers serve; a partnership with elected tribal leadership may not capture the views of urban tribal members or of dissenting factions. There is no formula for resolving this, but there are better and worse practices. Better partnerships involve several organizations with different constituencies, create channels for residents who are not affiliated with any organization, and revisit the question as the work evolves. They recognize that communities contain their own inequalities, of gender, age, class, caste, immigration status and disability, and that the most affected people may be those with least voice even within their own community. Cooke and Kothari's warning, that participation can reinforce local power structures by treating the loudest voices as the community's voice, applies with full force here. For sovereign Indigenous nations the question has a clearer answer: the nation's government, through whatever processes it has established, has authority to approve or refuse research involving its citizens and lands. Many tribal nations in the United States have established research review boards or research codes; the Navajo Nation's Human Research Review Board, created in the 1990s, is among the best known. In Canada, the Tri-Council Policy Statement on ethical conduct for research, the joint policy of the federal research agencies, devotes a chapter to research involving First Nations, Inuit and Métis peoples, requiring engagement with the relevant community where research is likely to affect its welfare. These arrangements do not eliminate internal debate, but they locate authority where a partnership can find it. A worked example: Kahnawake The Kahnawake Schools Diabetes Prevention Project, begun in 1994 in the Mohawk community of Kahnawake near Montreal, is one of the most thoroughly documented examples of how governance can be designed to keep authority with a community over decades. The project was initiated in response to high rates of type 2 diabetes in the community and was designed as a school- and community-based prevention program with an evaluation component led jointly by community and academic researchers. Several features of its design repay attention. First, the project adopted a written code of research ethics, first in the mid-1990s and revised later, which set out the obligations of community and academic researchers, the principles of the partnership and the procedures for approving publications and handling data. Second, it established a community advisory board, made up of community members, which was not a sounding board for researchers' plans but a body with authority over the project's direction, including review of research proposals and of manuscripts. Third, the research team included community researchers employed by the project alongside academic investigators, so that expertise in both research methods and community life was held inside the team rather than traded across its boundary. Fourth, the project was structured from the outset as long-term: the intervention and its evaluation continued for years, and the partnership outlived several funding cycles. Researchers who worked with the project, including Ann Macaulay and Margaret Cargo, have written about both its strengths and its tensions: the time required for community review, the occasional disagreements between the advisory board and academic researchers over what to measure and publish, and the effort required to keep a partnership vital as individuals moved on. What made the arrangement durable was not the absence of tension but the fact that the procedures for resolving it had been agreed in advance and were understood to bind everyone, including the academics whose careers depended on publication. The lesson generalizes. When the community's authority is written into the project's founding documents and exercised routinely, it becomes part of how the work is done rather than an exceptional intervention. When it exists only as goodwill, it tends to be exercised only when nothing important is at stake. The hidden costs of sharing It is worth being candid about what power-sharing costs, because partnerships that pretend it is free tend to abandon it when the costs arrive. For researchers, the costs are mainly time and control. Joint decision-making is slower. Community review of instruments and manuscripts adds months. A community partner's objection can remove a variable a researcher wanted, delay a paper past a promotion deadline, or end a line of inquiry altogether. Researchers also carry reputational risk inside their institutions, where colleagues may regard participatory work as less rigorous, and where tenure committees may not know how to credit a jointly authored report written for a city council. For community partners the costs are different and often larger. Participation consumes the time of people who are already stretched: staff of small organizations with no slack, residents with jobs and caring responsibilities. Community partners carry the relational risk of the project, because it is their credibility with neighbors that is spent if the research disappoints or harms. They are frequently asked to educate researchers about their community, a form of labor that is seldom paid and rarely acknowledged. And when a project ends, the researchers move to the next grant while community partners remain to answer for what happened. Recognizing these asymmetries is part of sharing power. A partnership that compensates the time of community partners, that protects their credibility by honoring agreements, and that accepts slower timelines as the price of legitimacy has done more to share power than one with an elaborate governance chart and no budget line for community staff. When partners disagree Genuine power-sharing means that the community can say no, and eventually it will. A community partner may object to a question that researchers consider scientifically essential, refuse to allow a finding to be published, or conclude that the partnership is no longer worth its time. How a partnership handles these moments reveals what it actually is. The agreement should anticipate them. Common provisions include a period of discussion before any partner can withdraw, a commitment to seek mediation from a trusted third party, and rules about what happens to data and publications if the partnership dissolves. Some agreements distinguish between the community's right to review and comment on all publications and its right to veto publications that would identify or harm the community; others give a broader veto. Researchers sometimes worry that such provisions threaten academic freedom. The counterargument is that academic freedom protects researchers from interference by their employers and the state; it does not entitle them to publish data that others contributed on condition of shared control. Where the conflict is real, the honest course is to negotiate the terms before the research begins, when both parties can walk away without loss. Disagreement is not failure. Jagosh and colleagues found that partnerships which worked through conflict openly often emerged stronger, because the experience demonstrated that community voices had real weight. What damages partnerships is not disagreement but the discovery that the community's agreement was never actually required. Chapter 3. Whose Question Is It? Of all the decisions in a research project, the choice of question is the one that most constrains every other. It determines what data will be collected and what will be ignored, which people will be counted and which variables will explain them, what kind of answer is possible and to whom it will be useful. A community brought in after the question has been fixed can improve the recruitment materials, suggest better wording for a survey and help interpret the results, but it cannot change what the study is about. For that reason the question is where participatory research most often fails without anyone noticing, because the failure takes the form of an absence: the question the community would have asked and never got to. How questions are usually made In conventional research, questions come from three sources: the researcher's discipline, which defines what is interesting and publishable; the funder, which defines what will be paid for; and the researcher's own career, which rewards questions that extend a line of work. None of these sources is illegitimate. Disciplines accumulate knowledge by building on previous findings; funders have mandates; researchers need to specialize. But none of them runs through the people whose lives the research is about. The consequences are visible in what gets studied. Funding in health research is organized largely by disease and organ system, while people experience their health through housing, work, income, neighborhood safety and the behavior of institutions. A community asked to partner on a diabetes study may care more about the absence of a grocery store, the cost of insulin, or the fact that the clinic closes before people get home from work. Those are researchable questions, but they fall between funding streams, and a researcher with a diabetes grant has limited room to pursue them. Framing also matters within a topic. Research on marginalized communities has a long tradition of what critics call deficit framing: asking what is wrong with the community, its behaviors, its knowledge, its compliance, rather than what is wrong with the conditions and institutions it faces. A study asking why residents fail to attend appointments will look for explanations in residents; a study asking what the clinic does that makes attendance difficult will look in the clinic. The data may overlap, but the findings, and the interventions they justify, will differ. Israel and colleagues' principle that CBPR builds on strengths and resources within the community is partly a corrective to this habit. It asks researchers to begin from what a community already does to protect its health, and from the conditions that undermine those efforts. Communities that arrive with questions Some of the most consequential participatory research began when a community already had a question and could not get anyone to take it seriously. The Flint water crisis is the best-known recent case. After the city of Flint, Michigan, switched its water source to the Flint River in April 2014 under state-appointed emergency management, residents complained of discolored, foul-smelling water, rashes and hair loss. Officials repeatedly assured them the water was safe. One resident, LeeAnne Walters, whose household water tested very high for lead, contacted Marc Edwards, a civil engineer at Virginia Tech who had previously exposed lead contamination in Washington, DC. In 2015 the Virginia Tech team and Flint residents organized a citywide sampling effort in which residents collected water samples from their own homes using kits the researchers supplied, and the results showed lead levels well above federal action levels in many homes. At about the same time the pediatrician Mona Hanna-Attisha and colleagues analyzed children's blood lead records and found that the proportion of children with elevated blood lead levels had increased after the water switch, most sharply in the neighborhoods with the highest water lead. The two lines of evidence together forced officials to acknowledge the problem. Flint is not a textbook CBPR partnership; the collaboration was assembled in an emergency and some residents later criticized how credit and attention were distributed. But it shows with unusual clarity what happens when research follows a community's question. The residents already knew something was wrong. Official monitoring had been designed, and in some respects conducted, in ways that failed to detect it. What the researchers contributed was not the question but the capacity to answer it in a form that institutions could not dismiss. The environmental justice movement has produced many such cases, because environmental hazards are often first detected by the people who live beside them. In California's San Joaquin Valley, residents of small rural communities, many of them Latino farmworker households, had long worried about nitrate and arsenic in their drinking water. Working with the Community Water Center, a local advocacy organization, researchers including Carolina Balazs and Rachel Morello-Frosch at the University of California, Berkeley, examined the distribution of contamination across community water systems and found that systems serving higher proportions of Latino residents and renters tended to have higher nitrate levels. Balazs and Morello-Frosch later used the experience to argue, in a 2013 paper in the journal Environmental Justice, that community participation strengthens what they called the "three Rs" of science: its rigor, relevance and reach. Community partners, they found, directed attention to the right questions: not only whether water exceeded regulatory limits, but who was exposed, who paid for the costs of treatment and replacement water, and which small systems lacked the capacity to fix their problems. Those questions made the research both more accurate about the real burden and more useful for policy. Methods for finding the question together When a partnership begins without a pre-formed question, it needs ways to generate one jointly. Several approaches have become established. Community assessment brings together existing data, from health departments, census records and service providers, with residents' own accounts, gathered through listening sessions, community forums or door-to-door conversations. The important feature is that residents help interpret the data rather than simply supplying stories to illustrate it. A map of asthma hospitalizations means one thing to an epidemiologist and something richer to residents who know which blocks have mold-infested housing, which schools sit next to truck routes and which landlords ignore complaints. Photovoice, developed by Caroline Wang and Mary Ann Burris and described in a 1997 article in Health Education & Behavior, gives participants cameras to document their own community's strengths and concerns, followed by group discussion of the photographs and, typically, an exhibition for policymakers. Wang and Burris drew explicitly on Freire's idea of critical consciousness and on feminist theory. Photovoice has been used thousands of times since, sometimes as a data collection method within a study defined elsewhere, but its original purpose was agenda-setting: letting people who are rarely asked show what they think matters. Priority-setting partnerships formalize the process at a larger scale. The James Lind Alliance, established in the United Kingdom in 2004 and now part of the National Institute for Health and Care Research, brings together patients, carers and clinicians to identify and rank the most important unanswered questions about a particular condition. Its method moves from an open survey of uncertainties, through checking which are genuinely unanswered in the existing literature, to a facilitated workshop at which participants agree a top ten. The resulting lists have repeatedly diverged from the questions researchers had been funding, with patients and carers placing more weight on quality of life, side effects and practical management than on new drugs. Funders, including the National Institute for Health and Care Research itself, have used these lists to commission studies. Concept mapping and deliberative workshops offer other structured ways to reach agreement. What these methods share is a sequence: open generation of concerns by those affected, joint sorting and prioritization, and a deliberate step in which researchers help translate priorities into questions that can be answered with available methods and resources, while the community checks that the translation has not changed the meaning. Translation without capture That last step is where control is most easily lost. Residents may name a concern such as "our kids can't breathe here," which must be turned into a research question: what exposures, measured how, compared with what, over what period, linked to what outcomes? Every one of these choices involves technical judgment, and the person making it holds power. If researchers make the choices alone, the resulting question may be answerable but no longer the community's. Good practice keeps the translation visible. Researchers lay out options and their consequences in plain terms: measuring particulate matter at fixed monitors is cheaper and comparable with official data, while personal monitors carried by children capture actual exposure but are costlier and burdensome; comparing hospitalization rates across neighborhoods is fast but will not show what happens inside homes. The community chooses among options with an understanding of the trade-offs, and it can insist on the more expensive or slower design if that is the one that answers its question. Where no feasible design can answer the community's question, researchers say so, and the partnership decides whether a narrower study is still worth doing. Translation also involves outcomes. What would count as an answer, and what would be done with it? Communities often want research that can be used in a specific venue, a zoning hearing, a school board decision or a funding application, and the design must produce evidence in a form that venue will accept. This is not a corruption of science. It is the same consideration that leads a pharmaceutical trial to use endpoints a regulator will recognize. Questions that harm Communities sometimes resist questions not because they lack interest in the topic but because they have learned what such questions can do. Research on stigmatized conditions, such as alcohol use, mental illness, HIV or violence, produces findings that attach to the whole group studied, including people who never took part. A community-level finding travels further than any individual's data, and it can be used by outsiders to justify prejudice, disinvestment or intervention. The Barrow Alcohol Study is a standard warning. In 1979 researchers from the University of Pennsylvania, working under contract to the North Slope Borough in Alaska, surveyed residents of the Iñupiat town of Barrow (now Utqiaġvik) about drinking. The results were released at a press conference in 1980 before the community had reviewed them, and national newspapers reported that alcoholism was threatening the survival of the Iñupiat people. The borough's bond rating reportedly suffered, residents felt publicly humiliated, and the episode became a case study in research ethics courses. Edward Foulks, one of the investigators, later wrote an account of what went wrong, describing a partnership in which the community had commissioned the research but had no say over how its findings were framed and released. The data might have been accurate; the harm lay in a framing that described the community through its pathology and in a release process that bypassed the people described. The lesson for question-setting is that communities have a legitimate interest in how they will be represented, and that this interest is engaged at the start, not only at publication. A community may agree to study alcohol use if the question is framed around the availability of treatment, the effect of alcohol outlet density or the success of community-led sobriety movements, and refuse if it is framed as the prevalence of drinking. Such preferences can look to researchers like spin. Often they are an accurate recognition that the choice of what to measure determines what story will be told, and that the community, not the researcher, will live with the story. Group harm is poorly handled by conventional research ethics, which focus on the risks to individual participants. An individual can consent to answer questions about her drinking; she cannot consent on behalf of her town to be described as a town of drinkers. Participatory research offers one of the few mechanisms through which a group can weigh these risks collectively, and it does so best at the moment the question is chosen. Questions researchers would rather not ask The reverse problem also arises. Communities sometimes want questions that researchers, or their institutions, would rather avoid. Residents near a university may want to study the university's own effects on housing costs and displacement. Patients may want to examine how a hospital treats them. Communities policed heavily may want research on police conduct rather than on crime. Workers may want to study their employer's safety practices, and the employer may be a research funder. These questions test whether a partnership's commitment to community priorities holds when the priorities become inconvenient. The institution may have legitimate concerns about conflicts of interest, and researchers may fear retaliation from colleagues or funders. But a partnership that systematically steers away from questions implicating the institutions involved has a structural bias worth naming. One sign of a mature partnership is that it has at least once pursued a question that made its academic partner uncomfortable, and survived. When the funder has already chosen A frequent difficulty is that the question is partly fixed before any partnership exists. Funding announcements define topics, and applications have deadlines that leave little time for community deliberation. Researchers then approach community organizations with a grant already half-written and ask them to sign a letter of support. There are honest ways to handle this. The first is to build partnerships before funding opportunities arise, so that when an announcement appears, the partnership already has a list of priorities and can decide together whether this opportunity matches any of them. Long-standing partnerships such as the Detroit center operated in exactly this way, reviewing proposed projects against the partnership's agreed priorities. The second is to be explicit about what is fixed and what is open: the topic is set by the funder, but within it the community will choose the specific question, the population and the outcomes. The third is to decline. A community organization that turns down an opportunity because it does not serve its priorities is exercising exactly the power participatory research claims to respect, and researchers who treat such refusals as obstacles have misunderstood the enterprise. Some funders have changed their practices in response. Planning grants that pay for partnership development before a full proposal, application processes that require evidence of community involvement in defining the question, and review panels that include community members all shift the point at which communities enter. The Patient-Centered Outcomes Research Institute in the United States, for example, has required applicants to describe how patients and other stakeholders were involved in developing the research question, and includes patients and stakeholders as reviewers alongside scientists. Such measures do not guarantee genuine partnership, but they make it harder to recruit a community after the fact. The question is the first allocation of power in a research project. Once it is made, the choice of data follows, and with it a new set of decisions about who collects that data, on what terms, and at whose risk. That is the subject of the next chapter. Hashtags: #CommunityBasedParticipatoryResearch #CBPR #ParticipatoryResearch #CommunityEngagement #CollaborativeResearch #ParticipatoryActionResearch #ActionResearch #CommunityAcademicPartnerships #PowerSharing #SharedDecisionMaking #CommunityGovernance #ResearchEquity #SocialImpact #CommunityEmpowerment #DecolonizingMethodologies #IndigenousResearch #Photovoice #CommunityAssessment #PrioritySetting #ResearchPartnerships #DataOwnership #CommunityAuthorship #CoLearning #HealthEquity #FutureOfParticipatoryResearch

Download the Book (PDF): Introduction A hospital wants to know whether a new discharge programme reduces readmissions. A retailer wants to know whether a discount brings back lapsed customers or merely subsidises people who would have returned anyway. A ministry of labour wants to know which unemployed workers benefit from a training course and which would do as well without it. Each of these organisations already owns a great deal of data and, increasingly, a team that can fit a gradient-boosted model or a neural network to that data in an afternoon. The models will predict readmission, repurchase and re-employment with impressive accuracy. None of them, on their own, will answer the question that was asked. The gap between those two things, between predicting an outcome and estimating what an intervention does to it, is the subject of this book. It is an old gap. Statisticians and econometricians spent most of the twentieth century building a vocabulary for it: potential outcomes, confounding, identification, instruments, the propensity score. Machine learning grew up largely on the other side of the gap, with a different vocabulary and different habits: held-out test sets, loss functions, regularisation, the relentless pursuit of lower prediction error. For a long time the two traditions regarded each other with a mixture of interest and suspicion. Econometricians saw black boxes that could not produce a standard error. Machine learners saw economists fitting linear regressions with a dozen hand-picked controls to data sets that could support a thousand. Over roughly the past fifteen years, a body of work has emerged that takes both sides seriously. It is usually called causal machine learning, and its central achievement is surprisingly specific. It does not make machine learning causal. What it does is show precisely where flexible predictive models can be inserted into a causal analysis, what goes wrong when they are inserted carelessly, and how to repair the damage so that the final estimate comes with a valid confidence interval. Three families of methods carry most of the weight: double or debiased machine learning, developed chiefly by Victor Chernozhukov and colleagues in econometrics; targeted maximum likelihood estimation, developed by Mark van der Laan and colleagues in biostatistics; and generalized random forests, developed by Susan Athey, Julie Tibshirani and Stefan Wager at the meeting point of the two. They grew up in different departments, publish in different journals and use different notation. They are much closer to one another than their literatures suggest. The argument of this book The controlling claim of what follows is that machine learning earns its place in causal inference only as a servant of a well-defined estimand and a defensible identification strategy, and that the methods which make this partnership work all rest on one idea: building the estimator around a score that is insensitive to small errors in the predictive models it depends on. Get the estimand and the identifying assumptions right, and flexible learners let you drop the fragile functional-form assumptions that have always haunted applied work. Get them wrong, and no amount of predictive accuracy will rescue the answer; if anything, a sophisticated model will make a wrong answer look more authoritative. This claim cuts in two directions, and the book pursues both. Against the enthusiast who believes that a sufficiently powerful model will discover causal structure in observational data by itself, it insists that causal conclusions are purchased with assumptions, and that those assumptions come from knowledge of how the data were generated, not from the data alone. Against the sceptic who regards machine learning as a fashionable distraction, it shows that the classical workhorse of observational research, a linear regression with controls, silently imposes assumptions that are frequently false, and that the new methods relax those assumptions in a principled, well-understood way, with inference that is as rigorous as anything in the textbook canon. The idea that unifies the three families deserves a sentence of preview here, because it recurs in almost every chapter. Every estimator in this book needs to learn some auxiliary functions from the data: how the outcome depends on covariates, how the probability of treatment depends on covariates, or both. These are called nuisance functions because they are not what we care about; they are tools for getting at the treatment effect. Machine learning is extremely good at estimating nuisance functions and extremely bad at estimating them without bias, because regularisation, the very device that makes it predict well, deliberately shrinks estimates away from the truth. The trick, which goes back to Jerzy Neyman in the 1950s and to the semiparametric statistics of the 1980s and 1990s, is to construct the estimate of the treatment effect so that small errors in the nuisance functions have no first-order effect on it. Double machine learning calls this Neyman orthogonality. Targeted learning calls it solving the efficient influence curve equation. Generalized random forests build it in by centring the data before growing trees. The language differs; the mathematics is essentially the same. What the book covers and what it leaves out The first chapter sets out the causal framework the rest of the book relies on: potential outcomes, the estimands most analyses target, and the assumptions that allow those estimands to be recovered from data. It is deliberately insistent on the point that identification comes before estimation. The second chapter explains what goes wrong when one simply plugs a machine learning model into a causal formula, working through the regularisation bias and overfitting bias that make naive approaches unreliable. The third develops the remedy: orthogonal scores, influence functions and double robustness, explained without more mathematics than an attentive reader can follow. The next two chapters turn the remedy into procedures. Chapter 4 describes double/debiased machine learning in practice, including cross-fitting, the main model variants and the choices an analyst must make. Chapter 5 describes targeted maximum likelihood estimation, the super learner that usually accompanies it, and how the targeting step differs from, and resembles, the debiasing step of double machine learning. The last three chapters address heterogeneity, the question of whom a treatment helps and by how much. Chapter 6 introduces the conditional average treatment effect and the family of meta-learners and Bayesian tree methods that estimate it. Chapter 7 is devoted to generalized random forests and the causal forest, the most widely used tool for this problem. Chapter 8 addresses the uncomfortable fact that heterogeneous effect estimates are much harder to validate than average effects, and describes the tools, from best linear projections to rank-weighted average treatment effects and policy learning, that make them usable. It also returns to the assumptions of the first chapter, examining overlap and unmeasured confounding in the high-dimensional settings where machine learning is most tempting and those assumptions are most strained. Several important topics are left aside or touched only lightly. Causal discovery, the attempt to learn causal graphs from data, is a distinct enterprise with its own literature and receives no treatment beyond a warning. Difference-in-differences, regression discontinuity and synthetic control designs have all acquired machine learning extensions, but they would each need a chapter of background, and the book concentrates on the setting where the three core methods are most fully developed: a treatment whose assignment is explained by measured covariates, with instrumental variables as the main extension. Time-varying treatments and dynamic regimes, where targeted learning is particularly strong, are mentioned where relevant but not developed. Deep-learning architectures designed specifically for treatment effect estimation, such as representation-balancing networks, are acknowledged but not surveyed; they have not displaced the methods covered here in applied work, and they fit into the same framework of nuisance estimation and orthogonal scoring. How to read it The book assumes a reader who is comfortable with regression, conditional expectations and the idea of a sampling distribution, and who has at least a passing familiarity with methods such as random forests, the lasso and gradient boosting. It does not assume training in semiparametric theory. Where formulas help, they are written out in plain notation and explained in words; where they do not help, they are left out. Notation is kept consistent across chapters: Y is an outcome, D or A a treatment (economists prefer the first, biostatisticians the second, and the book follows whichever literature it is discussing), X or W the covariates, and Y(1) and Y(0) the potential outcomes under treatment and control. Readers from economics will find the potential outcomes framework and the partially linear model familiar and may move quickly through the first chapter; the material on targeted learning and super learning in Chapter 5 is where their tradition is thinnest. Readers from epidemiology and biostatistics will be at home with double robustness and TMLE, and may find the econometric treatment of instruments and the partially linear model in Chapter 4 the most novel. Data scientists trained in machine learning will find the first three chapters the most important, because they explain why a model that validates beautifully on a held-out set can still produce a badly biased treatment effect. The discipline this book asks of its reader can be put simply. Before choosing a learner, name the estimand. Before trusting an estimate, state the assumptions under which it means what you want it to mean, and check the ones that can be checked. Before acting on an estimated pattern of heterogeneity, test whether it is there. None of this is new. What is new is that we now have estimators flexible enough to take the data seriously and principled enough to report honestly how uncertain they are. Chapter 1. Estimands Before Estimators Machine learning is organised around a single question: given inputs, what output should we expect? A causal question has a different shape. It asks what the output would be if we reached into the world and changed one of the inputs, holding fixed the process that generated everything else. The two questions coincide only under special conditions, and the whole of causal inference can be read as a disciplined account of what those conditions are and what to do when they fail. A familiar example makes the difference concrete. Across many health systems, patients who receive intensive care are more likely to die in the following month than patients who do not. A model trained to predict mortality will rightly learn that admission to intensive care is a strong predictor of death. Nobody concludes that intensive care kills people. Sicker patients are sent there, and sickness drives both the treatment and the outcome. The predictive relationship is real and useful; it is simply not the answer to the question "what would happen to this patient's chances if we admitted them?" The predictive model has no way of telling the two apart, because the difference lies not in the data but in the process that produced them. Potential outcomes The most widely used language for making causal questions precise is the potential outcomes framework, associated with Jerzy Neyman's work on agricultural experiments in the 1920s and developed for observational studies by Donald Rubin from the 1970s onward. Its central move is to imagine, for each unit, a set of outcomes indexed by the treatment the unit might receive. For a binary treatment, each unit i has two potential outcomes: Yᵢ(1), the outcome it would experience if treated, and Yᵢ(0), the outcome it would experience if not. The individual causal effect is the difference Yᵢ(1) − Yᵢ(0). The difficulty, which Paul Holland in 1986 called the fundamental problem of causal inference, is that we never observe both. A patient is either admitted or not; a customer either receives the discount or does not. What we observe is Yᵢ = Dᵢ·Yᵢ(1) + (1 − Dᵢ)·Yᵢ(0), where Dᵢ records which treatment was actually received. Half of the information needed to compute any individual effect is always missing. Causal inference is, in this sense, a missing data problem, and many of the tools in this book were first developed in the missing data literature. Two assumptions are built into this notation and deserve naming. The first is that the treatment received by one unit does not affect the outcomes of others, so that Yᵢ depends only on Dᵢ and not on Dⱼ. The second is that there is only one version of treatment, so that "treated" means the same thing for everyone. Together these are known as the stable unit treatment value assumption. It fails in obvious ways in vaccination programmes, where immunising your neighbour protects you, and in marketing experiments where treated customers talk to untreated ones. It fails in subtler ways when "the treatment" bundles together interventions that differ across sites. The methods in this book take it as given; when it is doubtful, the estimand itself must be redefined before any estimation begins. Choosing the estimand Because individual effects are unobservable, causal analyses target summaries of them. The choice of summary is not a technicality. It determines what question the analysis answers, and different summaries can have different signs in the same data. The average treatment effect, E[Y(1) − Y(0)], is the mean effect across the whole population. It answers the question of what would happen, on average, if everyone were treated rather than no one. The average treatment effect on the treated, E[Y(1) − Y(0) | D = 1], restricts attention to those who actually received treatment. It is often the more natural target when evaluating an existing programme, because the policy question is whether the programme helped the people it reached, not whether it would have helped people it was never designed for. The conditional average treatment effect, E[Y(1) − Y(0) | X = x], describes how the average effect varies with observed characteristics X. It is the building block for personalisation: for deciding who should be treated. Beyond these, there are estimands defined by instruments, such as the local average treatment effect of Guido Imbens and Joshua Angrist, which describes the effect among "compliers" whose treatment status is moved by an instrument; estimands defined by policies, such as the expected outcome if treatment were assigned according to some rule; and estimands defined by continuous or multi-valued treatments, such as a dose-response curve. The ones used most often in this book, and the questions each answers, are set out in Table 1. Table 1. Common causal estimands and the questions they answer. Estimand Definition Question answered Typical use ATE E[Y(1) − Y(0)] Effect of treating everyone versus no one Population-wide policy ATT E[Y(1) − Y(0) given D = 1] Effect on those actually treated Programme evaluation CATE E[Y(1) − Y(0) given X = x] Effect for units with characteristics x Targeting, personalisation LATE Effect among compliers with an instrument Effect for those moved by the instrument Instrumental variables designs Policy value E[Y(π(X))] for a rule π Mean outcome if treatment follows rule π Choosing whom to treat The practical lesson is that the estimand must be named before the method is chosen, and named in terms of the decision the analysis is supposed to inform. A pharmaceutical regulator deciding whether to approve a drug for a population needs something like an average effect in that population. A clinician deciding whether to prescribe it to the patient in front of her needs something closer to a conditional effect given that patient's characteristics. A health system deciding how to allocate a limited number of places in a programme needs a policy value, or at least a ranking of patients by expected benefit. These are three different targets, and a single analysis that reports "the effect" without specifying which one it means is not finished. The estimand is also where the most consequential choices about the target population are made. The ATE in a sample of trial volunteers is not the ATE in the population of patients who will eventually receive a drug, and no amount of statistical sophistication within the sample can close that gap without additional assumptions about how the two populations differ. Machine learning has made it easier to estimate effects conditional on many covariates, and therefore, in principle, easier to transport them to new populations with different covariate distributions. But transport requires the assumption that the conditional effects themselves are the same across populations, which is a claim about the world, not about the model. Identification: from potential outcomes to data An estimand is defined in terms of potential outcomes, which we cannot observe. Identification is the step that expresses it in terms of the distribution of variables we can observe. This step is where causal knowledge enters the analysis, and it is logically prior to estimation. If an estimand is identified, then with enough data it could be computed exactly; the remaining questions are statistical. If it is not identified, then even an infinite sample would not reveal it, and no estimator, however clever, can help. In a randomised experiment, identification is straightforward. Because treatment is assigned by a coin flip, it is independent of the potential outcomes, and the difference in mean observed outcomes between treated and control groups equals the ATE. This is why randomised trials occupy their privileged place: the design itself delivers identification without any assumption about how covariates relate to outcomes. In observational data, the standard route to identification rests on three assumptions. The first is conditional exchangeability, also called unconfoundedness, ignorability or selection on observables depending on the discipline: within groups of units that share the same covariates X, treatment is as good as randomly assigned. Formally, the potential outcomes are independent of D given X. The second is overlap or positivity: every unit has a probability of treatment strictly between zero and one given its covariates, so that for every kind of treated unit there are comparable untreated units, and vice versa. The third is consistency, which links the observed outcome to the potential outcome under the treatment actually received and is closely related to the no-multiple-versions condition above. Under these assumptions, the ATE can be written in two ways that will recur throughout the book. The first uses the outcome regressions μ₁(x) = E[Y | D = 1, X = x] and μ₀(x) = E[Y | D = 0, X = x]: the ATE equals the average over the population of μ₁(X) − μ₀(X). This is sometimes called the g-formula or standardisation, and it says that we can compare treated and untreated units within each covariate stratum and then average across strata. The second uses the propensity score e(x) = P(D = 1 | X = x), introduced by Paul Rosenbaum and Donald Rubin in 1983: the ATE equals the average of D·Y/e(X) − (1 − D)·Y/(1 − e(X)). This is inverse probability weighting, and it says that we can reweight the observed sample so that treated and untreated groups each resemble the whole population. Both formulas are exact under the identifying assumptions. Both involve functions, μ and e, that must be estimated from data. And both, as the next two chapters show, behave very differently when those functions are estimated by flexible machine learning methods. But the point to register here is that neither formula comes from the data. The claim that conditioning on X removes confounding is a substantive assertion about the world, justified, if at all, by knowledge of how treatment decisions were made. What machine learning cannot supply It is tempting to think that the more covariates one includes, the more plausible unconfoundedness becomes, and that machine learning, by making it practical to condition on hundreds or thousands of variables, therefore strengthens identification. This is only partly true, and the part that is false causes real damage. It is true that many observational analyses have been undermined by conditioning on too few variables, often because the linear models used could not accommodate more without overfitting. Machine learning relaxes that constraint. If the relevant confounders are among the measured variables but interact with one another in complicated ways, a flexible model can adjust for them where a linear regression with main effects could not. But adding variables can also make things worse. Conditioning on a variable that is affected by the treatment, a mediator or a later consequence, blocks part of the effect one is trying to estimate or opens spurious paths. Conditioning on a collider, a variable caused by both treatment and outcome or by their causes, can create association where there was none. Conditioning on a strong predictor of treatment that has no relationship with the outcome, an instrument in disguise, does nothing to reduce confounding bias but inflates variance and, when some unmeasured confounding remains, can amplify the bias that is left. The graphical framework developed by Judea Pearl and others gives precise rules, the back-door criterion among them, for deciding which sets of variables are sufficient for adjustment, and those rules depend on the causal structure, not on how well the variables predict anything. This is why "throw every available variable into the model and let the algorithm sort it out" is not a causal strategy. The algorithm can sort variables by their predictive value for outcome or treatment. It cannot sort them by their causal role, because causal role is not a property of the joint distribution of the observed data. Two data-generating processes with identical observational distributions can have different causal structures and different treatment effects. The choice of adjustment set therefore has to be made, at least in outline, before the machine learning begins, using subject-matter knowledge about timing and mechanism. A useful working rule is to include pre-treatment variables that plausibly affect the outcome, to exclude anything measured after treatment was determined, and to think carefully before including variables that affect treatment but plausibly not the outcome. The same caution applies to overlap. Machine learning can estimate propensity scores for rich covariate sets, and those estimates frequently reveal that some kinds of units are almost always treated or almost never treated. That is useful information: it shows where the data contain no comparison. But it is information about a limitation, not a solution to one. When overlap fails, the average treatment effect in the full population is not identified from the data without extrapolation, and a flexible model's extrapolation is no more trustworthy than a linear model's. Chapter 8 returns to this point, because the high-dimensional settings in which machine learning is most attractive are exactly those in which strict overlap is least likely to hold. Beyond selection on observables Unconfoundedness is a strong assumption, and in many applications it is not credible. The econometric tradition has developed designs that achieve identification by other means. An instrumental variable is a source of variation in treatment that affects the outcome only through treatment and is itself as good as random: lotteries for school places, distance to a hospital, the random assignment of judges of differing severity. Regression discontinuity exploits rules that assign treatment on the basis of a threshold in a running variable. Difference-in-differences compares changes over time between groups that were and were not exposed to a policy, assuming their trends would otherwise have been parallel. Each design replaces unconfoundedness with a different assumption, and each has been extended with machine learning in the same spirit as the methods of this book: the design provides identification, and flexible learners estimate the nuisance functions that the identified estimand depends on. Double machine learning, in particular, handles instrumental variable models naturally, and Chapter 4 describes how. The general lesson carries over. Machine learning enters after identification has been secured and serves to estimate, as flexibly as possible, the quantities that identification has shown to be relevant. Why the distinction matters in practice A reader who has absorbed the argument so far might wonder why causal machine learning needs a separate literature at all. If identification is settled first and machine learning simply estimates the regression μ or the propensity score e, why not fit the best predictive model available for each and plug it into the relevant formula? The answer, which occupies the next chapter, is that a good predictor of Y or of D is not automatically a good ingredient for estimating the treatment effect. Predictive models are tuned to minimise prediction error, and the errors they are allowed to make are chosen with that goal in mind. Those errors do not cancel when the model is plugged into a causal formula; they propagate into the estimate, often in systematic directions and at rates that swamp the sampling uncertainty. The resulting bias can be large even when the model predicts beautifully, and, more insidiously, it invalidates the standard errors, so that the analyst is not only wrong but confidently wrong. Seeing why this happens is the key to understanding every method in the rest of the book. Double machine learning, targeted learning and generalized random forests are, at bottom, three answers to one question: how can we use predictive models that are good but inevitably biased as components of an estimator that is itself unbiased enough, at the scale of its own standard error, to support inference? The question makes sense only once the estimand and its identification are fixed. That is why they come first. Chapter 2. Why Plugging In Prediction Fails The simplest way to combine machine learning with causal inference is to take one of the identification formulas from the previous chapter, estimate the functions it requires with the best available predictive model, and compute the result. Fit a gradient-boosted model for the outcome given treatment and covariates, predict each unit's outcome under treatment and under control, and average the difference. Or fit a random forest for the probability of treatment, and use its predictions as weights. These plug-in estimators are intuitive, easy to implement and, in general, wrong in a way that matters. This chapter explains why, because the explanation contains the whole logic of the remedy. A model that economists already use The clearest way to see the problem is through a model that has been the backbone of applied econometrics for decades: the partially linear regression. It supposes that the outcome depends on a treatment D with a constant effect θ, plus an unknown function g of covariates X, plus noise: Y = θ·D + g(X) + U, with E[U | X, D] = 0. Treatment itself depends on the covariates through another unknown function m: D = m(X) + V, with E[V | X] = 0. In words, the covariates confound the relationship between D and Y, because they affect both, and we want θ, the effect of D on Y holding X fixed. Traditional practice replaces g with a linear function of X, perhaps with a few interactions and squares chosen by the analyst, and runs ordinary least squares. When X contains many variables or g is highly nonlinear, this is a serious restriction: if the linear specification misses part of g that is correlated with D, the estimate of θ absorbs the omitted confounding and is biased. The motivation for machine learning is exactly to estimate g without committing to a functional form. The natural first attempt is iterative. Guess θ, estimate g by fitting a random forest or lasso to Y − θ·D on X, update θ by regressing Y − ĝ(X) on D, and repeat until the estimates settle. Or, more directly, split the sample, estimate g on one half and regress Y − ĝ(X) on D in the other. Either way the final step is a regression of the residualised outcome on the treatment. It looks like a sensible generalisation of what econometricians have always done. Regularisation bias The trouble lies in what the machine learning model does to g. Every modern predictive method controls its variance by some form of regularisation: the lasso shrinks coefficients toward zero, random forests average over trees grown on subsamples and stop splitting when leaves become small, boosting takes small steps and stops early, neural networks use weight decay and dropout. Regularisation deliberately introduces bias in exchange for lower variance, and in high-dimensional problems this trade is what makes prediction possible at all. For estimating g, the resulting ĝ converges to the truth, but more slowly than the familiar root-n rate of parametric statistics. Typical rates for nonparametric learners in reasonably high dimensions are closer to n to the power minus one quarter, or slower. Now examine what this slow convergence does to the estimate of θ. Writing out the error of the naive estimator, as Chernozhukov and co-authors do in their 2018 paper introducing double/debiased machine learning, it separates into two parts. One part is a well-behaved average of noise terms, which, scaled by root-n, converges to a normal distribution just as in textbook regression. The other part is driven by the product of the treatment's dependence on X and the error in ĝ: roughly, the scaled sum over observations of m(Xᵢ) multiplied by the gap between the true g(Xᵢ) and its estimate. Because ĝ converges more slowly than root-n, this second term does not vanish when multiplied by root-n. It grows. The estimator is consistent, in the sense that it eventually gets close to θ, but its bias shrinks more slowly than its standard error. At any realistic sample size the confidence interval is centred in the wrong place, and as the sample grows the problem, measured in units of standard error, gets worse rather than better. The mechanism is intuitive once stated. Regularisation shrinks ĝ toward something simpler than the true g. Whatever part of g the learner fails to capture is left in the residual Y − ĝ(X). Since g and m are both functions of the same covariates, the uncaptured part of g is typically correlated with D through m. The final regression therefore attributes some of the omitted confounding to the treatment, exactly as an under-specified linear regression would. Machine learning has not removed omitted variable bias; it has reduced it to a level that is small in absolute terms but still large relative to the precision the analyst claims. Simulations make the damage vivid. The 2018 paper shows that, with a moderately complex g and a random forest used to estimate it, the naive estimator's sampling distribution is visibly shifted away from the true θ, while the corrected estimator described in the next chapter is centred on it. The shift is not small noise. It is a systematic displacement of a size comparable to the width of the distribution, which means that nominal ninety-five per cent confidence intervals cover the truth far less often than advertised. An earlier warning from the lasso The same lesson had already been learned in a narrower setting. In the early 2010s, economists began using the lasso to select control variables from large candidate sets. The obvious procedure was to regress the outcome on the treatment and all candidate controls with a lasso penalty on the controls, keep the controls with non-zero coefficients, and then run ordinary least squares of the outcome on the treatment and the selected controls. Alexandre Belloni, Victor Chernozhukov and Christian Hansen showed in 2014 that this procedure can fail badly. The lasso selects variables that predict the outcome well. A confounder that strongly affects treatment but only moderately affects the outcome may be dropped, because its predictive contribution to Y is small relative to the penalty. Yet omitting it biases the treatment effect precisely because it is strongly related to D. Worse, whether it is dropped depends on the particular sample, so the post-selection estimator has a distribution that is not normal and not centred at the truth, and standard confidence intervals are unreliable. This is a specific instance of a general result in the post-selection inference literature, associated particularly with Hannes Leeb and Benedikt Pötscher, that inference after data-driven model selection is fragile. Their remedy was double selection: run one lasso of Y on the controls and a second lasso of D on the controls, and then include the union of the variables selected by either in the final regression. A confounder that matters strongly for treatment will be picked up by the second lasso even if the first misses it. The resulting estimator is robust to the selection mistakes that each lasso inevitably makes, and it supports valid inference under conditions the authors spell out. The name "double" in double machine learning descends from this idea: learn both the outcome's and the treatment's dependence on covariates, and use both. Overfitting bias Regularisation bias is the first of two problems. The second arises when the same data are used both to fit the nuisance functions and to estimate the treatment effect. Flexible learners can fit the training data very closely. A deep random forest or a large boosted ensemble will, on the observations it was trained on, produce predictions that track the noise as well as the signal. If the residuals Y − ĝ(X) are computed on those same observations, they are artificially small and systematically related to the noise in the outcome. Using them in the final regression introduces a bias term involving the correlation between the estimation error in ĝ and the outcome noise, which need not vanish at the required rate. Classical semiparametric theory avoided this problem by imposing conditions, known as Donsker conditions, that limit how complex the class of estimated functions can be. Those conditions are satisfied by many traditional smoothers but are hard to verify, and often false, for the adaptive, high-capacity learners that practitioners actually use. The remedy for overfitting bias is sample splitting: estimate the nuisance functions on one part of the data and evaluate them on another. Because the evaluation sample was not used for fitting, the errors in the nuisance estimates are independent of the noise in the evaluation observations, and the problematic term averages out. Splitting on its own wastes half the data, which is why the methods of this book use cross-fitting, rotating the roles of the folds so that every observation is used for estimation and every observation also receives nuisance predictions from a model that did not see it. Cross-fitting is described in detail in Chapter 4. Here it is enough to note that sample splitting fixes overfitting bias but does nothing about regularisation bias. A naive estimator with sample splitting is still biased, because the omitted part of g remains correlated with D whether or not ĝ was fitted on the same observations. The same problem, other formulas The partially linear model makes the mechanism easy to see, but the problem is general. Consider the two identification formulas for the ATE from Chapter 1. The outcome regression estimator averages μ̂₁(X) − μ̂₀(X), where μ̂₁ and μ̂₀ are machine learning models of the outcome in the treated and control arms. Its error is essentially the average of the errors in μ̂₁ and μ̂₀ across the population. Those errors are dominated by regularisation bias in regions where data are sparse, and they are systematic: a learner that shrinks toward the overall mean will understate differences between arms in regions where treated units are rare. The error shrinks at whatever rate the learner converges, which is typically slower than root-n, so the estimator again has bias that dominates its standard error. There is also no general way to compute a valid standard error for it, since the learner's own sampling behaviour is complicated and poorly characterised. The inverse probability weighting estimator divides by ê(X), a machine learning estimate of the propensity score. Its bias depends on the error in ê, again typically converging slowly, and the division makes matters worse: where the true propensity is small, small absolute errors in ê produce large relative errors in the weights. Machine learning classifiers are also frequently poorly calibrated, meaning that their predicted probabilities do not match observed frequencies, particularly near zero and one. A boosted model tuned for classification accuracy can push predicted probabilities toward the extremes, producing enormous weights for a handful of observations and an estimator whose variance is dominated by them. Neither formula, used alone with a machine learning estimate plugged in, delivers what the analyst needs: an estimate whose error is dominated by sampling noise of a known, root-n size, so that a normal approximation and a confidence interval are available. Good prediction, wrong target A deeper point sits underneath both problems. When a learner is tuned by cross-validation to predict Y as accurately as possible, it is optimising a criterion that weighs all errors in the prediction of Y equally, wherever in covariate space they occur and whatever their relation to treatment. For the causal estimate, however, errors are not equally costly. An error in ĝ that is uncorrelated with D does no harm to the estimate of θ; it only adds noise. An error that is correlated with D, even a small one, translates directly into bias. The predictive criterion is blind to this distinction, so the learner happily trades away accuracy in exactly the directions that matter most for the causal question if doing so buys accuracy elsewhere. Consider a stylised evaluation of a job-training programme. Suppose participation is strongly driven by recent unemployment spells, which also depress future earnings, while future earnings are driven mainly by education and prior earnings, with recent unemployment playing a smaller role. A learner fitted to predict future earnings will concentrate its capacity on education and prior earnings, where most of the predictable variation lies, and will model the unemployment effect coarsely, perhaps shrinking it heavily. From a predictive standpoint this is the right choice: the unemployment variable adds little to the accuracy of earnings forecasts. From a causal standpoint it is the worst possible choice, because the part of the earnings process the learner neglected is precisely the part that is entangled with participation. The residual earnings after subtracting the learner's predictions will still contain a component that is lower among participants, and the programme will look less effective than it is. This is the same mechanism as the dropped confounder in the naive lasso, generalised to any learner. It explains a pattern that practitioners sometimes find baffling: switching to a more accurate outcome model, as judged by held-out error, does not reliably improve a plug-in treatment effect estimate and can make it worse. Predictive accuracy on Y is simply not the quantity that governs the causal error. What governs it is how well the learner captures the part of the outcome process that co-varies with treatment, and no predictive criterion measures that directly. The remedy must therefore change the structure of the estimator rather than rely on ever-better predictions. What an answer must look like It helps to be precise about the requirement. Causal inference with flexible nuisance estimation asks for an estimator θ̂ such that root-n times (θ̂ − θ) is approximately normal with mean zero and a variance that can be estimated. The nuisance functions will be estimated at slower rates; that is unavoidable once we refuse to assume a parametric form. So we need the error in the final estimate to depend on the nuisance errors only weakly, weakly enough that slow convergence of the nuisances does not contaminate the fast convergence of θ̂. One way to achieve this would be to find estimators whose error depends on the nuisance errors only through a product of two of them. If the error in the outcome model is of order n to the minus one quarter, and the error in the propensity model is also of order n to the minus one quarter, their product is of order n to the minus one half, which is exactly the scale of sampling noise. Multiplied by root-n, it vanishes. Each nuisance can then be learned at a pedestrian rate, one achievable by lasso, random forests, boosting or neural networks under reasonable conditions, while the combined estimator converges at the parametric rate. Estimators with this property exist, and they are not new. They are built from what semiparametric statisticians call influence functions and what econometricians, following Neyman, call orthogonal scores. The augmented inverse probability weighted estimator of James Robins, Andrea Rotnitzky and Lue Ping Zhao, published in 1994, has it. So does the partialling-out estimator for the partially linear model that Peter Robinson analysed in 1988, which regresses the residual Y − E[Y | X] on the residual D − E[D | X]. The contribution of the modern literature was to see that these old constructions are precisely what is needed to make machine learning usable for causal inference, to state the conditions under which they work with arbitrary learners, and to combine them with cross-fitting so that the conditions become easy to meet. It is worth pausing on Robinson's estimator, because it is the simplest example of the remedy and the direct ancestor of both double machine learning and the causal forest. Instead of residualising only the outcome, it residualises both outcome and treatment on the covariates, and then regresses one residual on the other. If the outcome model ℓ(X) = E[Y | X] is slightly wrong and the treatment model m(X) = E[D | X] is slightly wrong, the errors enter the final regression only through their product. The treatment residual D − m̂(X) is approximately uncorrelated with any function of X, including the part of the outcome that the outcome model missed, so the omitted confounding no longer leaks into the estimate. This is the Frisch–Waugh–Lovell theorem of linear regression, made robust to the imperfect nonparametric estimation of both projections. The next chapter explains why it works, and why the same principle generalises to almost any estimand one might care about. Chapter 3. Orthogonal Scores and the Influence Function Every method in this book depends on one idea, which appears under several names. Econometricians speak of Neyman orthogonality and orthogonal moment conditions. Biostatisticians speak of efficient influence functions, efficient influence curves and doubly robust estimating equations. Machine learning researchers increasingly speak of debiasing. This chapter explains the idea without assuming a background in semiparametric theory, because once it is understood the three families of methods become variations on a theme rather than separate techniques to be memorised. Estimating equations and their sensitivity Most estimators can be written as the solution to an estimating equation: choose θ so that the sample average of some score function equals zero. Ordinary least squares, for instance, sets the average of the residual times each regressor to zero. Maximum likelihood sets the average score, the derivative of the log-likelihood, to zero. In causal problems, the score typically depends not only on the parameter of interest θ but also on nuisance functions such as the outcome regression and the propensity score. Write the nuisances collectively as η. The estimator solves average of ψ(Wᵢ; θ, η̂) = 0, where Wᵢ collects the observed data for unit i and η̂ is the estimated nuisance. The question the previous chapter raised is how sensitive θ̂ is to errors in η̂. Suppose we perturb the nuisance slightly from its true value, moving η in some direction by a small amount r. The expected score changes. If it changes in proportion to r, then an error of size r in the nuisance produces an error of roughly size r in θ̂. That is the situation of the naive plug-in estimators: nuisance errors pass through one-for-one, and slow nuisance convergence means slow convergence of θ̂. Neyman orthogonality is the requirement that the derivative of the expected score with respect to the nuisance, evaluated at the truth, be zero in every direction. If this holds, then a perturbation of size r in the nuisance changes the expected score only by an amount proportional to r squared, or to a product of errors in different nuisance components. Small errors become very small errors. An estimator built on such a score inherits this insensitivity: to first order, it behaves as though the true nuisance functions were known. Jerzy Neyman introduced the idea in 1959 in the context of hypothesis testing with nuisance parameters, as the basis of his C(α) tests. He noticed that a test statistic could be made locally insensitive to the estimation of nuisance parameters by projecting the score for the parameter of interest onto the space orthogonal to the scores for the nuisances. That projection is what gives the idea its name. The modern contribution is to apply it when the nuisance is not a handful of parameters but an entire function learned by a flexible algorithm. The partialling-out score The partially linear model from the previous chapter provides the simplest example. The naive score, which regresses Y − g(X) on D, is ψ = (Y − θ·D − g(X))·D. Differentiating its expectation with respect to g in any direction gives a term proportional to the expected value of D times the direction, which is not zero because D depends on X. The score is sensitive to errors in g. The orthogonal score residualises both variables. Let ℓ(X) = E[Y | X] and m(X) = E[D | X]. The score is ψ = (Y − ℓ(X) − θ·(D − m(X)))·(D − m(X)). Setting its sample average to zero gives Robinson's estimator: the regression of the outcome residual on the treatment residual. Differentiate with respect to ℓ: the result involves E[D − m(X) | X], which is zero by the definition of m. Differentiate with respect to m: the result involves E[Y − ℓ(X) − θ·(D − m(X)) | X] and a term in E[D − m(X) | X], both of which vanish at the truth. The score is orthogonal to both nuisances. The intuition is worth making explicit. The treatment residual D − m(X) is the part of treatment that cannot be predicted from covariates, the "as good as random" variation that identification rests on. Because it is by construction uncorrelated with every function of X, any error in the outcome model, which is a function of X, is also uncorrelated with it. The error washes out in the final regression instead of being attributed to treatment. Symmetrically, errors in m shift the treatment residual by a function of X, but the outcome residual is also uncorrelated with functions of X at the truth, so these errors wash out too. Only when both models are wrong in related ways does an error survive, and then it enters as a product. Influence functions The partialling-out score was easy to find because the model is simple. For other estimands, a systematic route to orthogonal scores runs through the influence function, a concept from robust statistics and semiparametric theory. The influence function of an estimand describes how the estimand changes when the distribution of the data is perturbed slightly toward a point mass at a single observation. Informally, it measures each observation's contribution to the estimate. For well-behaved estimators, the estimation error is approximately the sample average of the influence function evaluated at each observation, which is why the influence function determines the estimator's asymptotic variance: the variance of θ̂ is approximately the variance of the influence function divided by n. For an estimand such as the ATE, many estimators are possible, each with its own influence function. Among them there is one with the smallest variance, the efficient influence function. Its variance defines the semiparametric efficiency bound, the lowest variance any regular estimator can achieve without additional assumptions. For the ATE under unconfoundedness, this bound was derived by Jinyong Hahn in 1998, building on earlier work by James Robins and colleagues. The general theory is set out in the 1993 monograph by Peter Bickel, Chris Klaassen, Ya'acov Ritov and Jon Wellner. The crucial property for our purposes is that the efficient influence function, viewed as a score, is automatically Neyman orthogonal. This is not a coincidence. The efficient influence function is constructed by projecting out every direction in which the nuisance could vary, which is exactly what orthogonality requires. So a practical recipe for building a debiased estimator for any smooth estimand is to find its efficient influence function and use it as the estimating equation. Oliver Hines, Oliver Dukes, Karla Diaz-Ordaz and Stijn Vansteelandt give an accessible account of how to do this in a 2022 article in The American Statistician, and Aaron Fisher and Edward Kennedy offer a visual introduction to the same ideas in a 2021 article in the same journal. The doubly robust score for the average treatment effect The efficient influence function for the ATE under unconfoundedness produces the estimator that sits at the centre of modern practice. With μ₁(x) and μ₀(x) the outcome regressions in the treated and control arms and e(x) the propensity score, the score for each observation is μ₁(X) − μ₀(X) + D·(Y − μ₁(X))/e(X) − (1 − D)·(Y − μ₀(X))/(1 − e(X)) − θ. The first part, μ₁(X) − μ₀(X), is the outcome-regression estimate of the individual contrast. The second part is a correction: the inverse-probability-weighted average of the outcome model's residuals. If the outcome model were perfect, the residuals would average to zero within each covariate stratum and the correction would vanish. If the outcome model is biased in some region, the correction measures and removes that bias using the observed outcomes, weighting by the inverse propensity so that the removal is representative of the whole population. Averaging this score over the sample gives the augmented inverse probability weighting estimator, usually abbreviated AIPW, introduced by Robins, Rotnitzky and Zhao in 1994. It is also what Heejung Bang and James Robins, in an influential 2005 paper, described as doubly robust, and the term has stuck. The estimator is consistent if either the outcome model or the propensity model is correctly specified, even if the other is wrong. If the outcome model is right, the correction term has mean zero regardless of the propensity. If the propensity model is right, the weighting correction exactly offsets any error in the outcome model on average. Double robustness was originally valued as insurance against misspecification of parametric models: an analyst fitting logistic and linear regressions could be wrong about one and still obtain a consistent estimate. In the machine learning setting, its more important consequence is quantitative. The bias of the AIPW estimator, when both nuisances are estimated with error, is governed by the product of the two errors. Specifically, the leading bias term is proportional to the expected product of the propensity error and the outcome-regression error, roughly the integral of (e − ê)·(μ − μ̂) weighted by the inverse of the estimated propensity. This is the mixed bias, or product rate, property. The product rate condition The product structure is what makes machine learning usable. For the bias to be negligible relative to the standard error, which is of order one over root-n, the product of the two nuisance errors must shrink faster than one over root-n. This holds, for instance, if both errors shrink faster than n to the minus one quarter. It also holds if one nuisance is estimated very well and the other rather poorly: a nearly known propensity score, as in a randomised experiment with known assignment probabilities, compensates for a crude outcome model, and vice versa. The rate n to the minus one quarter is attainable by many machine learning methods under reasonable conditions. The lasso achieves it when the true regression is approximately sparse, meaning that a modest number of variables capture most of the signal. Random forests, boosted trees and neural networks achieve it under various smoothness or structural assumptions that have been established in the theoretical literature over the past decade. None of these guarantees is unconditional, and in any particular application one cannot verify that the rate holds. But the condition is far weaker than the root-n rate that plug-in estimators would require, and it is weaker than the correct specification that parametric methods require. That relaxation, from "the model must be right" to "the models must be reasonably good, jointly", is the practical gain. It is worth being clear about what the rate condition does not say. It does not say that any learner will do, or that nuisance quality does not matter. The product must be small, and in finite samples the constant in front of it matters as much as the rate. Poorly tuned learners, or learners that systematically miss the structure of the problem, produce product terms that are not negligible at realistic sample sizes. Chapter 4 discusses how to choose and tune learners with this in mind. The limits of double robustness Double robustness is sometimes presented as if it made an estimator safe. The history of the idea contains a useful corrective. In 2007, Joseph Kang and Joseph Schafer published a simulation study in Statistical Science, pointedly titled "Demystifying double robustness", in which both the outcome and propensity models were moderately misspecified in a realistic way. Several doubly robust estimators performed worse than a simple outcome regression, some of them dramatically so. The reason was that the misspecified propensity model produced a few very small estimated probabilities, and the inverse weights attached to those observations were enormous. The correction term, which is supposed to repair the outcome model's errors, instead amplified noise from a handful of units. The discussion that followed, with responses from Robins, Rotnitzky, van der Laan and others, clarified several points that remain important. First, double robustness is a statement about consistency, about what happens as the sample grows; it offers no guarantee about finite-sample performance when both models are wrong. Second, the variance of inverse-weighted corrections depends heavily on overlap, and estimators that let weights explode inherit that instability. Third, there are ways of constructing doubly robust estimators that are far more stable, for example by keeping predictions within the range of the observed outcomes, which is one of the motivations for the targeted learning approach of Chapter 5. In the machine learning setting, the lesson translates directly. Flexible propensity models can produce predicted probabilities very close to zero or one, especially when many covariates are available and the model is allowed to overfit. A doubly robust estimator built on such predictions is formally orthogonal and practically fragile. The product rate condition assumes, among other things, that the propensity is bounded away from zero and one; when it is not, the constant multiplying the product of errors grows without limit and the theoretical guarantees become empty. Diagnosing and responding to weak overlap is therefore not an optional refinement but part of the method, and Chapter 8 treats it in detail. Why orthogonality alone is not enough Orthogonality addresses regularisation bias. It does not by itself address overfitting bias, the second problem from the previous chapter. The proof that an orthogonal-score estimator behaves as if the nuisances were known requires controlling a term involving the empirical process: the difference between the sample average and the population average of the score, evaluated at the estimated rather than the true nuisance. Classical theory handles this term by restricting the complexity of the nuisance estimators, the Donsker conditions mentioned earlier. Modern learners often violate those conditions. Cross-fitting removes the need for them. If the nuisance used to evaluate the score for observation i was fitted without observation i, then conditional on the fitted nuisance the score terms are independent across the evaluation fold, and the troublesome empirical-process term can be bounded with elementary arguments that require only that the nuisance errors shrink. This combination, an orthogonal score plus cross-fitting, is the complete recipe of double/debiased machine learning, and its essential logic is shared by every other method in this book. Standard errors for free A further benefit of building estimators on influence functions is that inference follows almost automatically. Because the estimation error is approximately the average of the influence function across observations, the variance of the estimator can be estimated by the sample variance of the estimated score values divided by n. There is no need to bootstrap the whole machine learning pipeline, which would be computationally expensive and, for some learners, theoretically unjustified. The analyst computes each observation's score contribution, takes their mean for the point estimate and their standard deviation for the standard error, and forms a confidence interval in the usual way. This simplicity is easy to underrate. Earlier attempts to use flexible models for causal effects often foundered on inference: an estimate from a random forest or a neural network could be computed, but no honest measure of its uncertainty was available. The influence function approach converts the question of inference about a complex pipeline into the question of estimating the variance of a scalar quantity, which is the most elementary task in statistics. Beyond the average treatment effect The same construction extends well beyond the ATE. The average treatment effect on the treated has its own efficient influence function, involving the outcome model in the control arm and the odds of treatment. Instrumental variable estimands, such as the local average treatment effect, have orthogonal scores built from models of the outcome, treatment and instrument given covariates. Effects of continuous treatments, mediation effects, and the value of treatment policies all admit orthogonal scores, as do the parameters of the best linear approximation to a heterogeneous effect. Victor Chernozhukov, Whitney Newey and Rahul Singh have shown, in work published in Econometrica in 2022, that for a broad class of estimands that are linear functionals of a regression, the correction term can be learned automatically by estimating a function called the Riesz representer, without deriving its analytic form. For the ATE, the Riesz representer is the familiar inverse-propensity weight; for other estimands it has no simple closed form, and learning it directly avoids the numerically fragile step of inverting estimated probabilities. Edward Kennedy's 2022 review, "Semiparametric doubly robust targeted double machine learning", whose title deliberately strings together the names of the competing traditions, argues that these literatures describe one body of theory. Having reached this point, a reader can see why. Double machine learning takes the efficient influence function, uses it as an estimating equation, and adds cross-fitting. Targeted maximum likelihood, described in Chapter 5, takes the same efficient influence function and uses it to adjust an initial estimate so that the equation is solved by a substitution estimator. The causal forest of Chapter 7 uses the partialling-out score locally, within neighbourhoods defined by a forest, to estimate how the effect varies. The practical differences between these methods are real and worth understanding. But they sit on top of a common foundation, and it is the foundation, more than any particular learner, that makes their results trustworthy. Hashtags: #CausalMachineLearning #CausalInference #TreatmentEffectEstimation #PotentialOutcomes #AverageTreatmentEffect #ConditionalAverageTreatmentEffect #HeterogeneousTreatmentEffects #DoubleMachineLearning #DebiasedMachineLearning #TargetedMaximumLikelihood #TMLE #GeneralizedRandomForests #CausalForests #NeymanOrthogonality #InfluenceFunctions #DoubleRobustness #CrossFitting #PropensityScores #ConfoundingAdjustment #NuisanceFunctions #PolicyLearning #MetaLearners #TreatmentEffectHeterogeneity #SemiparametricInference #FutureOfCausalMachineLearning

Download the Book (PDF): Introduction Every archival document arrives with three questions attached, and most researchers answer only one of them. The question everyone asks is what the document says. The questions that decide whether that answer can be trusted are how the document survived to be read, and whose hands and whose purposes carried it from the moment of its making to the moment it lay on a reading-room table. A letter that says a minister opposed a war is evidence of one thing if it sat in his private office files, another if it was retyped for a memoir twenty years later, and something else again if it was released by a successor government keen to embarrass his party. The words are identical in all three cases. The evidence is not. This booklet is built on a single claim: in archival research, the material, the script, and the chain of custody are not background to the evidence but part of it. Preservation, paleography and provenance are usually taught as separate skills, the first to conservators, the second to medievalists and early modernists, the third to archivists. The historian or political scientist who arrives at an archive tends to treat all three as obstacles standing between them and the content. That is a mistake of method, not just of manners. The condition of a file tells you how it was used and what may be missing. The hand in which it is written tells you who produced it and often when. The route by which it reached the archive tells you why it exists at all, and why its neighbours on the shelf do not. Researchers who learn to read those three layers produce arguments that are harder to overturn, and they notice things that others walk past. Who this is for The intended reader is a working researcher who has been trained to handle evidence but not necessarily to handle archives. That includes the doctoral student in political science planning a first trip to a presidential library, the historian of the twentieth century who suddenly needs a set of eighteenth-century court records, the international relations scholar trying to make sense of a redacted cable, and the scholar in an adjacent field (sociology, law, economics, anthropology) who has discovered that the question they care about cannot be answered from published data. Some readers will have spent months in archives and want a more systematic account of what they have been doing by instinct. Others will be preparing for their first visit. History and political science are paired in the title deliberately. The two disciplines use archives differently and have learned different things from them. Historians have long treated the archive as a place where arguments are built from the ground up, where the texture of a source matters and where context is assembled document by document. Political scientists have come to archives more recently and more instrumentally, often to test a hypothesis about decision-making, to trace a causal process, or to build a dataset from records that were never meant to be counted. Each discipline has blind spots the other can correct. Historians can be casual about how representative their sources are; political scientists can be casual about what their sources actually are. The chapters that follow try to give both groups the tools the other takes for granted. What the booklet covers The first chapter starts with the archivist's view of the world, because a researcher who does not understand how archives are arranged will misread them. It explains the principle of provenance, the idea of original order, and the hierarchy of fonds, series, files and items that organizes nearly every archive in the world, and it shows how to read a finding aid for what it conceals as well as what it lists. The second chapter turns to discovery: how to find out where the relevant records are, how to plan a visit, how to work with archivists rather than around them, and how to think about the vast body of digitized material that now sits alongside, and sometimes in place of, the physical archive. Digitization has changed what researchers find and what they miss, and the chapter treats the searchable database as a source with its own biases rather than a neutral window. The third chapter concerns the physical care of documents. Conservation ethics is usually framed as a set of rules imposed on researchers, but the underlying principles (minimal intervention, reversibility, documentation, respect for the object as evidence) are the same principles a careful researcher should apply to interpretation. The chapter explains the main ways in which paper, ink, film and photographs decay, what a researcher should and should not do when handling them, and why the ethics of access extends beyond the physical to the people named in the records. Chapters four and five deal with reading. The first sets out the general method of paleography: how to learn an unfamiliar script, how to handle abbreviations, how to transcribe consistently, and how to date a document from its hand and its calendar. The second applies that method to particular problems: English secretary hand, German Kurrent and Sütterlin, pre-reform Russian, Ottoman and Arabic chancery scripts, and the specifically modern illegibility of carbon copies, faded mimeographs, telegrams and marginal scrawls. It also assesses what automated handwriting recognition can and cannot do. The sixth chapter is about declassified collections: how state secrecy works, how records move from classified to open, how to track what has been released and where, and how to read a redacted document. It concentrates on the United States and the United Kingdom, whose systems are the most documented and most used, but draws out principles that apply wherever governments decide what their citizens may know. The seventh chapter returns to provenance, this time as a problem of interpretation rather than arrangement. It covers authentication and forgery, the chain of custody, the silences built into every archive, and the selection problems that arise when researchers treat what survived as a sample of what happened. The eighth chapter addresses the practical crisis of modern archival research: the thousands of photographs that a researcher now takes in a single visit, and the photographic collections that archives themselves hold. It sets out how to capture, name, describe, store and retrieve large volumes of images so that each one remains a citable piece of evidence years later, and how to read historical photographs as documents with provenance of their own. The conclusion draws out what follows from treating preservation, paleography and provenance as parts of the evidence: for how research is designed, how it is reported, and how the disciplines should train the people who will do it next. A note on scope No short book can cover every archival tradition. The examples here lean toward European and North American state and private archives, with attention to colonial and post-colonial records, because those are the collections most researchers in the two disciplines use and because their histories expose the general problems most clearly. Where the practice differs sharply elsewhere, the text says so. The book deals with records up to the late twentieth century; born-digital records, email archives and web archives raise distinct problems of their own and are mentioned only where they bear on the older material. The book also avoids giving precise environmental figures for storage, legal deadlines that change with legislation, or claims about particular archives' current policies beyond what is well established. Reading-room rules, photography permissions and declassification schedules change, sometimes quickly. Anyone planning a visit should confirm the details with the institution itself, a habit the second chapter argues is valuable for its own sake. How to use it The chapters can be read in order, and the argument builds in that order: arrangement first, then discovery, then handling, then reading, then the particular case of secrecy, then interpretation, then managing what you bring home. But readers facing a specific problem can go straight to the relevant chapter. A researcher about to confront a box of seventeenth-century manuscripts will want chapters four and five. One about to file a declassification request will want chapter six. One who has returned from an archive with twelve thousand unlabelled photographs on a phone will want chapter eight, and quickly. What every chapter shares is a conviction that archival work is a craft with standards, not a matter of luck and endurance. Good archival researchers are not simply people who sat in reading rooms for longer. They are people who knew what they were looking at, knew what they were not seeing, and kept records of their own work careful enough that someone else could check it. Those habits can be taught, and the rest of this booklet tries to teach them. Chapter 1: How Archives Are Arranged, and Why It Matters to Evidence A library and an archive can look alike from the doorway. Both have reading rooms, catalogues and staff who guard the materials. But they are organized on opposite principles, and a researcher who approaches an archive with library habits will misread it. A library arranges items by subject so that a reader can find everything on a topic in one place. An archive arranges records by origin, so that a reader can see how they were created and used. The library asks what a book is about. The archive asks who made this, in the course of doing what, and alongside which other records. That difference is not an administrative detail. It is the foundation of the archive's value as evidence. A document pulled out of its context is a quotation. A document seen in the place its creator filed it, next to the drafts that preceded it, the replies it provoked and the memoranda that ignored it, is a piece of a working process. Historians and political scientists who want to understand how decisions were made, rather than simply what was decided, need that process, and they can only recover it if they understand how archivists preserve it. Provenance and original order Two principles govern the arrangement of most archives in the world. The first is provenance, sometimes called by its French name, respect des fonds. It holds that records created or accumulated by one person, family or organization should be kept together and not mixed with records of any other creator. The principle is usually traced to a French circular of 1841, issued under the Interior Ministry, which instructed departmental archives to group documents by the body that produced them rather than by subject. Before that, many European archives had been rearranged by well-meaning custodians into thematic collections, with the result that the documents of a single institution were scattered and the relationships between them lost. The second principle is original order. It holds that records should be kept in the order in which their creator arranged them, because that order itself is evidence of how the records were used. The classic statement comes from three Dutch archivists, Samuel Muller, Johan Feith and Robert Fruin, whose manual of 1898 became the founding text of modern archival science and was translated into many languages. The idea is simple and powerful. If a foreign ministry filed a telegram in a folder labelled for one crisis rather than another, that choice tells you how officials understood the telegram at the time. If a politician kept certain letters in a separate bundle tied with ribbon, the bundle is a fact about the politician. Neither principle is absolute in practice. Original order is often lost before records reach the archive: offices clear out files in a hurry, families sort a dead relative's papers into piles, and earlier archivists rearranged collections according to the fashions of their time. Archivists then must decide whether to restore an order that can be reconstructed, impose a sensible new one, or leave the chaos as found. Different traditions answer differently. The English archivist Hilary Jenkinson, whose Manual of Archive Administration appeared in 1922, emphasized the archivist as a custodian whose job was to preserve records as received, with as little interference as possible. The American archivist T. R. Schellenberg, whose Modern Archives appeared in 1956, faced the massive volume of modern government paper and argued that archivists must actively appraise and select, keeping records for their evidential and informational value and discarding the rest. The researcher inherits the consequences of both views. In a Jenkinsonian archive, you may find more of the original mess and more of the original meaning. In a Schellenbergian one, you may find a cleaner, smaller body of records whose shape reflects an archivist's judgment about what future researchers would need. There is a further complication that matters more every year. Australian archivists, beginning with Peter Scott in the 1960s, argued that a single fixed creator is a fiction for many modern records, since government functions pass between agencies, departments merge and split, and a series of files may be created by several bodies over its life. The Australian series system describes records at the level of the series and links each series to all the agencies that created or controlled it. Later theorists, notably Frank Upward and colleagues in the 1990s, developed the records continuum model, which treats records as moving through overlapping dimensions of creation, capture, organization and pluralization rather than through a simple life cycle ending in the archive. For the researcher, the practical lesson is that provenance can be multiple. A file on a colonial police force may have been created by the colonial government, captured by the metropolitan ministry, weeded by an intelligence service and finally transferred to a national archive decades later. Each of those hands shaped what survived. The hierarchy of description Archivists describe records in a hierarchy, and researchers need to understand it to read a catalogue correctly. The international standard, ISAD(G), published by the International Council on Archives in 1994 with a second edition in 2000, defines levels of description running from the whole body of records of a creator down to the single item. In the United States, the equivalent content standard is DACS, Describing Archives: A Content Standard, maintained by the Society of American Archivists, and descriptions are often encoded in a structured format called Encoded Archival Description (EAD), which is why so many online finding aids look alike. As Table 1 sets out, the vocabulary differs slightly between traditions, but the logic is shared. Table 1. Common levels of archival description and what each tells a researcher. Level Typical name What it contains What it tells you Whole body Fonds, record group, collection All records of one creator Who made the records and why they exist Subdivision Subfonds, subgroup Records of an office or function Which part of an organization acted Series Series Records filed together by one system How the creator organized its work File File, folder, dossier Records on one matter or period What was grouped as a single case Item Item, piece, document A single letter, report, photograph The unit you will actually cite The level that matters most to researchers, and that they most often neglect, is the series. A series is a set of records kept together because they resulted from the same activity or were filed under the same system: the minutes of a cabinet committee, the incoming correspondence of an embassy, the case files of a court, the registered files of a department using a numerical classification scheme. The series tells you the logic of the creator's filing, and that logic determines where a given document is likely to be. If you understand that a foreign ministry kept its political correspondence in files numbered by country and subject, and that the numbering scheme changed in a particular year, you can predict where to look for a document that the catalogue does not name. If you do not, you are at the mercy of the catalogue's keyword index. The records of the United States Department of State offer a clear example. For most of the twentieth century the department filed its correspondence centrally, and the filing system itself changed in ways a researcher must know. From 1910 to 1963 documents were filed under a decimal scheme, in which a number encoded the subject and the country, so that all material on a given country's internal political affairs fell under a predictable number regardless of which office produced it. In 1963 the department switched to a subject-numeric system using letter codes for broad subjects, and in the 1970s it moved to an electronic telegram system whose records the National Archives later made searchable online. A researcher studying American policy toward one country across the 1960s must therefore search two different filing logics, and one studying the 1970s must combine a database search with the paper records, such as memoranda and briefing papers, that never went into the electronic system. None of this is a secret; the National Archives publishes guides to each system. But researchers who skip the guides and rely on keyword searches in the online catalogue routinely miss the bulk of what exists. It also tells you what is missing. A registry series in which files numbered one to two hundred survive, except for numbers forty through fifty-five, has a gap that demands explanation. Were those files destroyed under a retention schedule, withheld on security grounds, transferred to another department, or simply lost? The gap is visible only to a researcher who looks at the series as a whole rather than jumping to the files that keyword searches return. Reading a finding aid critically A finding aid is the archivist's description of a collection: a document that introduces the creator, explains the scope and content of the records, sets out their arrangement and lists the boxes, files or items. Most researchers treat it as a map. It is better treated as a source in its own right, written by someone with particular knowledge, limits and assumptions, at a particular time. Start with the date and authorship of the finding aid, if they are given. A finding aid written in 1950 may describe a collection in terms that reflect the research interests of 1950, highlighting diplomatic correspondence and ignoring household accounts that a social historian now finds precious. It may use categories that are now offensive or misleading, particularly for colonial records and records concerning Indigenous peoples, enslaved people, women or minorities. Many archives are now revising such descriptions, and some retain the older text with a note explaining the change. Either way, the language of description can hide material. A box listed as "miscellaneous native affairs" may contain the only surviving petitions from a community to a colonial governor. Next, read the biographical or administrative history. It tells you how the archivist understood the creator, and it often contains the most compressed and useful account of how the records came to the archive: by transfer from a government department, by gift from a family, by purchase at auction, by deposit on loan, by seizure. That account of custodial history is the beginning of provenance analysis, which chapter seven develops. It may also reveal that the collection was assembled by a collector rather than created by a single person, in which case the principle of provenance applies only loosely, and the arrangement reflects the collector's interests. Then read the scope and content note and the arrangement note together. The first tells you what the archivist thought the collection was about. The second tells you whether the arrangement is original, reconstructed or imposed. If the arrangement is imposed, the folder titles are the archivist's, not the creator's, and a folder labelled "Correspondence, 1934" may have been assembled from several of the creator's original files. You cannot then treat co-location as evidence that the creator connected those letters. Finally, look at the level of detail. Descriptions are uneven. Since the publication in 2005 of Mark Greene and Dennis Meissner's influential article "More Product, Less Process" in The American Archivist, many repositories have adopted minimal processing to reduce backlogs: collections are described at the series or box level rather than item by item, and are made available sooner with less arrangement. This is a good trade for researchers in most ways, because it opens material that would otherwise have waited decades. But it means a finding aid may list forty boxes as "subject files, A–Z" with nothing more. Such a collection rewards systematic reading and punishes keyword searching. It also means that the archive's own staff may not know what is in it. Beyond the catalogue Finding aids have a boundary, and much of the most useful information about a collection lies beyond it. Archivists keep accession registers recording when and from whom records arrived. They keep correspondence with donors and depositing agencies, and sometimes that correspondence records conditions of access, material withheld, or earlier arrangements. They keep files on previous research requests and on items transferred elsewhere. In many national archives, departmental records also survive about the records themselves: registers of files opened and closed, destruction schedules and certificates, transfer lists. These meta-records are the best evidence of what an archive once contained and what happened to it. A case shows why this matters. When historians of British decolonization worked on the end of empire in Kenya, they drew on the Colonial Office files at the National Archives in London, supplemented by records in Nairobi. The picture those files gave was incomplete in a way that could not be seen from them. In 2011, during litigation brought by Kenyans who had been detained and abused during the Mau Mau emergency, the British government acknowledged that it held a large body of files removed from colonial administrations at independence and kept for decades at a Foreign Office facility at Hanslope Park. Those "migrated archives", covering many former territories, were then transferred to the National Archives in tranches and catalogued as the series FCO 141. Researchers who had used the older files had not been careless. But the existence of the hidden series was, in retrospect, signalled by gaps and references in the records that were available, and by the logic of how colonial governments were instructed to handle their papers at independence. Reading the archive as a system, with attention to what it should contain, is the discipline that makes such absences visible. Much of what historians and political scientists use comes not from government records but from personal papers: the letters, diaries, notes and drafts of politicians, officials, activists, journalists and ordinary people. These collections follow the principle of provenance, but their arrangement is often less systematic and their custodial history more complex. Papers pass through heirs who remove embarrassing material, biographers who borrow and do not return, dealers who split collections for sale, and archives that acquire different portions at different times. The papers of a single prominent figure may be divided among several repositories on two continents. Personal papers also carry restrictions set by donors, which may close portions for a period of years or until the death of named individuals. The finding aid should state these restrictions, and the researcher should record them, because they shape what can be cited and when. Restrictions also shape the evidence itself. A politician who kept the right to close sensitive files during his lifetime, and whose heirs then gave everything to an archive, left a different record from one whose heirs weeded the papers before deposit. The practical rule is to locate every portion of a person's papers before drawing conclusions from any one. National union catalogues, discussed in the next chapter, are the usual starting point, but correspondents' papers are often the best supplement. A letter missing from one person's papers may survive in the recipient's, and the recipient's filing may reveal how the letter was read. The lesson of this chapter is that archival arrangement is not the neutral packaging of evidence but part of it. The principle of provenance keeps together what one creator made, so that the researcher can see the creator's activity. Original order preserves the creator's own sense of how records related. The hierarchy of description allows researchers to move from the whole body of records down to the single item without losing their place. Finding aids and the meta-records behind them tell the researcher what the archive thought it held and how it came to hold it. Each of these structures can mislead. Provenance can be multiple or contested. Original order can be lost, reconstructed or invented. Descriptions reflect the interests of their authors. Catalogues can omit whole series. The response is not to distrust the archive but to read it as carefully as any other source, asking at each level who arranged these records, when, and why. A researcher who asks those questions before opening the first box will find more, misread less, and be able to explain to readers why the documents they cite are the ones that survived. The next chapter turns to the practical work of finding those boxes in the first place. Chapter 2: Discovery Before the Reading Room: Finding Records, Physical and Digital Most archival projects fail, when they fail, before anyone opens a box. They fail because the researcher went to the wrong archive, or the right archive at the wrong time, or asked for the wrong series, or spent a week reading material that had already been published in full. Discovery is the stage at which a project either acquires a coherent body of evidence or drifts into a collection of interesting fragments. It rewards the same habits as any other research design: define what you need, find out what exists, estimate what it will cost to see it, and decide in advance how you will know when you have enough. Discovery now happens in two worlds at once. There is the physical archive, with its boxes, reading-room hours and retrieval schedules. And there is a vast digital layer of catalogues, databases and scanned documents that makes some material instantly available and leaves the rest harder to see by contrast. A researcher needs to work both, and to understand how the digital layer shapes, and sometimes distorts, what seems to exist. Locating the records The first question is not where the documents are but who would have created them. If your question concerns a decision by a government, think through every body that would have produced a record of it: the ministry that proposed it, the ministry that paid for it, the cabinet or council that approved it, the embassy that reported reactions to it, the legislature that debated it, the courts that later ruled on it, the provincial authority that carried it out. If your question concerns a social movement, think of its leaders, its opponents, the police who watched it, the newspapers that reported it and the churches, unions or firms whose members joined it. Each creator implies an archive, and the principle of provenance, discussed in the previous chapter, means that each archive will hold that creator's records together. A worked example shows how this reasoning structures a search. Suppose a political scientist wants to explain why the British government decided to intervene militarily at Suez in 1956. The obvious starting point is the record of cabinet discussion, which at the National Archives in London sits in two series: the cabinet conclusions, recording what was decided, and the cabinet memoranda, the papers circulated to ministers beforehand. But the cabinet minutes of that crisis are notoriously thin on the most sensitive discussions, which took place in smaller groups of ministers and officials. The researcher therefore needs the Prime Minister's Office files, which hold the papers that passed across the prime minister's desk; the Foreign Office's general political correspondence, filed by country and subject; the Treasury's files on sterling, because the financial pressure on the pound shaped the decision to halt the operation; the defence ministries' planning papers; and, outside Britain, the published American diplomatic record and the French and Israeli archives, since the intervention was coordinated with both governments in ways that British officials were at pains to keep off paper. Each of those creators implies a series, and each series has its own filing logic and its own gaps. A search that began with a keyword query for "Suez" in a single catalogue would find some of these and miss the reasoning that connects them. Only then turn to catalogues. Most national archives maintain online catalogues that describe their holdings at least to the series level and often to the file level: the National Archives of the United Kingdom through its Discovery catalogue, which also indexes records held in many other British repositories; the National Archives and Records Administration in the United States through its online catalogue; and comparable systems in most European and many other states. Union catalogues bring together descriptions from many institutions. ArchiveGrid, maintained by OCLC, aggregates finding aids from thousands of American and other repositories. The Archives Portal Europe aggregates descriptions from national and regional archives across the continent. In Germany, the Kalliope union catalogue covers personal papers and manuscripts. These tools are the fastest way to locate the scattered papers of an individual or the records of a small organization. Catalogues miss a great deal, however, and three other routes often find more. The first is the footnotes of earlier scholarship. Reading the references of the best existing work on a topic, and noting every archival citation, produces a list of series that others have found useful, together with a sense of what they did not use. The second is published guides to sources, which many archives and scholarly societies produced in print before online catalogues existed and which often contain detailed commentary on the content and quality of series that the online catalogue describes in a line. The third is the archivist. Writing to an archive before visiting is not a courtesy but a research method. Archivists know their holdings in ways no catalogue captures: which uncatalogued accessions have arrived recently, which series were refiled under a different reference, which collections are closed for conservation, and which researchers have worked on related questions. A short, specific inquiry that explains the research question, names the series already identified, and asks whether anything else might be relevant will usually receive a useful reply. A vague inquiry asking what the archive has on a broad subject will not. The relationship matters beyond the single inquiry, because the archivist may later alert you to newly opened material or help resolve a problem with a citation. For records outside the major national systems, discovery can take longer. Regional, municipal, church, business, university and private archives often have partial catalogues or none online. Colonial and post-colonial records may be divided between a former imperial capital and the successor state, with different descriptive traditions and different levels of access in each. Records in states that have experienced war or regime change may have been moved, dispersed or captured. The records of the Iraqi Ba'ath Party, for example, were taken to the United States after the 2003 invasion and held there for years before being returned to Iraq, which made them available to some researchers and not others and raised questions about the ethics of using captured records at all. In such cases the discovery stage includes learning the custodial history of the records before trying to locate a particular file. Planning the visit A visit to a physical archive is expensive in time and money, and most researchers cannot return easily. Planning therefore pays for itself many times over. Begin with the practical rules, which vary more than newcomers expect. Many archives require registration and identification, and some require a letter of introduction from a university or a formal research permit from a government ministry. In some countries the permit process takes months. Opening hours may be short, and some archives close for extended periods in summer or for stock-taking. Records may be stored off site, requiring days of notice before they can be produced. There may be a limit on the number of items a reader can order per day, and orders may be fulfilled only at fixed times. Photography may be permitted freely, permitted for a fee, permitted only without flash or stand, or forbidden. Confirming these details in advance, and in writing where possible, avoids the familiar disaster of arriving to find that the key series is at a storage facility two hundred kilometres away. Where the rules allow, order the first documents in advance so that they are waiting on the first morning. Then plan the time in the reading room around a distinction between survey and study. In the survey phase, the aim is to look at every potentially relevant file quickly, noting what it contains, how it relates to other files, and whether it needs close reading. In the study phase, the aim is to read and record the files that matter. Researchers who begin by reading the first file in full often run out of time before discovering that the most important material was in a series they never reached. A survey of a collection's structure in the first days, even if it feels unproductive, is the best protection against that outcome. Budgeting is part of planning. A realistic estimate of reading speed varies enormously with the material: a researcher may survey a hundred typed files in a day but take all day over a single folder of eighteenth-century correspondence in an unfamiliar hand. Photography has changed the arithmetic, because it allows the researcher to capture material for reading later, but it has not abolished it. Chapter eight discusses the costs of photographing everything; the short version is that a photograph is not a reading, and material captured without being understood is often never understood. For researchers who cannot travel, alternatives exist. Many archives provide copies of documents on request for a fee, although this works only for material that can be identified precisely. Professional record agents and independent researchers will work in an archive on a client's behalf, a common arrangement in genealogical research and increasingly in academic work. Collaborations with scholars based near the archive can be productive for both sides, provided that credit and access are agreed openly. Each of these arrangements places another person between the researcher and the evidence, and the researcher should document who selected what, because that selection becomes part of the provenance of the evidence as used. The digital archive and its shadows A large share of archival research now begins, and sometimes ends, online. National archives have digitized popular series. Commercial publishers have scanned newspapers, parliamentary papers, pamphlets and government documents into subscription databases. Libraries and universities have created digital collections from their own holdings. The United States Central Intelligence Agency placed its declassified records search tool, known as CREST, online in January 2017, making available some thirteen million pages that had previously been searchable only on terminals at the National Archives in Maryland. The Wilson Center's Digital Archive gathers translated documents from the archives of many states on the Cold War and other topics. For many questions, the quantity of digitized material exceeds what any researcher could read in a lifetime. This abundance is real, and it has changed what can be done. It has also introduced biases that are easy to overlook because they are built into the search tools themselves. The historian Lara Putnam set these out in an influential 2016 article in the American Historical Review on "the transnational and the text-searchable". Her argument was that keyword search across digitized sources makes it easy to find connections across borders and to follow people and ideas across the world, which has enriched history, but that it also strips away the local context that researchers once acquired by reading whole runs of sources in a place. The researcher who finds a name in a searchable newspaper database sees the hit, not the paper, the page, the week, or the other stories that surrounded it. The researcher who travelled to read that newspaper would have absorbed all of those, and would have noticed what was absent as well as what was present. Several specific distortions follow. First, the choice of what to digitize is not random. It reflects funding, perceived demand, copyright, the condition of originals and the priorities of institutions, which tend to favour material in major languages, from major institutions, about well-known subjects. Records of the powerful are digitized before records of the governed; material in English before material in other languages; published before unpublished. A digital corpus that looks comprehensive may represent a narrow slice of the physical archive. Second, keyword search depends on machine-readable text, and much of that text was produced by optical character recognition applied to old print or typescript. Recognition errors are common in faded, damaged or unusually typeset material, and they are not evenly distributed: they concentrate in exactly the kinds of source that are already hardest to read. A search that returns no hits for a name does not show that the name does not appear. It shows that the name does not appear in the recognized text, which is a different claim. Handwritten material is usually not searchable at all unless someone has transcribed it or applied handwriting recognition, whose limits chapter five discusses. Third, digitized documents are often presented as individual items, stripped of the file and series around them. A scanned telegram in a database may carry a reference number but no indication of what came before and after it in the original file. The contextual reading that chapter one recommends is still possible, but only if the researcher reconstructs the file from its reference, which may require going back to the physical archive or to a microfilm copy. Fourth, access to commercial databases depends on institutional subscriptions, which vary widely. A scholar at a wealthy university may search collections that a scholar elsewhere, including in the countries whose history the collections record, cannot. That inequality shapes who can do certain kinds of research and whose interpretations dominate. None of this argues against digital sources. It argues for treating the database as a source with a provenance of its own. When using a digitized collection, a careful researcher records which database was searched, on what date, with which search terms, and what the collection claims to cover. They check a sample of hits against page images to see how reliable the recognized text is. They read some material in its full sequence rather than only through search hits. And when a digital search fails, they consider whether the failure reflects the absence of evidence or the limits of the tool. Designing for transparency Political science has in recent years developed an explicit discussion of transparency in qualitative and archival research, and the discussion is useful to historians as well. The core concern is that readers cannot evaluate claims based on archival evidence if they cannot see how the evidence was found and selected. A quantitative researcher reports the sample and the selection procedure. An archival researcher often reports only the documents quoted, leaving the reader to guess what else was read and what was set aside. Several practices address this. One is to keep, from the start of discovery, a research log recording which catalogues and series were searched, which files were requested and examined, and what each contained, including files that proved irrelevant. That log allows the researcher to state with confidence what the records do not show, which is often as important to an argument as what they do show. Another is to describe, in the published work, the archival base in terms of series and coverage rather than only individual citations, so that readers know whether the key claims rest on a complete run of records or a few surviving pieces. A third is the kind of annotation developed by the Qualitative Data Repository at Syracuse University, which allows authors to link passages in a published article to excerpts from the sources and to notes explaining how each source was interpreted. Not every project needs that level of apparatus, and not every archive permits the reproduction of its documents, but the underlying discipline of recording the path from question to evidence applies to all. Transparency also has a practical payoff that has nothing to do with readers. Archival projects last years. A researcher who returns to a question after a gap, or who is challenged by a reviewer, needs to know exactly what was searched and found. The notes made during discovery are the only reliable record. Memory is not. Discovery, then, is a design problem. It asks the researcher to think about who created the records that might answer the question, to find where those records now are, to plan how to see them within real limits of time and money, to use digital collections without being captured by their biases, and to record the whole process well enough to defend it. Once in the reading room, however, a different set of obligations begins: to the documents themselves, whose physical survival depends partly on how researchers treat them. That is the subject of the next chapter. Chapter 3: Handling the Past: Conservation Ethics for Researchers Every time a document is read, it is changed a little. Fingers leave oils and moisture. Pages flex along folds that are already weak. Light fades inks and dyes, and the damage is cumulative and cannot be reversed. A bound volume opened too far cracks at the spine. A brittle page turned carelessly breaks at the edge. Each of these events is small. Over decades and thousands of readers, they add up to the loss of the record. The archive exists in tension between two duties that cannot both be fully met: to preserve the documents for future readers and to make them available to present ones. Researchers usually encounter that tension as a list of reading-room rules: pencils only, no bags at the desk, use the foam supports, turn the pages from the corner. The rules make more sense, and are easier to follow well, once one understands what they protect against. And the principles behind them, developed by the professional field of conservation over the past century, turn out to be principles of good scholarship as well. They ask the handler to treat the object as evidence, to change it as little as possible, and to record whatever is done to it. A researcher who absorbs those principles handles documents better and reads them better. How documents decay Documents decay from within and from without. Internal decay arises from the materials themselves; external decay from the environment and from handling. Knowing which is at work in a given document helps a researcher understand both how to handle it and what its condition says about its history. Paper made before the middle of the nineteenth century was usually made from linen or cotton rags and, if kept dry and cool, can be remarkably stable. Much of the paper made after that point is not. The industrialization of papermaking brought in wood pulp, which contains lignin, and acidic sizing processes that together make paper produce acid as it ages. The acid breaks down the cellulose fibres, and the paper yellows and becomes brittle. The problem affects an enormous share of nineteenth- and twentieth-century records, including much of the government paper that political scientists and modern historians rely on, and it is the reason a file from the 1940s may be more fragile than a letter from the 1640s. Permanent, alkaline paper became common for records only in the late twentieth century. Inks have their own problems. Iron gall ink, made from iron salts and tannins from oak galls, was the standard writing ink in Europe for many centuries. It is dark and durable when fresh, but it is acidic and its iron catalyses reactions that attack the paper beneath. In severe cases, the letters themselves eat through the page, leaving a text like lace that breaks when handled. Documents with ink corrosion need particular care, and a researcher who sees brown haloes around letters or cracks along strokes should handle the page as little as possible and tell the archivist. Modern copying and communication technologies have produced some of the least stable records of all. Thermal fax paper, common in offices in the 1980s and early 1990s, fades to illegibility in light and heat. Carbon copies on thin onionskin tear easily and smudge. Early photocopies and diazo copies fade. Spirit duplicator and mimeograph copies, the standard way to circulate documents in many organizations and movements before cheap photocopying, fade unevenly. Pressure-sensitive tape, used over the decades to repair tears, yellows, loses its grip and leaves adhesive stains that migrate through the paper. Paper clips and staples rust and stain. A researcher working on the second half of the twentieth century will encounter all of these, often in the same file. Photographic materials are chemically complex and vulnerable. Cellulose nitrate film, used for many negatives and motion pictures until roughly the middle of the twentieth century, decomposes as it ages, becoming sticky and then powdery, and is highly flammable; archives store it separately and handle it under special rules. Cellulose acetate film, its "safety" replacement, suffers from a form of decay called vinegar syndrome, in which the film releases acetic acid, shrinks and warps; the vinegar smell is the warning sign. Prints may silver, fade, crack or stick to their enclosures. Chapter eight returns to photographs as sources; here the point is that they demand more careful handling than paper, including gloves. External threats include light, heat, humidity, pollutants, pests and mould. High humidity encourages mould and accelerates chemical decay; very low or fluctuating humidity makes materials brittle and causes parchment to cockle. Light damage is cumulative and permanent, which is why archives restrict exhibition and why sustained exposure under a lamp or camera light is not trivial for sensitive items. Mould is also a health hazard to readers, and a researcher who finds it should stop and report it rather than continue. Table 2 summarizes the threats a researcher is most likely to meet and the response each calls for. Table 2. Common forms of deterioration a researcher may encounter and how to respond. Material or problem Warning signs Researcher response Acidic wood-pulp paper Yellowing, brittle edges, crumbs Support fully, turn slowly, do not flex Iron gall ink corrosion Brown haloes, cracked or missing strokes Minimize handling, report to staff Thermal fax paper Faded grey text, curling Limit light exposure, request copy Old pressure-sensitive tape Yellow strips, sticky or lifting edges Do not remove or press, report it Acetate film Vinegar smell, warping, shrinkage Stop, report, request surrogate Mould Fuzzy growth, spots, musty odour Stop handling, isolate item, report What conservation ethics asks of researchers Conservation became a profession with explicit ethical standards over the twentieth century, and those standards are now codified by professional bodies. The American Institute for Conservation's Code of Ethics and Guidelines for Practice is one widely used statement; the International Council of Museums' Committee for Conservation adopted a widely cited terminology in 2008 distinguishing preventive conservation, which acts on the environment and handling to slow decay, from remedial conservation, which acts directly on an object to arrest damage, and from restoration, which acts on an object to improve its legibility or appearance. Across these statements, several principles recur, and each has a meaning for the researcher. The first is respect for the integrity of the object. A document is not merely a carrier of text but a physical artefact whose materials, marks, folds, damage and repairs are all potential evidence. A conservator is trained not to remove any of these without good reason and documentation. The researcher's version of this principle is to leave everything as found: not to remove paper clips, unfold letters that have been flattened by the archive, straighten bent corners, or clean marks. A rusted pin may show that two documents were attached by their creator. A fold pattern may show that a letter was sent in an envelope of a certain size, or sealed without one. A coffee ring may show that a report was actually read. The researcher who "tidies" a file destroys this evidence. The second is minimal intervention. Conservators aim to do the least that achieves the goal, because every treatment carries risk and alters the object. Researchers apply this principle by handling documents only as much as reading requires. That means keeping documents flat on the desk, using the book supports and weights the archive provides, turning pages by their corners or edges rather than sliding a finger under them, and never leaning on or writing over a document. It means asking staff to handle items that are fragile or oversize rather than wrestling with them. It means resisting the temptation to hold a page up to the window to read a watermark or a faint pencil note without permission, since that is exactly the kind of flexing and light exposure that damages brittle paper. It means pencils rather than pens, because a pen mark on a manuscript is permanent. The third principle is reversibility, or, in the more cautious formulation many conservators now prefer, retreatability. Conservators long aimed to use treatments that could be undone without harm to the object. Experience showed that few treatments are fully reversible, and the conservator Barbara Appelbaum and others have argued for the more modest goal of treatments that do not prevent future treatment. For researchers, the analogue is that nothing done to a document should close off future use: no marking, no annotation, no rearrangement, no removal of items for convenience. The fourth is documentation. Conservators record the condition of an object before, during and after treatment. Researchers rarely think of recording condition, but they should, in two senses. They should tell archive staff about damage they discover or cause, so that it can be treated and recorded. And they should note, in their own research records, features of the physical document that bear on its interpretation: the paper, the ink, the hand, annotations in other hands, damage, repairs, attachments. These notes are invaluable later, when the researcher is working from a photograph that no longer shows the pencil annotation that was barely visible in the reading room. A persistent misconception concerns gloves. Many people assume that archival documents should always be handled with white cotton gloves. For most paper documents the opposite is usually advised. Cotton gloves reduce dexterity and sensitivity, making it harder to turn thin or brittle pages without tearing them, and loose fibres can snag on damaged edges. The conservators Cathleen Baker and Randy Silverman made this case in a widely cited 2005 article in International Preservation News, and many archives now ask readers to work with clean, dry, bare hands for paper and to reserve gloves, usually nitrile, for photographs, film, metal objects and other materials that fingerprints damage directly. The broader point is that rules should be understood, not merely followed. A researcher who knows why gloves are used for photographs and not for paper will handle both better than one who wears gloves everywhere. Reading the object as evidence The ethics of conservation and the method of the researcher converge on a single habit: attention to the document as a thing. That attention yields evidence that no transcription can capture. Consider the paper itself. Watermarks and chain lines, visible when paper is examined with transmitted light, can date and place handmade paper, because paper mills used distinctive moulds whose marks have been catalogued; the great reference work by Charles-Moïse Briquet, published in 1907, catalogued many thousands of watermarks from European paper before 1600. Machine-made paper has less distinctive features but can still be dated by its composition and by printed letterheads, official forms and stationery conventions. A letter on the letterhead of an office that its writer had left months earlier poses a question worth asking. Consider the marks of use. Registry stamps, file numbers, docketing notes and routing slips record how a document moved through an organization. A minute sheet attached to a file may carry the initials of each official who read it, with dates and comments, allowing a researcher to reconstruct who saw what and when, often a central question in studies of decision-making. Underlining and marginal ticks show what readers found important. Erasures and pasted-over text show revision. The physical state of a file (how worn, how often refoldered, how many times renumbered) indicates how heavily it was used. Such marks are especially important in political science work that asks whether a leader received and read a particular warning or report, a question that transcriptions of the text cannot answer. Consider damage and repair. Fire damage to the edges of a set of records may date them to a known fire at the creating office. Water damage may show where records were stored. Old repairs may show that a document was already considered valuable at a particular time. And systematic removals (pages cut out, items withdrawn and replaced by a slip) are evidence of later intervention, sometimes by the creator, sometimes by later custodians, sometimes by security reviewers. The withdrawal slip, which chapter six discusses, is itself a document to be recorded. Digitization has made this attention both easier and harder. High-resolution images allow magnification and enhancement that reveal details invisible to the eye in the reading room. Multispectral imaging, which captures images under different wavelengths of light, has recovered erased and overwritten text from palimpsests and damaged manuscripts. At the same time, a scan presents the document as an image on a screen, flattened, cropped and colour-corrected, and loses the tactile information (weight, texture, thickness, the feel of a seal) that the physical object carries. A researcher who works from scans should know what the scan cannot show and consult the original when those features matter. Obligations beyond the object Conservation ethics concerns the physical object, but the researcher's obligations extend to the people the records concern. Archives hold information about individuals who never consented to its collection or its use: prisoners, patients, children in institutions, colonial subjects, informants, victims of state violence. Many such records are closed for a period set by law or by the archive's policy, often calculated from the date of the record or the likely lifespan of the individuals named. Data protection law, including the General Data Protection Regulation in the European Union, allows archival and historical research on personal data under safeguards, but it does not remove the researcher's responsibility to consider harm. That responsibility has particular force for communities whose records were created by others, often under coercion. In the United States, a group of Native American and non-Native archivists, librarians and scholars drafted the Protocols for Native American Archival Materials in 2006, arguing that tribal communities should have a voice in how records about them, especially culturally sensitive materials such as sacred knowledge or images of ceremonies, are described, accessed and used. The Society of American Archivists later endorsed the Protocols in 2018. Similar debates concern colonial records in many former imperial archives, records of enslaved people, and the records of political repression. A researcher using such material should know the relevant protocols and policies, consider whether identifying individuals serves the research, and weigh whether publishing certain images or details would cause harm to living people or communities. There is also an obligation to the archive as an institution. The American journalist Nicholson Baker's 2001 book Double Fold criticized libraries that had discarded original runs of newspapers after microfilming them, arguing that preservation reformatting had become an excuse for disposal and that the originals carried information the surrogates did not. His argument was contested by librarians, who pointed to space and budget constraints and to the real decay of newsprint, but it forced a useful debate about what is lost when an original is replaced. Researchers have a stake in these decisions because they are the people who know what the originals show. Reporting damage, supporting the archive's conservation work, citing its holdings accurately, and explaining in publications why the original mattered are all ways of contributing to the survival of the evidence on which the disciplines depend. The handling rules of the reading room, then, are the visible edge of a coherent ethic. Documents are physical objects that decay. Their physical features are evidence. Every handling is an intervention, and interventions should be minimal, recorded and respectful of the object's integrity. Records also concern people, and the obligation not to harm extends to them. These principles govern how a researcher treats a document. The next two chapters concern how to read it. Hashtags: #ArchivalResearch #HistoryResearch #PoliticalScienceResearch #ArchivalScience #Provenance #Paleography #DocumentPreservation #OriginalOrder #RespectDesFonds #FindingAids #ArchivalDiscovery #SourceCriticism #ChainOfCustody #DocumentAuthentication #DeclassifiedRecords #ArchivalSilences #DigitizationBias #HistoricalDocuments #ConservationEthics #ArchivalTransparency #ResearchArchives #DocumentAnalysis #HistoricalMethod #PoliticalHistory #FutureOfArchivalResearch

Download the Book (PDF): This capstone module consolidates the knowledge and skills of the programme into the integrated competence of the entry-level clinician. It addresses the most demanding aspects of applied practice: resolving complex comorbidity through precise differential diagnosis, delivering trauma-informed and crisis-responsive care, working safely with risk, and navigating the legal, ethical, and professional realities of clinical work. Throughout, the emphasis is on the disciplined integration of assessment, theory, evidence, culture, and ethics into formulations and treatment plans that can be articulated and defended before supervisors, colleagues, and review panels. The twelve units span advanced differential diagnosis, trauma and crisis intervention, substance use and dual diagnosis, advanced psychometrics and the integration of assessment data, qualitative clinical research, clinical risk assessment, legal and mandatory-reporting frameworks, transnational and displaced populations, clinical supervision, and professional identity and sustainable practice. The module culminates in a master case conceptualization that draws on every prior competence. Sensitive clinical material - trauma, risk of harm, substance use - is treated in a professional, safety-oriented register that foregrounds assessment, safety planning, referral, supervision, and scope-of-practice limits, and all case material is hypothetical. Unit 1 — Complex Co-morbidities and Differential Diagnosis Learning Outcomes • Explain the major mechanisms that generate comorbidity, including shared etiology, artefacts of classification, causal sequencing, and common risk factors, and justify why comorbidity is the norm rather than the exception in complex presentations. • Conduct a structured differential diagnosis, generating a ranked list of candidate diagnoses and systematically ruling competing possibilities in or out using confirming and disconfirming evidence. • Analyse patterns of symptom overlap across depressive, anxiety, trauma-related, and neurodevelopmental presentations, and distinguish overlapping surface features from the criteria that discriminate one disorder from another. • Apply DSM-5-TR and ICD-11 conventions for specifiers, provisional diagnoses, and diagnostic hierarchy to structure and communicate a defensible formulation. • Identify and counter the cognitive biases that corrupt diagnostic reasoning, including confirmation bias, anchoring, and premature closure, and build these safeguards into a differential diagnosis report. Key Concepts • Comorbidity — The co-occurrence of two or more distinct disorders in the same person within a defined period, which may reflect genuine separate conditions, shared underlying causes, or limitations in how our classification systems carve up psychopathology. • Differential diagnosis — The systematic reasoning process of generating a set of plausible candidate diagnoses that could account for a presentation and then progressively ruling them in or out until the most defensible explanation remains. • Diagnostic hierarchy — A set of rules within a classification system specifying that certain disorders take precedence over others, so that a condition is not diagnosed separately when its symptoms are better accounted for by a higher-order or more pervasive disorder. • Specifier — A standardised extension to a diagnosis that records clinically important features such as severity, course, or subtype, sharpening a broad category into a more precise and treatment-relevant description. • Provisional diagnosis — A diagnosis assigned when the clinician judges that criteria are likely to be met but full confirmation awaits further information, time, or corroboration, formally signalling diagnostic uncertainty rather than concealing it. • Confirmation bias — The tendency to seek, notice, and weight evidence that supports a favoured hypothesis while overlooking or discounting evidence that would contradict it, a pervasive threat to accurate diagnosis. • Premature closure — The error of settling on a diagnosis before it has been adequately verified and before reasonable alternatives have been excluded, thereby ending the reasoning process too soon. • Phenotypic overlap — The situation in which distinct disorders share observable symptoms, so that the same surface feature, such as poor concentration or disturbed sleep, can arise from several different underlying conditions. Why Comorbidity Is the Rule, Not the Exception Newcomers to diagnosis often imagine that a client arrives with one clean disorder that a skilled clinician simply identifies, much as a mechanic identifies a single faulty part. Real clinical presentations rarely behave this way. In routine practice, and especially in the complex cases that define advanced work, people present with clusters of difficulties that cross diagnostic boundaries. A person referred for low mood may also describe panic attacks, intrusive memories of a past assault, chronic difficulties with attention that predate the depression, and problematic alcohol use that has crept upward over the year. The question is not simply which disorder this person has, but how many conditions are genuinely present, how they relate to one another, and which of them are real and separate as opposed to being facets of a single underlying problem. This is the terrain of comorbidity, and understanding why it is so common is the first step toward reasoning through it rather than being overwhelmed by it. Comorbidity is best understood not as a single phenomenon but as the visible result of several distinct mechanisms, and disentangling them is central to formulation. The first mechanism is shared etiology: two disorders may co-occur because they arise from overlapping causes. Genetic studies of internalising disorders, for example, point to broad heritable vulnerabilities that are not specific to any single diagnosis but predispose a person to a whole family of related conditions, so that depression and generalised anxiety appear together far more often than chance would predict. When two categories draw on a common pool of risk, their co-occurrence is expected rather than surprising. Recognising shared etiology restrains the clinician from treating every additional diagnosis as an independent event demanding a separate causal story. A second mechanism is causal sequencing, in which one disorder raises the risk of another over time. A person with a long-standing anxiety disorder may, after years of avoidance, restricted life, and demoralisation, develop a secondary depressive episode; a person with untreated attention-deficit/hyperactivity disorder may accumulate academic failure, relationship strain, and self-critical beliefs that seed later mood and substance problems. Here the disorders are genuinely distinct, but they are linked in a chain, and the temporal order matters enormously for treatment, because addressing the primary condition may prevent or resolve the secondary one. A third mechanism is the presence of common environmental risk factors, such as childhood adversity, poverty, or chronic medical illness, which are non-specific and elevate the probability of many disorders at once, producing comorbidity through a shared external pathway rather than a shared biology. A fourth mechanism is the most humbling, because it concerns the limits of our own instruments. Some comorbidity is at least partly an artefact of how classification systems are constructed. The DSM-5-TR and ICD-11 carve the continuous, messy territory of human distress into discrete categories, and where the boundaries are drawn affects how much overlap appears. When two categories share several defining symptoms, a person with a single coherent problem may cross the threshold for both and be counted as comorbid, even though clinically there is one disorder wearing two labels. This does not mean comorbidity is unreal; genuine separate conditions certainly co-occur. It means the careful clinician always asks whether an apparent comorbidity reflects two true disorders, one disorder generating symptoms that mimic another, or an artefact of overlapping criteria. Holding these four mechanisms in mind, shared etiology, causal sequencing, common risk factors, and classification artefact, transforms comorbidity from a confusing pile-up of labels into a set of testable hypotheses about how a person's difficulties are organised. The Differential Diagnosis Process Differential diagnosis is the disciplined counterpart to comorbidity. Where comorbidity asks how difficulties combine, differential diagnosis asks which of several competing explanations best accounts for what the clinician observes. Borrowed from general medicine, the method rests on a simple but powerful discipline: before committing to a diagnosis, deliberately generate a list of alternative conditions that could produce a similar presentation, and then work through them systematically, gathering evidence that either supports or undermines each one. The goal is not merely to arrive at a label but to be able to say why that label, and not the plausible rivals, is the most defensible conclusion. This is what makes a formulation defence-ready: it can survive challenge because the alternatives have already been considered and addressed rather than ignored. The process typically unfolds in stages. It begins with a broad information-gathering phase in which the clinician takes a careful history of presenting symptoms, their onset, course, and context, alongside developmental, medical, family, and psychosocial background. From this material the clinician identifies the salient clinical features, the phenomena that most demand explanation, such as pervasive low mood, recurrent intrusive imagery, or lifelong inattention. The next stage is hypothesis generation: casting a deliberately wide net to produce candidate diagnoses, including conditions that may initially seem unlikely, precisely so that they are not missed. A common teaching maxim is to consider what must not be missed, meaning conditions that are dangerous or highly treatable, alongside what is most probable. Crucially, this stage must include non-psychiatric explanations. Thyroid dysfunction, anaemia, neurological conditions, medication side effects, and substance use can all produce psychological symptoms, and a differential that omits medical and substance-related causes is incomplete. Once a candidate list exists, the clinician moves to the discriminating phase, in which each hypothesis is tested against the evidence. This is where diagnostic criteria earn their keep. For each candidate, the clinician asks which features would be expected if this disorder were present, which features would be expected to be absent, and what the actual evidence shows. The presence of a pathognomonic or highly specific feature can rapidly raise one hypothesis; the absence of a required criterion can rule another out. The clinician also weighs course and onset, since the same cross-sectional picture can point to different disorders depending on its history. Throughout, the reasoning is probabilistic and iterative: hypotheses are ranked, the ranking is revised as new information arrives, and the clinician remains willing to reopen candidates that seemed excluded if fresh evidence warrants. The endpoint is a formulation that names the most likely diagnosis or diagnoses, explains the reasoning, and acknowledges residual uncertainty rather than papering over it. Ruling In and Ruling Out Competing Diagnoses The heart of differential diagnosis is the logic of ruling in and ruling out, and mastering it means understanding what kinds of evidence actually move a hypothesis. To rule a diagnosis in is to accumulate evidence that the specific criteria for that disorder are met and that the pattern coheres over time and context. To rule a diagnosis out is subtler and often more important, because it protects against the seductive pull of the first plausible answer. A diagnosis can be ruled out in several legitimate ways: a required criterion is definitively absent; the duration or onset does not fit; the symptoms are fully explained by another condition higher in the diagnostic hierarchy; or the presentation is better accounted for by the direct effects of a substance or medical condition. Each of these is a principled exclusion, not a hunch, and each can be articulated to a supervisor, an examiner, or a multidisciplinary team. A recurring difficulty is that many symptoms are non-specific, meaning they occur across numerous disorders and therefore have limited power to discriminate. Insomnia, fatigue, irritability, and poor concentration are present in depression, anxiety, post-traumatic stress disorder, ADHD, substance withdrawal, and several medical conditions. Because such symptoms are common to many hypotheses, their presence rules little in and their absence rules little out. The skilled diagnostician learns to prize discriminating features, the symptoms that are relatively specific to one condition and rare in its rivals. Recurrent intrusive re-experiencing of a traumatic event, tied to an identifiable trauma, points toward a trauma-related disorder in a way that generic distress does not. A pattern of inattention and impulsivity that is documented across settings and reaches back into early childhood points toward ADHD in a way that recent-onset concentration problems do not. Building a differential is largely the work of identifying which features carry discriminating weight and letting those, rather than the ubiquitous non-specific symptoms, drive the ranking. It is equally important to recognise when ruling out is premature or unsafe. In genuinely complex cases, the evidence may be insufficient to exclude a serious possibility, and the responsible action is not to force a decision but to hold the hypothesis open, gather corroborating information, and, where appropriate, arrange further assessment or referral. Collateral history from family, school, or previous clinicians frequently changes the picture, as does observing the presentation over time. This is why differential diagnosis is properly understood as a process rather than a single moment of judgement. A hypothesis is not truly ruled out until the clinician can state the specific basis for excluding it, and until that basis is available, honest uncertainty, recorded as such, is preferable to false confidence. Throughout, the clinician must work within their scope of practice, recognising that ruling out medical causes, for example, may require referral to a physician rather than an assumption made alone. Symptom Overlap Across Disorders Nowhere is differential reasoning more demanding than at the crossroads of depression, anxiety, trauma-related disorders, and ADHD, four families of presentation whose symptoms overlap extensively at the surface while differing in their underlying organisation. Consider concentration difficulty, a complaint that appears in all four. In a major depressive episode, impaired concentration typically arrives with the mood disturbance, is worse when mood is worst, and lifts as the episode remits; it is state-dependent and relatively recent. In generalised anxiety disorder, concentration is disrupted because the mind is repeatedly pulled into worry, and the person can often identify the anxious preoccupations that interrupt them. In post-traumatic stress disorder, attention is fragmented by hypervigilance and by intrusive re-experiencing, and it is bound to reminders of the trauma. In ADHD, inattention is trait-like and pervasive, present across settings and stretching back to childhood, independent of current mood or anxiety. The same word, concentration, thus points to four different mechanisms, and only the surrounding pattern, onset, course, and context, tells them apart. Sleep disturbance behaves similarly. Early-morning waking with diurnal mood variation is characteristic of certain depressive presentations; difficulty falling asleep because of racing anxious thoughts suggests an anxiety process; nightmares that replay a traumatic event and lead to fearful avoidance of sleep point toward PTSD; and a lifelong difficulty settling and a delayed sleep phase are common in ADHD. Irritability, restlessness, and fatigue are equally promiscuous across these categories. The lesson is not that these symptoms are useless but that they must be read in context. The discriminating questions concern the timeline and the accompanying features: When did this begin, and against what background? Does it fluctuate with mood, with worry, or with trauma reminders, or is it a stable lifelong trait? What else travels with it? A person whose inattention, restlessness, and impulsivity have been present since early childhood, across home and school, is telling a different story from a person whose identical-sounding symptoms emerged in the aftermath of an assault six months ago. Trauma-related presentations deserve particular care because they can masquerade as almost anything. The emotional numbing and loss of interest of PTSD can look like depression; the hyperarousal and exaggerated startle can look like an anxiety disorder; the concentration difficulties and restlessness can look like ADHD; and dissociative symptoms can be mistaken for psychosis by an inexperienced observer. What anchors a trauma-related diagnosis is the linkage of symptoms to an identifiable traumatic exposure and the presence of the specific re-experiencing and avoidance phenomena that define the category, which the DSM-5-TR and ICD-11 both require, though they organise the criteria somewhat differently. ICD-11 in particular draws a distinction between post-traumatic stress disorder and complex post-traumatic stress disorder, the latter adding disturbances in self-organisation such as difficulties with affect regulation, negative self-concept, and relationships, a distinction that matters greatly when a presentation might otherwise be filed under a personality or mood disorder. The clinician who does not ask about trauma will not find it, and its symptoms will be misattributed to the neighbouring categories. The overlap between anxiety and depression is so extensive and so common that it deserves separate comment. The two co-occur at high rates, share a broad genetic vulnerability, and present with substantial symptom overlap, so much so that debates continue about how sharply they should be separated at all. Both DSM-5-TR and ICD-11 provide ways of handling mixed presentations, and the clinician must decide whether the picture is best captured as a depressive disorder with anxious features, an anxiety disorder, both as genuine comorbid conditions, or a mixed presentation. The discriminating work involves establishing which cluster is primary in time and severity, whether the person meets full criteria for each condition independently, and how the symptoms cohere. Getting this right is not academic point-scoring; it shapes whether treatment prioritises behavioural activation, exposure, worry-focused work, or a combination, and it shapes the client's understanding of their own experience. Specifiers, Provisional Diagnoses, and Diagnostic Hierarchy A single diagnostic label is often too blunt to guide treatment, which is why modern classification systems attach specifiers to record clinically important detail. Specifiers capture features such as severity, whether mild, moderate, or severe; course, such as single episode versus recurrent, or in partial versus full remission; and descriptive subtypes, such as a depressive episode with anxious distress, with melancholic features, with psychotic features, or with peripartum onset. These additions convert a broad category into a precise clinical description. The difference between a first, mild depressive episode and a recurrent, severe episode with psychotic features is not captured by the word depression alone, yet it transforms prognosis, risk, and treatment. In complex comorbid cases, specifiers do much of the real work of communication, allowing a clinician to convey, in standardised language, exactly what kind of depression or anxiety is present and how it is unfolding. Provisional diagnosis is the formal mechanism for acting responsibly under uncertainty. When a clinician has strong reason to believe a disorder is present but cannot yet confirm every criterion, perhaps because the required duration has not elapsed, or because collateral information is pending, the diagnosis may be recorded as provisional. This is not a hedge or an evasion; it is an honest and standardised way of signalling that a working diagnosis is guiding care while confirmation is actively sought. Provisional diagnoses are especially valuable in early assessment, in crisis presentations where full history is unavailable, and in conditions such as PTSD or schizophrenia-spectrum disorders where duration criteria mean the picture must be watched over time. The alternative, forcing a premature definitive diagnosis to avoid appearing uncertain, is precisely the error that good diagnostic practice is designed to prevent. Recording uncertainty transparently is a mark of rigour, not of weakness. Diagnostic hierarchy provides the rules that prevent double-counting and impose order on overlapping categories. Classification systems contain exclusion criteria stating that a diagnosis should not be made if its symptoms are better explained by another, usually more pervasive, condition. The most familiar hierarchical principle is that symptoms attributable to the direct physiological effects of a substance or another medical condition are not diagnosed as an independent primary disorder; the depressive syndrome caused by hypothyroidism is coded as a depressive disorder due to another medical condition, not as major depressive disorder. Similarly, certain broad disorders subsume symptoms that would, in isolation, meet criteria for narrower ones. Understanding hierarchy stops the clinician from stacking up redundant labels for what is really one process, and it forces the disciplined question that anchors all differential diagnosis: is this a separate disorder, or is it better accounted for by something already on the list? At the same time, hierarchy must be applied thoughtfully, because both DSM-5-TR and ICD-11 have moved away from rigid rules that once suppressed genuine comorbidity, recognising that two real disorders often do coexist and both deserve to be named. Cognitive Bias, Confirmation, and Premature Closure The greatest threats to accurate diagnosis are not gaps in knowledge but predictable distortions in reasoning. Cognitive science has catalogued a set of biases that operate largely outside awareness and that corrupt clinical judgement in characteristic ways. Anchoring is the tendency to fix on an early impression, often the referral label or the first striking symptom, and to insufficiently adjust as new information arrives. A client referred as depression is at risk of being seen only through that lens, so that trauma, ADHD, or a medical cause is never seriously entertained. Confirmation bias then compounds the error: once a favoured hypothesis is in mind, the clinician unconsciously seeks and remembers evidence that fits it and discounts evidence that does not, asking questions that invite confirming answers and skimming past disconfirming details. The hypothesis becomes self-sealing, appearing ever more certain not because it is correct but because contrary evidence has been filtered out. Premature closure is the endpoint of these biases: the reasoning process is halted as soon as one plausible answer appears, before alternatives have been excluded and before the answer has been adequately tested. It is especially dangerous in comorbid presentations, where a satisfying explanation for part of the picture, such as identifying a depressive episode, can cause the clinician to stop looking and miss the intrusive trauma symptoms, the escalating alcohol use, or the lifelong inattention that also demand attention. Related pitfalls include the availability heuristic, whereby recently or vividly encountered diagnoses come to mind too readily, and search satisficing, whereby finding one abnormality ends the search for a second. Diagnostic overshadowing is a further trap in which a prominent condition, such as an intellectual disability or a severe mental illness, leads clinicians to attribute all new symptoms to it and overlook a separate, treatable problem. The remedy is not to exhort clinicians to try harder but to build structural safeguards into the diagnostic process. The single most protective habit is the deliberate generation of a differential, forcing the mind to hold several hypotheses at once rather than committing early to one. A second safeguard is to actively seek disconfirming evidence for the favoured diagnosis, asking not only what fits but what does not, and explicitly considering what else this could be. A third is the diagnostic time-out or deliberate pause before finalising, in which the clinician asks whether the diagnosis explains all the salient features, whether any must-not-miss conditions have been excluded, and whether the evidence would convince a skeptical colleague. Structured tools support these habits: validated screening and diagnostic instruments, standardised criteria checked against the actual presentation, and collateral information that counters the clinician's own selective attention. None of these eliminates bias entirely, but together they convert diagnosis from an intuitive leap into an auditable argument, which is exactly what a defence-ready differential diagnosis report must be. Overlapping symptom Depressive disorder Anxiety disorder PTSD / trauma-related ADHD Poor concentration State-dependent, recent, worst when mood is lowest Attention pulled into identifiable worry Fragmented by hypervigilance and trauma reminders Lifelong, pervasive across settings, mood-independent Sleep disturbance Early waking, diurnal mood variation Onset insomnia from racing thoughts Trauma-linked nightmares and avoidance of sleep Long-standing difficulty settling, delayed phase Irritability / restlessness Part of the mood syndrome, recent onset Tied to apprehension and tension Hyperarousal and exaggerated startle Trait-like motor and mental restlessness since childhood Onset and course Discrete episodes, may be recurrent Often chronic and fluctuating Follows an identifiable traumatic exposure Present before age twelve, chronic and pervasive Discriminating feature Pervasive low mood or anhedonia Excessive, hard-to-control worry or specific fears Intrusive re-experiencing and avoidance tied to trauma Documented cross-setting inattention and impulsivity Table 1.1 — Discriminating overlapping symptoms across four disorder families Practical / Real-World Example: Building a Differential from a Complex Presentation Consider an illustrative and entirely hypothetical case that will echo the kind of task the unit deliverable requires. A thirty-four-year-old man, whom we will call Daniel, is referred to a community mental health service by his family doctor with a referral note reading low mood and possible depression. In the first assessment he describes several months of flat mood, loss of interest, poor sleep, low energy, and difficulty concentrating at work, where his performance has slipped. Taken alone, this cluster reads as a depressive episode, and an incautious clinician might anchor on the referral label, confirm the depressive symptoms, and close the case there. The disciplined clinician instead treats depression as one hypothesis among several and deliberately widens the differential before narrowing it. Careful history-taking reveals a richer and more complicated picture. Daniel discloses, when asked directly and sensitively, that eighteen months ago he was involved in a serious road traffic collision, since which he has experienced recurrent vivid intrusive memories, nightmares, and a determined avoidance of driving and of the road where it happened; he startles violently at car horns and feels persistently on edge. He also reports that his alcohol intake has risen substantially over the past year, initially to help him sleep and blunt the intrusive images, and that he now drinks most evenings and feels shaky in the mornings. Collateral history from his partner, gathered with consent, adds that Daniel has struggled with disorganisation, forgetfulness, and restlessness for as long as she has known him, difficulties that predate the collision by many years and that his school reports apparently documented in childhood. What began as a simple referral for depression has become a genuinely complex, comorbid presentation demanding a structured differential. The clinician now lays out the candidate diagnoses explicitly. Major depressive disorder remains on the list, supported by the pervasive low mood, anhedonia, sleep and concentration disturbance, and functional decline. Post-traumatic stress disorder is a strong candidate given the identifiable trauma, the intrusive re-experiencing, the avoidance, and the hyperarousal, features that also explain much of the sleep disturbance and concentration difficulty that might otherwise be attributed solely to depression. An alcohol use disorder must be considered in its own right, given the escalating pattern, the morning tremor suggesting physiological dependence, and the fact that alcohol can both cause and mimic depressive and anxiety symptoms, meaning some of the low mood and poor sleep could be substance-related rather than a primary mood disorder. Adult ADHD enters the differential because of the lifelong, cross-setting inattention, disorganisation, and restlessness reported by his partner and apparently documented in childhood, a pattern that long predates both the trauma and the mood change. Finally, the clinician retains a medical hypothesis, recognising that thyroid dysfunction, other endocrine or neurological conditions, and medication effects can produce overlapping symptoms, and that these require exclusion through appropriate physical investigation and referral to the family doctor. Ruling in and ruling out now proceeds feature by feature. The trauma linkage, the specific re-experiencing, and the avoidance rule PTSD firmly in as a genuine condition rather than a facet of depression, because these phenomena are discriminating and are anchored to an identifiable event. The lifelong, pre-existing, cross-setting pattern of inattention and restlessness, reaching back to childhood and independent of current mood, supports a provisional diagnosis of adult ADHD, provisional because confirming it responsibly requires obtaining the childhood records and using validated instruments and collateral rather than relying on current self-report alone, which could be confounded by depression and trauma. The alcohol use disorder is ruled in on its own criteria, and its presence forces a careful judgement about how much of the depressive picture is primary and how much is substance-related, a question that may only be answered by reassessing mood after a period of reduced drinking. Major depression is retained but recorded with appropriate caution, since some of its apparent symptoms may be better accounted for by PTSD, by alcohol, or by the demoralisation that lifelong ADHD can produce. The medical hypotheses are addressed by arranging the appropriate investigations through the family doctor rather than assuming their absence. The resulting formulation is exactly the kind of defence-ready product the deliverable calls for. It names the conditions judged genuinely present, orders them by confidence and by their temporal and causal relationships, records the provisional status of the ADHD diagnosis and the pending medical exclusions, and, crucially, explains why at least four competing or alternative explanations were considered and how each was ruled in, ruled out, or held open. It resists premature closure by refusing to collapse Daniel's difficulties into the single referral label, and it resists confirmation bias by actively seeking the trauma, substance, developmental, and medical evidence that the initial hypothesis of depression would have left unexamined. It also specifies treatment implications that flow from the differential: the sequencing of trauma-focused work, alcohol reduction, mood monitoring, and ADHD assessment cannot be planned until the diagnostic relationships are clear. Practical / Real-World Example: When Overlap Misleads A second hypothetical case illustrates how symptom overlap can lead a clinician astray when the differential is neglected. A nineteen-year-old student, whom we will call Priya, is referred with a query of generalised anxiety disorder after describing restlessness, difficulty concentrating, irritability, and disturbed sleep, all of which fit the anxiety label at first glance. An anchored clinician might confirm generalised anxiety and begin worry-focused treatment. A structured differential, however, asks what else produces this exact cluster and interrogates the timeline. On closer questioning, Priya reports that these difficulties are not accompanied by the excessive, hard-to-control worry across multiple domains that defines generalised anxiety disorder; rather, her restlessness and inattention have been present since primary school, across every setting, and her recent distress stems from mounting academic failure that she attributes to an inability to organise and sustain attention. The discriminating features point away from an anxiety disorder and toward a long-standing neurodevelopmental condition presenting for the first time under the increased demands of higher education. This example demonstrates several of the unit's themes at once. It shows how a non-specific cluster of restlessness, poor concentration, and irritability can be claimed by multiple hypotheses, and how only the discriminating features, here the lifelong, pervasive, and mood-independent course, adjudicate between them. It shows the danger of anchoring on a referral label and the protective value of asking what else this could be before accepting the first plausible answer. It also illustrates the proper use of provisional diagnosis and further assessment: rather than declaring adult ADHD on the spot, the clinician would record a provisional formulation, seek collateral and developmental history and validated measures, and rule out that the presentation is better accounted for by anxiety, depression, a substance, or a medical cause. And it reinforces the theme that overlap between disorders is not a nuisance to be brushed aside but the central intellectual challenge of diagnosis, one that structured reasoning, rather than intuition alone, is designed to meet. When you construct your own differential diagnosis report, cases like Daniel's and Priya's are the template: generate widely, discriminate carefully, exclude explicitly, and record uncertainty honestly. Sample Activities and Assessments Sample Activity • Task: Working in pairs, take a single non-specific symptom supplied by the tutor, such as poor concentration or disturbed sleep, and map how it would present differently across depression, an anxiety disorder, PTSD, and ADHD, specifying the onset, course, and accompanying features that would discriminate each. • Expected output: A completed discrimination grid for the symptom, plus a short paragraph identifying which additional questions you would ask to tell the four possibilities apart. • Assessment criteria: Accurate use of discriminating rather than non-specific features, correct DSM-5-TR and ICD-11 framing, recognition that surface similarity does not imply shared mechanism, and clarity of clinical reasoning. Sample Assessment • Task: Given a detailed hypothetical complex case supplied in the assessment brief, produce a structured differential diagnosis, generating a ranked list of candidate diagnoses and including at least one medical and one substance-related hypothesis. • Expected output: A written differential of roughly 1,200 words that lists each candidate, states the confirming and disconfirming evidence, and reaches a reasoned ranking, using specifiers and provisional status where appropriate. • Assessment criteria: Breadth of the initial differential, principled ruling in and out with explicit criteria, correct application of diagnostic hierarchy, transparent handling of uncertainty, and no fabricated statistics or citations. Sample Assessment • Task: Prepare the unit deliverable, a differential diagnosis report resolving a complex simulated case, explicitly ruling out at least four alternative diagnoses. • Expected output: A defence-ready report that presents the case, lays out the full differential, documents the ruling in or out of each of at least four alternatives with the specific evidential basis, names the final diagnosis or diagnoses with specifiers and any provisional status, and identifies the cognitive biases you guarded against. • Assessment criteria: Systematic and auditable reasoning, at least four alternatives genuinely excluded on stated grounds, accurate DSM-5-TR and ICD-11 terminology, attention to scope of practice and referral, and demonstrable safeguards against confirmation bias and premature closure. Bringing the Diagnostic Reasoning Together This unit has moved diagnosis from the recognition of single labels to the disciplined navigation of complexity. You have seen that comorbidity is the normal condition of clinical work, generated by shared etiology, causal sequencing, common risk factors, and the artefacts of classification, and that distinguishing these mechanisms turns a confusing pile of diagnoses into a set of testable questions about how a person's difficulties are organised. You have worked through differential diagnosis as a staged process of generating candidates broadly, discriminating among them with the specific criteria of the DSM-5-TR and ICD-11, and ruling each in or out on principled grounds rather than intuition. You have examined how extensively symptoms overlap across depression, anxiety, trauma-related disorders, and ADHD, and you have learned to prize discriminating features over the non-specific complaints that point everywhere and nowhere. You have also seen that the machinery of classification, specifiers, provisional diagnoses, and diagnostic hierarchy, exists precisely to let clinicians describe complex presentations with the right degree of precision and the right acknowledgement of uncertainty. And you have confronted the cognitive biases, anchoring, confirmation bias, premature closure, and diagnostic overshadowing, that threaten every diagnostic decision, together with the structural safeguards that convert diagnosis into an auditable argument rather than an intuitive leap. The thread running through all of it, and the thread to carry into your differential diagnosis report, is that a defensible diagnosis is one whose alternatives have been considered and addressed. To resolve a complex case is not to name the first fitting label but to show, transparently and within your scope of practice, why that conclusion survives when the competing explanations have each been given a fair and explicit hearing. That habit of structured, self-critical reasoning is the foundation on which the remaining units, and your professional practice, will build. Unit 2 — Trauma-Informed Care and Crisis Intervention Learning Outcomes • Explain the neurobiology of the human stress response and use the concepts of the window of tolerance, hyperarousal, and hypoarousal to formulate a client's trauma-related presentation. • Differentiate post-traumatic stress disorder from complex post-traumatic stress disorder using ICD-11 criteria, and account for the developmental and relational conditions that shape each. • Apply the six SAMHSA principles of trauma-informed care and the cultural dimension to the design of a safe therapeutic environment and to the counselor's own conduct. • Justify a phase-oriented approach to trauma treatment using Herman's three phases, and defend the clinical rule that stabilization must precede any trauma processing. • Describe, at a conceptual level, the leading evidence-based trauma therapies and the indications for grounding, crisis stabilization, and referral to specialist care. Key Concepts • Psychological trauma — The lasting adverse response to an event or series of events experienced as physically or emotionally harmful or life-threatening, which overwhelms the person's ordinary capacity to cope and has enduring effects on functioning and well-being. • Window of tolerance — A term coined by Daniel Siegel for the optimal zone of physiological and emotional arousal within which a person can process experience, think clearly, and stay engaged; states above or below this band impair integrated functioning. • Hyperarousal — A state of excessive sympathetic nervous system activation above the window of tolerance, marked by anxiety, panic, hypervigilance, racing thoughts, anger, and a felt sense of threat even in safe conditions. • Hypoarousal — A state of dampened arousal below the window of tolerance, associated with numbing, dissociation, emotional flatness, disconnection, and a collapse of engagement, often mediated by the dorsal vagal branch of the parasympathetic system. • Complex post-traumatic stress disorder (CPTSD) — An ICD-11 diagnosis arising typically from prolonged or repeated trauma from which escape is difficult, comprising the core features of PTSD plus enduring disturbances in self-organization: affect dysregulation, negative self-concept, and difficulties in relationships. • Trauma-informed care — An organizing framework in which every part of a service assumes that clients may have trauma histories and responds by prioritizing physical and psychological safety, trustworthiness, choice, collaboration, and empowerment, while resisting re-traumatization. • Phase-oriented treatment — A sequenced model of trauma care, most influentially articulated by Judith Herman, that establishes safety and stabilization first, undertakes remembrance and mourning of the trauma only when the client is resourced, and moves finally toward reconnection with life and relationships. • Grounding — A set of practical, in-the-moment techniques that help a client re-anchor attention in present-moment sensory reality, thereby interrupting flashbacks, dissociation, or escalating arousal and widening the window of tolerance. The Neurobiology of the Stress Response To work responsibly with trauma, a counselor must first understand that the symptoms clients bring are not signs of weakness or of a broken personality but the predictable output of a nervous system doing exactly what evolution designed it to do under threat. When a person perceives danger, sensory information travels a fast, subcortical route through the thalamus to the amygdala, the brain's threat-detection hub. The amygdala can trigger a defensive cascade before the slower cortical route, which passes through the prefrontal cortex and hippocampus, has finished appraising whether the threat is real. This is why a survivor may find their heart pounding and their body braced to run before they have consciously registered what set them off. The system prioritizes speed over accuracy because, across evolutionary time, a false alarm was far cheaper than a missed genuine threat. The physiological engine of this response is the hypothalamic-pituitary-adrenal axis and the autonomic nervous system. On threat detection the sympathetic branch mobilizes the body for fight or flight: adrenaline and noradrenaline are released, heart rate and respiration accelerate, blood is shunted to the large muscles, and non-urgent functions such as digestion are suppressed. In parallel, the hypothalamus signals the pituitary and then the adrenal glands to release cortisol, a slower-acting stress hormone that sustains mobilization and helps the body recover. In a healthy stress response the threat passes, the parasympathetic system restores equilibrium, and cortisol returns to baseline. The difficulty in trauma is that this return to baseline is disrupted, so the body remains primed as though the danger were ongoing. When fight and flight are impossible or futile, a third, older defense can take over: immobilization, mediated by the dorsal vagal branch of the parasympathetic nervous system. Stephen Porges's polyvagal theory offers a useful conceptual map here, distinguishing a ventral vagal state of safe social engagement, a sympathetic state of mobilized defense, and a dorsal vagal state of shutdown or collapse. This last state helps explain the freeze and collapse responses, and the numbing and dissociation that many survivors describe. It is clinically vital because a client who appears calm, flat, or checked out may not be regulated at all; they may be in a profound state of hypoarousal that looks like composure but is in fact a defensive shutdown. Trauma also affects the balance between the amygdala and the regulatory structures that normally keep it in check. Under chronic stress the amygdala can become sensitized and hyper-responsive, while the prefrontal cortex, which supports appraisal, inhibition, and reasoning, is functionally less able to modulate it. The hippocampus, central to placing memories in time and context, can also be affected, which contributes to one of the defining oddities of traumatic memory: it is often stored not as a coherent narrative with a clear past-tense location but as fragmented sensory and emotional imprints that intrude into the present as though the event were happening now. This neurobiological account is not merely academic; it directly justifies why stabilization and regulation, rather than immediate storytelling, are the first tasks of trauma work. The Window of Tolerance and States of Arousal Daniel Siegel's concept of the window of tolerance gives counselors an intuitive and clinically powerful model for understanding trauma-related dysregulation. Picture a horizontal band representing the zone of arousal within which a person can stay present, think, feel, and relate without being overwhelmed. Inside this band the nervous system is regulated enough that the thinking brain and the feeling brain work together. Above the band lies hyperarousal, the domain of the mobilized sympathetic system: panic, rage, flooding, racing thoughts, and the driving urge to act. Below the band lies hypoarousal, the domain of dorsal vagal shutdown: numbness, emptiness, dissociation, foggy thinking, and a sense of being far away from oneself and others. Trauma tends to narrow the window of tolerance, so that experiences a non-traumatized person would find manageable push the survivor rapidly into one extreme or the other. A raised voice, a particular smell, a physical sensation, or a date on the calendar can serve as a trigger that catapults the person out of the window before they have any conscious sense of why. Once outside the window, higher-order processing is compromised: it is neurobiologically difficult to reflect, learn, or absorb reassurance while flooded or shut down. This is the single most important practical implication of the model. Much of the counselor's early work is not about content at all but about helping the client notice their arousal state and develop reliable ways to return to and widen the window. It is worth stressing that neither hyperarousal nor hypoarousal is a moral failing or a lack of effort, and neither is safely overridden by willpower. A client who dissociates in session is not being resistant or uncooperative; their nervous system has enacted an old protective strategy. Recognizing arousal states, naming them non-judgmentally, and responding with regulation rather than more demand is the essence of a trauma-informed stance. A counselor who tracks the client's window of tolerance in real time, watching for the subtle early signs of escalation or shutdown, can titrate the intensity of the work so that the client stays within a zone where genuine learning and integration are possible. PTSD and Complex PTSD in ICD-11 The eleventh revision of the International Classification of Diseases made an important conceptual advance by distinguishing two sibling diagnoses: post-traumatic stress disorder and complex post-traumatic stress disorder. This separation reflected decades of clinical observation that the sequelae of a single, time-limited trauma often differ from those of prolonged, repeated, and inescapable trauma, particularly when that trauma occurs in childhood or within relationships of dependency. Understanding this distinction is central to accurate formulation and to matching a client with an appropriate pathway of care. ICD-11 PTSD is defined by three core clusters that persist after exposure to an extremely threatening or horrific event. The first is re-experiencing the trauma in the present, through vivid intrusive memories, flashbacks, or nightmares that carry the emotional and sensory charge of the original event rather than being remembered as safely past. The second is deliberate avoidance of reminders, whether internal thoughts and feelings or external people, places, and situations connected to the event. The third is a persistent sense of heightened current threat, shown in exaggerated startle and hypervigilance. These features must last for several weeks and cause significant impairment in functioning. The ICD-11 formulation is deliberately narrower and more streamlined than some earlier definitions, foregrounding the features that most sharply distinguish the disorder. Complex PTSD includes all of the core PTSD features and adds three further clusters known collectively as disturbances in self-organization. The first is affect dysregulation: heightened emotional reactivity, difficulty calming, and sometimes emotional numbing or dissociation under stress. The second is a persistent negative self-concept, marked by pervasive beliefs of being diminished, defeated, or worthless, and by deep feelings of shame or guilt related to the trauma. The third is disturbances in relationships: persistent difficulty feeling close to others and sustaining relationships, or a tendency to avoid them altogether. CPTSD typically follows trauma that is prolonged or repeated and from which escape was difficult or impossible, such as childhood abuse, domestic violence, torture, or captivity, though the diagnosis rests on the symptom picture rather than on the specific event. The distinction matters clinically because the disturbances of self-organization in CPTSD often require a longer and more relationally attentive course of care, with particular emphasis on the earliest phase of building safety, self-regulation, and a workable therapeutic relationship. It also cautions against premature or intensive trauma processing with clients whose sense of self and capacity to regulate are fragile. Counselors should note that trauma exposure can also present alongside or be mistaken for other conditions such as depression, panic disorder, substance use disorders, or borderline personality features, and that careful differential assessment, ideally in collaboration with qualified diagnosticians, is part of responsible practice. The Principles of Trauma-Informed Care Trauma-informed care is not a specific therapy but a lens and a set of organizing commitments that shape how an entire service, and every individual within it, engages with people who may have trauma histories. The framework developed by the United States Substance Abuse and Mental Health Services Administration, known as SAMHSA, is the most widely used articulation. Its central insight is a shift in orientation from asking what is wrong with this person to asking what has happened to this person, and then arranging every point of contact so that it does not inadvertently reproduce the dynamics of the original trauma, which so often involved powerlessness, betrayal, and the loss of control. • Safety Clients and staff feel physically and psychologically safe. The physical environment, the predictability of routines, and the interpersonal conduct of staff all communicate that this is a place where the person will not be harmed, cornered, or surprised. • Trustworthiness and transparency Decisions are made with transparency, and expectations are clear, so that trust can be built and maintained. For survivors whose trust has been violated, consistency between what is promised and what is done is itself therapeutic. • Peer support and mutuality The experience of others who have lived through trauma is valued as a vehicle for building trust, establishing safety, and modelling recovery, reducing the isolation that trauma so often imposes. • Collaboration Power is shared and decisions are made with the client rather than for them, so that the therapeutic relationship itself becomes a corrective experience of partnership rather than domination. • Empowerment, voice, and choice The client's strengths are recognized and built upon, and genuine choice is offered at every practical opportunity, restoring the sense of agency that trauma strips away. • Cultural, historical, and gender responsiveness The service actively moves past stereotypes and biases, responds to the racial, ethnic, and cultural needs of those it serves, and recognizes historical and intergenerational trauma, so that safety and trust are meaningful across difference. These principles are mutually reinforcing rather than a checklist to be ticked. Choice without safety is hollow, and collaboration without transparency breeds suspicion. In practice, trauma-informed care shows up in small, concrete acts as much as in policy: telling a client in advance what an assessment will involve and that they may pause at any time; arranging a room so the client is not trapped in a corner and can see the door; asking permission before shifting to a sensitive topic; noticing and naming the client's arousal state; and being scrupulously reliable about time, confidentiality, and follow-through. The counselor's own regulated, attuned presence is itself a core intervention, because a survivor's nervous system reads the safety of the relationship long before it can reason about it. Phase-Oriented Treatment: Herman's Three Phases Judith Herman's phase-oriented model, first set out in her work on trauma and recovery, remains the organizing spine of contemporary trauma treatment. Its enduring contribution is the insistence that recovery unfolds in a sequence and that the order cannot safely be reversed. The three phases are not rigid stages passed through once and left behind; clients commonly move back and forth between them, and a return to earlier stabilization work is expected whenever safety wavers. But the logical priority is fixed: safety comes first, always. The first phase is safety and stabilization. Its tasks are to establish safety in the client's external life and internal world: reducing ongoing danger such as an abusive relationship or self-harm, stabilizing basic functioning and daily rhythms, building the therapeutic alliance, and helping the client develop reliable skills for regulating overwhelming states and staying within the window of tolerance. This is the phase in which grounding, emotion-regulation skills, psychoeducation about trauma, and the strengthening of internal and external resources are the work. For many clients, and especially those with complex trauma, this phase is not a brief preliminary but the greater part of the treatment, and there is no shame or failure in a course of therapy that remains here for a long time. The second phase is remembrance and mourning. Only when the client is sufficiently stabilized and resourced does the work turn toward the trauma itself, so that fragmented traumatic memory can be processed and integrated into an autobiographical narrative that is experienced as belonging to the past. Crucially, this phase also involves genuine grief: the mourning of what was lost, whether childhood, safety, relationships, years, or an assumptive world in which the person believed themselves safe. This is specialized work. It should be undertaken by clinicians trained in evidence-based trauma-processing modalities, at a titrated pace, and only on a foundation of established safety and regulation. This unit deliberately does not provide processing techniques, because attempting them without training and without a stabilization foundation risks re-traumatizing the client. The third phase is reconnection. Here the client, no longer wholly defined by the trauma, reengages with ordinary life: rebuilding relationships and trust, developing a renewed sense of self and purpose, and often finding meaning or a survivor mission. Herman describes a movement from being a victim to being a survivor to being, simply, a person living a life. Reconnection reminds the counselor that the goal of trauma work is not merely symptom reduction but the restoration of a full and connected existence, and it dignifies the whole enterprise by keeping that horizon in view. Phase Primary goal Representative tasks Counselor's stance Safety and stabilization Establish external and internal safety Reduce ongoing danger, build alliance, psychoeducation, grounding and emotion-regulation skills, strengthen resources Steady, predictable, resourcing; do not push into trauma content Remembrance and mourning Process and integrate traumatic memory; grieve losses Titrated, specialist-led trauma processing; construction of a coherent past-tense narrative; mourning of what was lost Attuned, paced, trained in a specific evidence-based modality Reconnection Reengage with life, relationships, and self Rebuild trust and relationships, renew identity and purpose, consolidate gains, relapse-prevention Collaborative, future-oriented, empowering Table 2.1 — Herman's three phases of trauma recovery and their principal tasks Evidence-Based Trauma Therapies at a Conceptual Level Several structured therapies have accumulated substantial evidence for treating PTSD, and a counselor in training should understand what each is for at a conceptual level, even where delivering them requires specialist training and supervision. The overarching point is that these are specialized, second-phase interventions. They presuppose a foundation of safety and stabilization, and they are described here so that a counselor can recognize them, explain them to clients, and make appropriate referrals, not so that they can be improvised. The following account is conceptual and deliberately omits any step-by-step processing instructions. Trauma-focused cognitive behavioral therapy, or TF-CBT, is a structured, components-based treatment developed particularly for children, adolescents, and their caregivers. It integrates psychoeducation, relaxation and affect-regulation skills, cognitive coping, gradual and supported engagement with the trauma narrative, and work with caregivers to support the young person. Its front-loaded emphasis on skills and stabilization before any narrative work embodies the phase-oriented logic within a single protocol. Eye movement desensitization and reprocessing, or EMDR, developed by Francine Shapiro, is an eight-phase therapy in which the client attends to distressing memories while engaging in bilateral stimulation, most often guided side-to-side eye movements. It is theorized to facilitate the adaptive reprocessing of traumatic memories so that they lose their disturbing charge. EMDR, too, includes explicit preparation and stabilization phases and requires formal training and adherence to its protocol. Prolonged exposure, or PE, is a form of cognitive behavioral therapy in which the client, at a carefully managed pace and within a strong therapeutic alliance, engages with trauma memories through imaginal exposure and gradually approaches safe but avoided real-world situations through in-vivo exposure. Its mechanism is understood in terms of emotional processing and the extinction of learned fear responses, allowing the person to learn that reminders are not themselves dangerous. Because exposure work can be intensely activating, it depends on solid preparation and appropriate case selection. Cognitive processing therapy, or CPT, focuses on the meanings the person has made of the trauma. It helps clients identify and examine stuck points, the rigid and distressing beliefs about safety, trust, power, esteem, and intimacy that trauma so often installs, and to develop more balanced and accurate appraisals. CPT can be delivered with or without a written trauma account and is well suited to survivors whose distress is heavily bound up in self-blame and shattered assumptions. Across all four modalities, the shared clinical wisdom is unambiguous: processing is powerful but is safe only when it rests on stabilization, is delivered by trained clinicians, and is matched to a client who is ready. Crisis Intervention, Stabilization, and Grounding Crisis intervention is short-term, focused help aimed at restoring a person to at least their pre-crisis level of functioning when acute distress has temporarily overwhelmed their coping resources. In trauma work it comes into play when a client is flooded, dissociating, in the grip of a flashback, or otherwise pushed sharply out of their window of tolerance. The counselor's immediate goals are to ensure safety, to reduce arousal, and to help the client re-anchor in the present, not to explore or interpret the traumatic material. A widely taught frame, psychological first aid, captures the priorities well: establish safety and a sense of calm, foster connectedness and self-efficacy, and link the person to practical support and further help, while doing no harm. Grounding techniques are the practical core of in-session stabilization. Their purpose is to interrupt escalation or shutdown by redirecting attention to concrete, present-moment reality, thereby signalling to the nervous system that the danger is not here and now. Sensory grounding invites the client to notice what they can see, hear, and touch, for example naming several objects in the room, feeling the chair beneath them, or holding something cool or textured. Orientation reminds the client of the present time and place and of their own safety: their name, the date, where they are, and that the traumatic event is not happening now. Breath and body strategies, such as slow exhalation-lengthened breathing or pressing the feet into the floor, engage the parasympathetic system and support regulation. These are skills to be practiced collaboratively while the client is regulated, so that they are available when needed, rather than introduced for the first time in a crisis. • Sensory anchoring Naming what can be seen, heard, and physically felt in the room to draw attention out of the memory and into present reality. • Orientation to the present Confirming name, date, place, and the fact of current safety to distinguish now from the remembered then. • Grounded breathing Slow breathing with a lengthened exhale to recruit the parasympathetic system and lower arousal. • Physical grounding Feeling the feet on the floor, the weight of the body in the chair, or contact with a textured object to re-establish embodiment. Crisis work also encompasses risk. Trauma is associated with elevated risk of suicide and self-harm, and any acute crisis must include a calm, direct assessment of safety. Consistent with a trauma-informed stance, this means asking clearly and without alarm about thoughts of suicide or self-harm, collaborating on a safety plan that identifies warning signs, coping strategies, sources of support, and means of reducing access to methods of harm, and knowing the pathways for urgent escalation. A counselor must work within their scope of practice: recognizing when a presentation exceeds their competence or the safety a given setting can provide, and referring promptly to specialist trauma services, medical care, or emergency services. Detailed risk assessment and safety planning are addressed as their own topic elsewhere in the programme; the essential point here is that stabilization and safety, not exploration of the trauma, are the priorities in any crisis. Practical / Real-World Example: Stabilization Before Processing Consider a hypothetical client, Maria, a woman in her mid-thirties referred after a serious road traffic collision several months earlier. She reports vivid intrusive images of the crash, nightmares, avoidance of driving and of the road where it happened, poor sleep, and a constant sense of being on edge, startling at sudden sounds. Her presentation is consistent with the picture of PTSD, and she arrives hoping to talk through the accident in detail so she can put it behind her. A counselor working in a phase-oriented, trauma-informed way does not begin there. Doing so would risk pushing Maria straight out of her window of tolerance into flooding, reinforcing the sense that the memory is unbearable and that she cannot cope with it. Instead the early sessions focus on safety and stabilization. The counselor explains, in plain language, how the stress response and traumatic memory work, which helps Maria understand that her symptoms are a normal reaction rather than a sign that she is losing her mind. Together they establish predictable session structure, map Maria's triggers and early warning signs of escalation, and practice grounding and breathing skills while she is calm, so that these become reliable tools. They identify her existing strengths and supports, and stabilize sleep and daily routines. Only once Maria can notice rising arousal and bring herself back within her window, and once a solid alliance is in place, would the question of trauma processing arise, and at that point the counselor would consider whether a referral to a clinician trained in a specific evidence-based modality such as EMDR or prolonged exposure is the appropriate next step. The clinical logic is that competence with regulation is the ground on which any later processing must stand. Practical / Real-World Example: Complex Trauma and the Primacy of Safety Consider a second hypothetical client, David, a man in his forties with a history of prolonged childhood abuse who has never had continuous safety in relationships. He presents with chronic emotional dysregulation that swings between overwhelming distress and periods of numb dissociation, a deep-seated belief that he is worthless and to blame, and a pattern of relationships that either feel dangerously close or are avoided altogether. He also reports occasional passive thoughts that life is not worth living. His presentation aligns with complex PTSD, and it would be a serious error to treat it as though a few sessions of memory processing could resolve it. For David, the safety and stabilization phase is the substance of the work for a considerable time, and this is entirely appropriate rather than a sign of slow progress. The counselor prioritizes a reliable, transparent, and unhurried relationship, understanding that the therapeutic alliance is itself the corrective experience for someone whose trust was repeatedly betrayed. Early work attends to present-day safety, including a collaborative safety plan for his passive suicidal thoughts and clear pathways for escalation, and to building skills for regulating his oscillation between hyperarousal and hypoarousal. The counselor names arousal states without judgment, works within the window of tolerance, and is careful never to demand that David revisit the abuse before he is resourced and ready. Given the complexity and risk, the counselor also coordinates with other professionals, seeks supervision, and considers referral to a specialist complex-trauma service. David's case illustrates the unit's central discipline: with complex trauma above all, safety is not a preliminary to be rushed through but the foundation on which everything else depends. Building Toward the Deliverable: A Multi-Phase Treatment Protocol The deliverable this unit builds toward is a multi-phase trauma treatment protocol that establishes safety before any trauma processing. Drawing the strands together, such a protocol is organized around Herman's three phases and infused throughout with the SAMHSA principles. Its first and most detailed phase specifies how safety and stabilization will be established: how the therapeutic alliance is built, how psychoeducation about the neurobiology of trauma is delivered, how triggers and window-of-tolerance signs are mapped, which grounding and regulation skills are taught, how risk is assessed and a safety plan maintained, and, critically, what explicit readiness criteria must be met before any movement toward processing. The protocol then names, at a conceptual level only, how second-phase processing would be approached: by referral to or delivery by a clinician trained in an appropriate evidence-based modality, at a titrated pace, with continuous monitoring of the client's arousal and a standing agreement to return to stabilization whenever safety wavers. It concludes with a reconnection phase oriented toward rebuilding relationships, identity, and meaning, and toward relapse prevention. Throughout, the protocol makes the counselor's scope of practice explicit and identifies the referral pathways to specialist trauma services and emergency care. A well-constructed protocol is defensible precisely because it can articulate, at every step, why stabilization precedes processing and how the client's safety is protected. Sample Activities and Assessments Sample Activity • Task: Working from a provided hypothetical case vignette, map the client's presentation onto the window of tolerance. Identify specific signs of hyperarousal and hypoarousal, list plausible triggers, and select three grounding or regulation strategies matched to the client's states. • Expected output: A one-page annotated diagram of the client's window of tolerance plus a short rationale of roughly 400 words. • Assessment criteria: Accuracy in distinguishing hyperarousal from hypoarousal; appropriateness and justification of the chosen strategies; clarity that no trauma-processing techniques are proposed at this stage. Sample Assessment • Task: Draft a first-phase safety and stabilization plan for a hypothetical client presenting with complex PTSD, applying the SAMHSA principles and Herman's phase model, and specifying explicit readiness criteria that must be met before any trauma processing is considered. • Expected output: A structured protocol of roughly 1,200 words covering alliance-building, psychoeducation, regulation skills, risk and safety planning, scope-of-practice limits, and referral pathways. • Assessment criteria: Fidelity to trauma-informed principles; sound clinical reasoning for stabilization before processing; correct use of ICD-11 terminology; appropriate identification of referral and escalation points; absence of any processing how-to. Summary This unit has grounded trauma-informed practice in the neurobiology of the stress response, showing how the amygdala, the HPA axis, and the autonomic nervous system produce the very symptoms clients bring, and how the window of tolerance model translates that biology into a practical guide for pacing the work. It distinguished PTSD from complex PTSD in ICD-11, set out the six SAMHSA principles together with cultural responsiveness, and used Herman's three phases to establish the field's central discipline: safety and stabilization before remembrance and mourning, and reconnection as the horizon. It surveyed the leading evidence-based therapies at a conceptual level and detailed crisis intervention and grounding, while insisting throughout on scope-of-practice limits and referral to specialist care. The consistent thread, and the foundation of the multi-phase protocol the module builds toward, is that in trauma work safety always comes first. Hashtags: #AdvancedClinicalFormulationsEthicsAndAppliedPractice #ClinicalFormulation #AppliedClinicalPractice #ClinicalEthics #DifferentialDiagnosis #Comorbidity #DSM5TR #ICD11 #TraumaInformedCare #ComplexPTSD #PTSD #WindowOfTolerance #CrisisIntervention #SafetyPlanning #RiskAssessment #Psychometrics #ClinicalAssessment #DualDiagnosis #SubstanceUse #ClinicalSupervision #ScopeOfPractice #MandatoryReporting #ProfessionalEthics #CaseConceptualization #FutureOfClinicalPractice

Download the Book (PDF): This module develops the advanced competences required to conceive, structure, write, and successfully publish original research in high-ranking international journals. It treats scholarly publishing not as a clerical afterthought to research but as an integral scientific practice in its own right, governed by its own logic of argument, evidence, ethics, and audience. Across twelve linked units the module moves from the architecture of the publishing ecosystem, through the craft of each manuscript section, to the strategic and ethical decisions that determine whether excellent research is read, cited, and built upon. Each unit combines a rigorous theoretical foundation with worked, real-world examples and assessment tasks that accumulate into a coherent publication portfolio. The units are sequenced to mirror the lifecycle of a manuscript: understanding where to publish and why; building the skeleton of the argument; synthesising prior work; framing the contribution; defending method and data; interpreting findings; observing the ethics of authorship; maximising discoverability; surviving peer review; and managing the legal, financial, and editorial mechanics of submission. Read together, they form a complete apprenticeship in high-impact publishing for the independent researcher. Unit 1 — The High-Impact Publishing Ecosystem Learning Outcomes • Evaluate the principal bibliometric indicators used to appraise journals and researchers, including the Journal Impact Factor, CiteScore, SJR, SNIP, and the h-index, distinguishing what each metric actually measures and where it misleads. • Critique the structural hierarchies of contemporary academic publishing, including indexing databases, quartile stratification, and the commercial and scholarly interests that shape them. • Analyse competing peer-review models and editorial expectations, and appraise the governance frameworks (COPE, ICMJE, DORA, CRediT) that regulate research integrity and responsible metric use. • Diagnose predatory and deceptive publishing practices using recognised screening resources such as DOAJ, Cabells, and Think-Check-Submit, and defend a reasoned judgement about a venue. • Construct a comparative journal analysis report that identifies three defensible target journals for a defined manuscript, integrating aims and scope, time-to-publication, metrics, and author guidelines. Key Concepts • Bibliometrics — The quantitative study of publications and their citation relationships, used to describe the productivity, reach, and apparent influence of authors, articles, journals, institutions, and whole fields. Bibliometrics is descriptive by nature; it becomes evaluative only when institutions decide to attach consequences to the numbers. • Indexing database — A curated, structured collection of bibliographic records and citation links, such as Scopus or Web of Science, that determines which journals are visible, discoverable, and countable. Because most metrics are computed from a database's own citation graph, inclusion in an index is a precondition for a journal to be measured at all. • Journal Impact Factor (JIF) — A ratio published in Clarivate's Journal Citation Reports that divides citations received in a given year by the number of citable items a journal published in the two preceding years. It is a mean of a highly skewed distribution and describes a journal, not any individual article within it. • Quartile — A ranking band that places a journal in the top (Q1), upper-middle (Q2), lower-middle (Q3), or bottom (Q4) quarter of the journals in a subject category, ordered by a chosen metric. Quartiles are relative to a category and to a database, so the same journal can occupy different quartiles in different systems. • DORA — The San Francisco Declaration on Research Assessment, a 2012 statement signed by individuals and organisations that commits them to stop using journal-level metrics such as the JIF as a proxy for the quality of individual articles or the merit of researchers, and to assess research on its own content. • Predatory publishing — A business model in which a journal or publisher solicits article-processing charges while failing to deliver the editorial and peer-review services that legitimate scholarly publishing implies. The term covers a spectrum from outright fraud to negligent quality control rather than a single clean category. • Article-processing charge (APC) — A fee paid, usually by the author, their institution, or a funder, to make an article openly available at the point of publication. APCs finance much of open-access publishing and are a legitimate mechanism, but they also create the commercial incentive that predatory operators exploit. • Time-to-publication — The elapsed time across the editorial pipeline, commonly decomposed into time from submission to first decision, submission to acceptance, and acceptance to online publication. It is a practical selection criterion that trades off against selectivity, visibility, and fit. Framing the Ecosystem: Why Structure Precedes Strategy For an early-career researcher, the decision about where to submit a manuscript can feel like a matter of taste or ambition. In reality it is a decision made inside a dense institutional system with its own history, its own commercial logic, and its own rules of visibility. Before any strategy can be rational, the researcher has to see the architecture clearly: who counts publications, how they count them, which journals are even eligible to be counted, and what interests are served by the counting. This unit builds that map. It treats the publishing ecosystem not as a neutral pipeline through which good work flows to the top, but as a constructed set of markets, databases, and reputational signals that reward some behaviours and penalise others, sometimes for reasons that have little to do with scholarly merit. The central tension running through the whole unit is the gap between what metrics claim to measure and what institutions use them for. Almost every widely used indicator was designed for a modest, specific purpose, typically to help librarians decide which journals to subscribe to. Over four decades these librarian-facing tools drifted into becoming instruments of individual assessment, promotion, hiring, and national research funding. The mismatch between the original design and the eventual use is the source of most of the pathologies discussed here, from citation gaming to predatory journals to the quiet distortion of what research gets done at all. Understanding the ecosystem therefore means understanding both the tools and the drift. A doctoral researcher needs this literacy for two reasons. The first is defensive: to avoid predatory venues, to read a journal's metrics without being deceived by them, and to protect a nascent publication record from decisions that look attractive but damage a reputation. The second is strategic and constructive: to select venues that genuinely match a piece of work, to time submissions realistically, and to build a coherent body of published work that a future appointment or grant panel can read as a trajectory rather than a scatter of disconnected outputs. The comparative journal analysis report that this unit builds toward is the concrete instrument through which that literacy is exercised. Indexing Databases and the Manufacture of Visibility Nothing in modern academic publishing is measured until it has been indexed. Indexing databases are the gatekeepers that decide which journals are visible to the systems that compute reputation, and their editorial policies quietly shape the incentives of the entire field. The two databases that dominate research evaluation are Clarivate's Web of Science and Elsevier's Scopus. Both are selective, proprietary, and curated by editorial teams that apply inclusion criteria and, importantly, that can remove journals when they fall short. A journal that is not indexed in at least one of these can publish excellent work and yet remain effectively invisible to the machinery of assessment. Web of Science is the older system, built on the Science Citation Index created by Eugene Garfield in the 1960s. Its historical core is the set of journals in the Science Citation Index Expanded, the Social Sciences Citation Index, and the Arts and Humanities Citation Index, to which Clarivate later added the Emerging Sources Citation Index as a broader, more inclusive tier. Web of Science has traditionally positioned itself as the more selective and curated index, and the Journal Impact Factor is computed from its citation data and published annually in the Journal Citation Reports. Scopus, launched by Elsevier in 2004, is generally larger in journal coverage, indexes more titles from outside the traditional Anglophone core, and supplies the citation data behind CiteScore, SJR, and SNIP. The difference in coverage matters more than it first appears. Because metrics are computed from a database's own citation graph, a journal's measured performance depends on which database is doing the measuring. A citation only counts if it comes from a source the database indexes; work published in books, conference proceedings, or non-indexed regional journals may be invisible to the calculation even when it is intellectually central to a field. This structural fact systematically disadvantages disciplines whose scholarly communication runs through monographs and edited volumes rather than journal articles, which is to say much of the humanities and large parts of the qualitative social sciences. A researcher must therefore know not only a journal's numbers but the database those numbers came from, and must treat cross-database comparison with caution. It is worth naming the alternatives that sit outside this proprietary duopoly. Google Scholar indexes far more broadly, including preprints, theses, and grey literature, and computes its own h-index-based journal rankings; its openness is also its weakness, because its citation counts are inflated by material that the curated databases exclude and are trivially gamed. More recent open infrastructure, including OpenAlex, Crossref, and the initiatives grouped under the Initiative for Open Citations, is attempting to build a transparent, freely reusable citation graph that does not depend on a single vendor. For an evaluator committed to responsible metrics, the transparency and reproducibility of the underlying data source is itself a criterion of quality, and the movement toward open citation data is a direct response to the closed, unauditable nature of the incumbent systems. The Metric Zoo: What Each Indicator Actually Measures The proliferation of bibliometric indicators can be bewildering, and the confusion is not accidental; each metric was created to fix a perceived flaw in an earlier one, and the accumulation has left researchers facing a crowded field of numbers that look interchangeable but are not. The disciplined way to read them is to ask three questions of each metric: what is the unit of analysis, what is the citation window, and how is the distribution handled. Answering those questions dissolves most of the apparent equivalence and reveals what a given number can and cannot support. The Journal Impact Factor is the oldest and most consequential. It divides the citations received in one year by the citable items published in the two preceding years, producing a mean number of citations per recent article. Two structural features are essential to understand. First, the denominator excludes some document types, such as editorials and news items, that can nonetheless attract citations counted in the numerator, which introduces a well-known asymmetry that journals can exploit. Second, and more fundamentally, citation distributions are extremely skewed: a small number of articles collect most of the citations while the majority collect few or none. Because the JIF is a mean of a skewed distribution, it is a poor predictor of how any individual article will perform, and using it to judge a single paper commits a basic statistical error. CiteScore, published by Scopus, was designed as a more transparent competitor. It uses a longer citation window, counting citations over a multi-year period to documents published across that same window, and it applies a broader and more symmetrical definition of what counts in both numerator and denominator, which reduces the editorial-document asymmetry that afflicts the JIF. Because it draws on the larger Scopus corpus and is calculated with a publicly stated method, CiteScore values for a given journal often differ noticeably from its JIF, and neither is wrong; they are simply different measurements taken with different instruments over different windows. SJR, the SCImago Journal Rank, imports an idea from network science: not all citations are worth the same. Modelled on the same intuition as Google's PageRank algorithm, SJR weights each incoming citation by the prestige of the citing journal, so that a citation from a highly regarded venue contributes more than one from an obscure title. It also normalises for field, which allows more meaningful comparison across disciplines that have very different citation densities. SNIP, the Source Normalized Impact per Paper, takes a different route to the same fairness problem: it corrects raw citation counts for the citation potential of the field, dividing by the frequency with which papers in that subject area cite at all, so that a journal in a low-citing field is not unfairly penalised against one in a high-citing field. Both SJR and SNIP are, in effect, attempts to make citation counts comparable across the uneven terrain of the disciplines. The h-index is different in kind because its native unit is the individual researcher, not the journal, although it is sometimes applied to journals and groups. An author has an h-index of h if h of their papers have each been cited at least h times. Its appeal is that it combines productivity and impact into a single number and resists distortion by one runaway hit or by a long tail of uncited work. Its weaknesses are equally clear: it can only rise and never fall, so it favours long careers over young ones; it is strongly field-dependent, because citation norms differ; and it is sensitive to which database computes it, since Scopus, Web of Science, and Google Scholar will each return a different value for the same person. Variants such as the g-index and the m-quotient were proposed to patch specific weaknesses, but none escapes the underlying fact that a single number cannot capture the shape of a body of work. Metric Unit of analysis Primary source What it captures Key limitation JIF Journal Web of Science (JCR) Mean citations to recent articles over a two-year window Mean of a skewed distribution; denominator asymmetry; database-bound CiteScore Journal Scopus Mean citations over a longer, symmetrical multi-year window Different window and corpus make it non-comparable with JIF SJR Journal Scopus (SCImago) Prestige-weighted, field-normalised citation influence Weighting model is opaque to most users; still journal-level SNIP Journal Scopus (CWTS) Citations corrected for the citation potential of the field Complex to interpret; sensitive to field delineation h-index Author (or journal) Multiple (differs by database) Joint measure of productivity and sustained citation Cannot fall; favours long careers; field- and database-dependent Table 1.1 — Principal journal and author metrics compared by unit of analysis, source, and characteristic limitation. Quartiles, Categories, and the Politics of Ranking Because raw metric values are hard to interpret in isolation and impossible to compare across fields, evaluators lean heavily on quartiles. A quartile assignment ranks all the journals within a defined subject category by a chosen metric and divides them into four bands, so that Q1 contains the top quarter and Q4 the bottom. The appeal is obvious: saying a journal is Q1 in its field is more intuitive than quoting a bare CiteScore, and it appears to control for the differing citation cultures of different disciplines. Much of contemporary hiring and promotion practice, especially outside the United States, runs on quartile thresholds, and in some national systems a Q1 publication is worth materially more than a Q2 one. The apparent simplicity conceals several traps. The first is that quartile depends entirely on the category a journal is placed in, and many journals sit in more than one category. A journal can be Q1 in a broad, low-competition category and Q3 in a narrow, prestigious one, and it is standard practice for publishers to advertise the most flattering assignment. A careful reader always asks which category and which metric produced a quartile claim. The second trap is that quartile is database-relative: Scopus and Web of Science define categories differently and rank with different metrics, so a journal can be Q1 in one system and Q2 in another without any change in its actual performance. The third is the boundary problem: a journal ranked just above or just below a quartile cut-off is treated as categorically different from its near-neighbour, even though the underlying difference in the metric may be negligible and statistically meaningless. There is also a deeper conceptual objection. Ranking imposes a single linear order on journals that in fact serve different intellectual functions. A specialised methods journal, a broad flagship, and a regional venue that anchors a national scholarly community are not competitors on one scale; they do different work. Reducing them to a quartile flattens that diversity and pushes authors toward a narrow band of high-ranking generalist venues, which in turn concentrates prestige, raises rejection rates, and lengthens queues. The quartile system, in other words, is not merely a description of the ecosystem; it actively reshapes it by steering author behaviour, and a strategically literate researcher understands that they are one of the agents whose choices the system is trying to direct. Peer-Review Models and Editorial Expectations Metrics describe the outputs of publishing, but peer review is the process that is supposed to justify the reputations metrics track. It is essential to see that peer review is not one thing. Several distinct models coexist, they distribute information and accountability differently, and each carries characteristic strengths and failure modes that a submitting author should weigh alongside a journal's numbers. In single-anonymised review, the historical default in many fields, the reviewers know the authors' identities but the authors do not know the reviewers'. This protects reviewers from retaliation and lets them write candidly, but it also permits bias, whether against unfamiliar institutions, less prestigious countries, or particular authors, to operate unchecked. Double-anonymised review conceals both parties from each other and was widely adopted to reduce such bias; its limitation is that true anonymity is hard to maintain in small specialised fields where writing style, self-citation, and topic make authorship guessable. Open review, in which identities are known to all and sometimes the full referee reports and author responses are published alongside the article, aims for transparency and accountability; critics worry it discourages junior reviewers from criticising senior authors and can make it harder to recruit referees at all. Two more recent developments deserve attention because they are reshaping expectations. Registered Reports invert the usual sequence: a study's introduction and methods are peer-reviewed and provisionally accepted before the data are collected, so that acceptance depends on the quality of the question and design rather than on whether the results turn out to be positive or exciting. This format directly attacks publication bias and the file-drawer problem and is spreading from psychology into the biomedical and social sciences. Separately, the growth of preprint servers such as arXiv, bioRxiv, medRxiv, and SSRN has decoupled the dissemination of findings from their certification, so that work circulates and is cited before, and sometimes independently of, formal review. A researcher today publishes into a layered system in which a preprint, a peer-reviewed version of record, and a set of post-publication commentaries may all coexist. Editorial boards sit above the review process and embody a journal's standards. When appraising a venue, the composition and behaviour of its editorial board is a strong quality signal. A credible board lists identifiable scholars with verifiable affiliations who are genuinely active in the field and who, if contacted, would confirm their involvement. Warning signs include boards padded with names that do not appear elsewhere in the literature, editors listed without their knowledge, a single individual serving as editor across implausibly many unrelated journals, or no editorial board at all. Editorial expectations also express themselves through the specificity of a journal's instructions: a serious journal states its scope precisely, describes its review process, discloses its policies on data availability, authorship, and conflicts of interest, and adheres to recognised standards rather than inventing its own. Governance, Integrity, and the Responsible-Metrics Movement Because the incentives created by metrics are so powerful, the ecosystem has developed a layer of governance intended to keep publishing honest and to discipline the use of the numbers. A doctoral researcher should be able to name the principal frameworks and say what each governs, because these standards increasingly define what a defensible publication and a defensible evaluation look like. The Committee on Publication Ethics, COPE, is a membership organisation that issues guidelines and flowcharts for handling misconduct: plagiarism, data fabrication, authorship disputes, redundant publication, and undisclosed conflicts of interest. Membership of COPE, and visible adherence to its flowcharts, is a mark of a serious publisher. The International Committee of Medical Journal Editors, ICMJE, though rooted in biomedicine, produces recommendations that are influential far beyond it, most notably its widely used four-part definition of authorship, which requires substantial contribution, drafting or revising, final approval, and accountability. These criteria are the standard against which honorary and ghost authorship are judged. The CRediT taxonomy, the Contributor Roles Taxonomy, addresses a related problem by replacing the crude binary of author-or-not with a structured list of fourteen contribution roles, from conceptualisation and methodology to data curation, writing of the original draft, and supervision. By allowing each contributor's specific role to be stated, CRediT makes credit more granular and more honest and helps resolve the disputes that a single ordered author list cannot. ORCID, the Open Researcher and Contributor ID, provides the persistent digital identifier that ties a real person to their outputs across name changes, institutional moves, and the many people who share a common name; it is now a routine requirement at submission and the backbone of reliable author-level metrics. Standing above these operational standards is the responsible-metrics movement, whose central document is the San Francisco Declaration on Research Assessment, DORA. DORA's core recommendation is stark: do not use journal-based metrics such as the Journal Impact Factor as a surrogate for the quality of individual research articles, and do not use them to make hiring, promotion, or funding decisions about individuals. It asks institutions to assess research on its own merits and its content. The Leiden Manifesto, published in 2015, complements DORA with ten principles for the responsible use of metrics, insisting among other things that quantitative evaluation should support but not supplant expert judgement, that metrics be aligned with the mission of the unit being assessed, and that indicators be kept transparent and regularly scrutinised for the behaviour they induce. The Metric Tide report added the influential concept of responsible metrics, defined through the dimensions of robustness, humility, transparency, diversity, and reflexivity. It is important to grasp why this movement is not merely a matter of principle. Metrics are reflexive: once a number becomes a target, people optimise for the number, and the number stops measuring what it once did. This is a version of Goodhart's law, and its concrete manifestations are everywhere in publishing, from coercive citation, where editors pressure authors to add citations to the journal, to citation cartels, to the salami-slicing of results across many thin papers to inflate output counts. Responsible-metrics governance exists precisely because the naive use of indicators corrupts the behaviour the indicators were meant to observe. A researcher who understands this reflexivity can read metrics as evidence rather than as instructions, and can defend their own choices to a panel that may not yet have internalised the same sophistication. Predatory Publishing and the Screening Toolkit The commercial engine that drives much of the ecosystem also created a parasitic shadow economy. When open-access publishing tied revenue directly to the volume of accepted articles through article-processing charges, it created an incentive to accept rather than to select, and a class of predatory publishers emerged to exploit it. These operators solicit submissions with flattering emails, promise rapid publication, charge a fee, and deliver little or no genuine peer review. The damage is not only to the deceived author, whose work is buried in a disreputable venue and whose reputation suffers by association, but to the scholarly record itself, which becomes polluted with unvetted and sometimes fraudulent material that nonetheless carries the surface appearance of peer-reviewed science. The first influential attempt to name and list these venues was maintained by the librarian Jeffrey Beall, whose blog catalogued what he called potential, possible, or probable predatory publishers. Beall's List raised awareness and gave the phenomenon a vocabulary, but it also drew sustained criticism: the criteria were applied by one person, appeals were difficult, the list conflated genuine predators with merely young or non-Anglophone journals, and it was vulnerable to the charge of penalising publishers from the Global South whose practices differed from Anglo-American norms without being dishonest. The list was taken down in 2017. Its history is a cautionary lesson that blacklisting is itself an act of power that can be exercised badly, and that a binary predatory-or-not judgement is often too crude for a spectrum that runs from fraud through incompetence to mere unfamiliarity. Contemporary practice therefore relies on a toolkit rather than a single list, combining inclusion-based whitelists with more nuanced evaluative services. The Directory of Open Access Journals, DOAJ, is a community-curated whitelist that indexes open-access journals meeting a published set of quality criteria; inclusion in DOAJ is a positive signal, and its Seal marks journals that meet higher standards of best practice. On the commercial side, Cabells operates two complementary products: a Journalytics database of vetted journals and a Predatory Reports service that documents, against explicit and disclosed criteria, why a given journal has been flagged, which addresses the transparency problem that dogged Beall's List. Above all, the Think-Check-Submit checklist gives authors a simple, memorable procedure: do you know this journal and have you read its articles, can you verify its editorial board and contact details and indexing claims, and is the submission process what a legitimate journal would run. The point of the modern approach is that no single source is authoritative; a defensible judgement triangulates several. Some warning signs are reliable enough to internalise. Unsolicited flattering invitations to submit, especially outside one's specialism, are a classic lure. Promises of peer review completed in a few days are incompatible with genuine review. A journal title that closely mimics a well-known established one, a publisher address that cannot be verified, invented or misrepresented impact metrics such as a fabricated impact factor from an unrecognised body, a scope so broad that it accepts anything, and demands for payment that are unclear until after acceptance all point the same way. None of these is individually conclusive, and legitimate young journals may trip one of them innocently, which is exactly why the discipline is to gather several signals and weigh them rather than to react to any one. Strategic Journal Selection as Disciplined Reasoning With the architecture in view, journal selection can be reframed from a guess into a reasoned matching problem. The goal is to find the venue where a specific manuscript will be read by the right audience, reviewed fairly and improved, published in a defensible timeframe, and positioned to contribute to a coherent scholarly record. Several dimensions have to be balanced simultaneously, and they frequently pull against one another, which is why selection is a judgement rather than a lookup. Fit is the first and most decisive dimension, and it is where most avoidable rejections originate. Editors desk-reject manuscripts that fall outside their journal's aims and scope before review even begins, so a paper aimed at the wrong venue wastes months regardless of its quality. Reading a journal's recent tables of contents, not merely its scope statement, is the surest test of fit: if work resembling one's own has appeared there, the audience exists and the editors have signalled their interest. Fit also has a rhetorical dimension, since a paper often needs to be framed to speak to the conversation a particular journal hosts, and the same findings may be told differently for a methods journal, a disciplinary flagship, or an applied venue. Impact and selectivity form the second axis, and here the responsible-metrics literacy developed earlier becomes practical. The naive strategy of always aiming at the highest-JIF journal in reach is usually a mistake: it maximises expected delay and rejection while ignoring fit and readership. A more defensible approach treats metrics as one input among several, uses field-normalised indicators such as SJR or SNIP when comparing across subfields, checks the quartile in the correct category and database, and asks not merely how prestigious a journal is but whether its readership is the community the work needs to reach. Prestige that comes at the cost of speaking to the wrong audience is often a poor trade. Time-to-publication is the third axis and is frequently underweighted by inexperienced authors, who discover its importance only when a competitor publishes first or a funding deadline passes. It decomposes into the wait for a first decision, the total time to acceptance including revision cycles, and the lag from acceptance to online availability. These figures vary enormously between journals, and some venues now publish their own median times, while third-party datasets and colleagues' experience fill the gaps. Speed trades against selectivity and against thoroughness of review, so the right target depends on the work: a time-sensitive result argues for a faster venue even at some cost in prestige, whereas a career-defining study may justify a longer queue at a more selective one. Two further considerations complete the picture. Access and cost have become first-order strategic questions rather than administrative afterthoughts: whether a journal is subscription-based, fully open access with an APC, or hybrid; whether the author's funder mandates open access, as many now do through initiatives such as Plan S; and whether the institution holds a transformative or read-and-publish agreement that covers the APC. A high APC at a venue the author cannot fund is simply not an option, however attractive its metrics. Finally, ethical and integrity screening runs through every choice: the selected venue must be legitimately indexed, must adhere to recognised standards such as those of COPE, and must survive the Think-Check-Submit test. A strategically optimal but predatory venue is a contradiction, because publishing there destroys the value the strategy was meant to create. Practical and Real-World Examples Example one: reading two metrics for the same journal and refusing to be deceived. Suppose a researcher is comparing two candidate journals and finds that Journal A advertises a higher Journal Impact Factor while Journal B advertises a higher CiteScore. The naive reaction is to treat this as a contradiction and to pick whichever number is larger. The disciplined reaction is to recognise that the two figures are measurements taken with different instruments: the JIF uses a two-year window and the Web of Science corpus, while CiteScore uses a longer window and the larger Scopus corpus, so the two journals are not even being measured on the same scale. The researcher instead looks up both journals in a single database, compares their quartiles within the same subject category, checks a field-normalised indicator such as SNIP to control for differing citation densities, and only then reads the recent contents of each to judge fit. The lesson is that a metric is meaningless without its provenance, and that the appearance of a contradiction usually signals that two different things are being compared. This is exactly the reasoning that turns a raw number into evidence and protects the author from being steered by whichever figure a publisher chose to advertise. Example two: triaging a suspicious invitation. A doctoral candidate in environmental science receives a warmly worded email inviting a submission to a journal whose title closely resembles a well-known society journal, promising peer review within seven days and prominent open-access visibility for a stated fee. Applying the screening toolkit rather than reacting to flattery, the candidate runs the Think-Check-Submit sequence. The Think step asks whether the journal is known; it is not, and its articles are unfamiliar. The Check step is decisive: the journal is absent from DOAJ, the claimed indexing in Scopus cannot be confirmed against the source, several editorial-board members listed on the site do not mention the role on their own institutional pages and one appears on the boards of a dozen unrelated journals, and the advertised impact metric turns out to come from an unrecognised body rather than from Clarivate or Scopus. A cross-check against Cabells Predatory Reports documents the journal against explicit criteria. The seven-day review promise, incompatible with genuine refereeing, confirms the pattern. The candidate declines. The example matters because it shows that the judgement is reached not from a single blacklist but by triangulating several independent signals, which is both more reliable and more defensible than any one source and more just to the occasional legitimate young journal that trips a single criterion innocently. Sample Activities and Assessments Sample Activity • Task: Choose one journal that has published work close to your own doctoral research and build a one-page metric provenance profile for it. Record its JIF (with year), CiteScore, SJR, SNIP, and its quartile in each subject category to which it is assigned, noting for every figure which database produced it and over what citation window. • Expected output: A single annotated table plus a short paragraph explaining any apparent discrepancies between the metrics and identifying which indicator you would cite to a field-normalised evaluation panel and why. • Assessment criteria: accuracy of the figures and their sources; correct attribution of each metric to its database and window; and the quality of the reasoning about why the numbers differ, rather than mere reproduction of the numbers. Sample Activity • Task: Take one invitation-to-submit email (real or supplied) and evaluate the named journal against the full Think-Check-Submit checklist, cross-referencing DOAJ and, where possible, Cabells, and inspecting the editorial board and indexing claims independently. • Expected output: A completed checklist with a one-sentence verdict for each item and a final reasoned recommendation to submit, investigate further, or decline, supported by at least three independent signals. • Assessment criteria: use of multiple independent sources rather than a single list; correct identification of genuine warning signs versus innocent ones; and a proportionate, defensible final judgement. Sample Assessment • Task: Produce the unit's capstone deliverable — a comparative journal analysis report that identifies three target journals for a defined manuscript of your own and defends the selection. • Expected output: A structured report of roughly 1,500 to 2,000 words containing a comparison table across aims and scope, average time-to-publication (submission-to-first-decision and acceptance-to-publication where available), the full metric set with database provenance and quartile, access model and APC, and a summary of each journal's author guidelines, followed by a ranked recommendation with an explicit rationale. • Assessment criteria: demonstrable fit between the manuscript and each chosen venue; correct, provenance-aware use of metrics consistent with DORA and responsible-metrics principles; explicit legitimacy screening of each venue; a realistic treatment of time and cost trade-offs; and a recommendation that is argued rather than asserted. Taken together, the strands of this unit form a single competence: the ability to see the publishing ecosystem as a constructed system of databases, metrics, review models, governance frameworks, and commercial incentives, and to move within it deliberately rather than reactively. The metrics are tools with known limits, the databases are gatekeepers with known biases, peer review is a plural set of practices with known failure modes, and the governance frameworks are the field's own attempt to keep the incentives honest. The comparative journal analysis report is where all of this is exercised at once, forcing the researcher to convert abstract literacy into a defensible, documented decision about where their own work belongs. That capacity for reasoned, transparent, integrity-anchored judgement, rather than the memorisation of any particular number, is the durable outcome this unit is built to produce. Unit 2 — Deconstructing Manuscript Architecture Learning Outcomes • Analyse the IMRAD framework as a rhetorical architecture rather than a fixed template, identifying how each section performs a distinct argumentative function. • Apply Swales's Create-a-Research-Space (CARS) model to construct an introduction that establishes a research territory, exposes a gap, and occupies that gap persuasively. • Evaluate the hourglass model of macro-structure and diagnose where a draft manuscript loses logical or thematic coherence. • Construct paragraph-level progression using topic sentences, given-new information ordering, and explicit signposting to achieve cohesion across sections. • Design a full manuscript outline and structural blueprint that maps core arguments, evidence placement, literature integration, and paragraph progression for a target journal. Key Concepts • IMRAD — The canonical macro-structure of the empirical research article (Introduction, Methods, Results, and Discussion), understood not as a filing system but as a sequence of rhetorical moves that together reconstruct the logic of inquiry for a critical reader. • Hourglass Model — A heuristic describing the ideal flow of scope across a manuscript: broad at the opening (general problem and field context), narrowing to the specific study through Methods and Results, then widening again in the Discussion to reconnect findings with the broader field. • CARS Model — John Swales's Create-a-Research-Space framework for analysing research article introductions as three recurrent rhetorical moves: establishing a territory, establishing a niche (identifying a gap), and occupying the niche (announcing the present work). • Signposting — The deliberate use of metadiscourse (transitions, forecasting statements, and framing markers) to make the structure of an argument visible, so that readers can anticipate where the text is going and how each part relates to the whole. • Given-New Contract — A principle of information structure in which each sentence and paragraph opens with information already known or established (the given) before introducing new material (the new), producing a chain of cohesion that guides the reader forward without cognitive friction. • Thematic Progression — The patterned way that the themes (sentence-initial topics) of successive sentences relate to one another across a paragraph, forming linear, constant, or derived progressions that determine whether a passage reads as coherent or disjointed. • Cohesion and Coherence — Cohesion is the surface-level connective tissue between sentences (lexical repetition, reference, conjunction); coherence is the deeper conceptual unity that makes a text feel logically whole. Strong writing engineers both simultaneously. • Move Analysis — A genre-analytic method that segments a text into functional units (moves) and their constituent steps, revealing the communicative purpose each stretch of prose serves and exposing conventions that experienced readers and reviewers expect. Why Architecture Precedes Prose Doctoral writers frequently treat manuscript structure as an administrative afterthought: a set of labelled containers into which finished paragraphs are dropped once the science is done. This inversion is the single most common cause of manuscripts that are individually well-written yet collectively incoherent. A research article is not a report of what happened in the laboratory in chronological order; it is a reconstructed argument engineered for a sceptical expert reader who will grant assent only if each claim is positioned where its supporting evidence and its warrant are simultaneously available. Architecture, in this sense, is not the vessel that holds the argument but the argument itself made spatial. To deconstruct manuscript architecture is therefore to learn to see the invisible logic that top-tier reviewers read for, often below the level of conscious articulation. The IMRAD framework became the dominant structure of experimental science across the twentieth century precisely because it externalises the scientific method. Each of its four movements answers a question the reader is entitled to ask in a fixed order: What problem are you addressing and why should I care (Introduction)? How, exactly, did you generate your evidence, such that I could scrutinise or reproduce it (Methods)? What did you find, stripped of interpretation (Results)? And what does it mean, both for the specific question and for the field (Discussion)? When writers understand these as questions rather than headings, they stop asking what belongs in the Discussion and start asking whether they have yet earned the reader's willingness to hear an interpretation at all. The sections are load-bearing walls; moving content between them is not cosmetic but structural. It is worth stressing that IMRAD is a convention, not a law of nature, and its stability is a feature rather than a constraint. Because reviewers, editors, and readers have internalised the same architecture, a manuscript that respects it lowers the cognitive cost of evaluation: the reader knows where to look for the sample size, where to find the limitations, and where the novelty claim will be defended. Deviations are permissible and sometimes required, but they should be deliberate, motivated by the rhetorical needs of the specific argument rather than by inattention. The competent author learns the grammar of the genre so thoroughly that departures from it become expressive choices rather than errors. The Hourglass Model and the Management of Scope The most durable heuristic for the macro-structure of a research article is the hourglass. At the top, the Introduction begins broad, situating the study within a general problem that a wide readership recognises as important. It then narrows steadily, funnelling from the general field to the specific unresolved question and finally to the precise aim or hypothesis of the present study. The narrow neck of the hourglass corresponds to the Methods and Results: here the scope is at its tightest, concerned only with this study, these participants or specimens, these measurements, these outcomes. The Discussion then reopens the aperture, moving from the specific findings back out to their implications for the field, and ultimately to the broad significance with which the piece began. The symmetry is not decorative; it signals to the reader that the study has kept its promises, returning to answer the very question it opened with. The value of the hourglass as a diagnostic tool becomes clear when a manuscript fails. A common pathology is the Introduction that stays broad for too long, never narrowing to a specific gap, so that the reader reaches the aim without understanding why this particular study, rather than a dozen adjacent ones, was necessary. An equally frequent failure is the top-heavy Discussion that widens too fast, leaping from a modest, bounded result to sweeping claims about clinical practice or public policy that the neck of the hourglass never supported. Reading a draft against the hourglass, the author can literally sketch the changing width of scope paragraph by paragraph and identify where the shape is wrong: where it bulges, pinches prematurely, or fails to close. Crucially, the two halves of the hourglass must mirror each other. Every element introduced on the way down should be revisited on the way up. If the Introduction frames the study as addressing a controversy about mechanism, the Discussion must return to that controversy and state what the results contribute to it. If a secondary aim is announced at the neck, it cannot silently vanish. This principle of promissory symmetry is one of the clearest markers reviewers use to distinguish a disciplined manuscript from a loose one, and it is the reason that outlining the Introduction and Discussion together, before drafting either, produces more coherent articles than writing them sequentially. The Introduction as Rhetorical Space: Swales's CARS Model No section is more consequential, or more frequently mishandled, than the Introduction, and no analytic tool illuminates it more sharply than John Swales's Create-a-Research-Space model. Developed through close genre analysis of research article introductions across disciplines, CARS describes the introduction as the accomplishment of three rhetorical moves. Move 1, establishing a territory, situates the work within an active, significant area of inquiry, typically by asserting the centrality or importance of the topic and reviewing what is already known. Move 2, establishing a niche, disrupts that settled territory by indicating a gap, a contradiction, an unresolved question, or an unexplored population or method: the move that converts a summary of prior work into a rationale for new work. Move 3, occupying the niche, announces the present study as the response to that gap, outlining its purpose, and often previewing its principal findings or structure. The power of CARS lies in exposing why so many introductions fail: they execute Move 1 at length, sometimes brilliantly, and then jump directly to Move 3 without ever performing Move 2. The reader is told what is known and then, abruptly, what the authors did, but never why the second follows from the first. The niche, the gap that licenses the entire enterprise, is left implicit or absent. Swales identified several characteristic steps for Move 2, and it repays study to know them: counter-claiming (asserting that existing accounts are wrong), indicating a gap (noting that something has not been done), question-raising, and continuing a tradition (extending prior work). Each casts the study's contribution in a different rhetorical light, and choosing among them deliberately is a mark of authorial control. Move 2 also carries a delicate interpersonal charge. To establish a niche is, in effect, to say that the existing literature is in some respect insufficient, which risks alienating the very scholars who populate the peer review pool. Skilled writers therefore modulate the force of the gap statement: rather than declaring that prior work is flawed, they frame the niche as a natural next question, a boundary condition not yet tested, or a synthesis not yet attempted. The lexical grammar of hedging (may, appears, remains to be established) and of concession (while X has advanced understanding of Y, the question of Z persists) does real diplomatic work here. The niche must be wide enough to justify the study but narrow enough not to insult the field. Finally, Move 3 should do more than restate an aim. In high-impact venues, the occupation of the niche often includes an explicit statement of contribution and, increasingly, a compressed preview of the principal result. This forward reference is not a spoiler; it is a service to the reader, who evaluates evidence more critically when the claim it supports is already in view. It also disciplines the author, because a preview that overreaches the eventual Results will be caught by the reader at the moment of greatest suspicion. Aligning the Move 3 promise with the Discussion's delivery closes the outer loop of the hourglass and is among the most reliable revisions an author can make. Move Communicative purpose Typical realisations Common failure mode Move 1: Establish a territory Show the topic is real, active, and important Centrality claims; synthesis of prior findings; definitional framing Over-long literature summary with no evaluative stance Move 2: Establish a niche Convert prior work into a rationale for new work Gap statements; counter-claims; question-raising; hedged concession Move skipped entirely, leaving the study unmotivated Move 3: Occupy the niche Announce purpose, contribution, and preview Statement of aim; explicit contribution; forward reference to findings Bare aim with no stated significance or preview Table 2.1 — The CARS model mapped to typical linguistic realisations and common failure modes. Sequencing Complex Arguments Across Sections Once the macro-architecture is set, the harder craft is sequencing the argument within and across sections so that the reader is never asked to accept a claim before its support is available, and never made to hold an unexplained term in suspense. This is a problem of dependency ordering. Every substantive claim in a manuscript rests on prior claims, definitions, or evidence; a well-sequenced article is a topological sort of that dependency graph, arranged so that nothing is used before it is introduced. The Methods, for example, must define a measure before the Results report values of it, and the Results must report a finding before the Discussion interprets it. Violations of this ordering, forward references to material not yet established, are the textual equivalent of a compile error: the reader stalls, scrolls back, and loses trust. Within the Results, the sequence of findings is itself an argument and should rarely follow the chronological order in which analyses were run. The disciplined approach is to order results in the sequence that most efficiently builds toward the study's central claim, typically moving from the primary outcome to secondary and then to supporting or exploratory analyses, and mirroring the order of aims announced at the neck of the hourglass. When the Results order tracks the Introduction's promises, the Discussion can proceed with the same rhythm, and the whole manuscript acquires a through-line. When it does not, the Discussion is forced into constant cross-referencing that fatigues the reader. The Discussion carries the heaviest sequencing burden because it must perform several distinct functions without letting them blur: restating the principal finding in plain terms, interpreting it against the prior literature, acknowledging limitations, and specifying implications. A robust convention is to open the Discussion with a single paragraph that answers the research question directly and unhedged, before any qualification, so the reader's central question is resolved immediately. Subsequent paragraphs can then situate that answer, compare it with prior work (explaining both agreements and discrepancies rather than only the flattering ones), and delimit its reach. Deferring the direct answer until the end of the Discussion, a frequent novice error, forces the reader to infer the take-home message from a mass of qualification and is a reliable way to weaken an otherwise strong paper. Transitions between sections deserve explicit engineering rather than reliance on the section headings to do the work. The last sentence of the Introduction and the first of the Methods, the last of the Results and the first of the Discussion, are hinge points where a well-placed sentence of orientation dramatically improves flow. A single forecasting sentence at the close of the Introduction that names the analytic strategy, or an opening sentence of the Discussion that restates the aim before answering it, bridges the scope shift of the hourglass and reassures the reader that the author is in control of the whole structure, not merely of its parts. Signposting, Metadiscourse, and the Visible Skeleton Signposting is the family of linguistic devices by which an author makes the structure of the argument visible to the reader. It is a subset of what applied linguists call metadiscourse: language that is not about the subject matter but about the text itself and the writer-reader relationship. Forecasting statements (this section first describes the cohort, then the analytic model), framing markers (turning now to the secondary outcome), code glosses (that is, in other words), and endophoric references (as noted above, see Table 2) all belong here. Well-judged metadiscourse is invisible in the sense that the reader is guided without noticing the guidance; poorly judged metadiscourse announces its own machinery and reads as padding. There is a tension in the scientific register that authors must navigate. The prevailing style values economy and impersonality, which can push writers toward stripping out all metadiscourse and producing prose that is technically dense but structurally opaque. The remedy is not to add more signposts but to add better ones at the load-bearing joints: at the transitions between sections, at the opening of each Discussion paragraph, and wherever the argument turns or a potential objection is anticipated. A useful discipline is to read only the first sentence of every paragraph in sequence; if that reduced text tells a coherent story, the signposting is doing its job. If it reads as a list of disconnected topics, the paragraphs are not yet in conversation with one another. Hedging and boosting, though often discussed under the heading of tone, are structurally significant forms of metadiscourse because they calibrate the strength of claims and thereby manage where the reader places weight. A finding introduced with a booster (these data clearly demonstrate) invites scrutiny proportional to the confidence asserted; the same finding hedged (these data are consistent with) sets a lower bar that the evidence can more easily clear. Miscalibration in either direction damages credibility: over-hedging makes a solid contribution sound tentative and unpublishable, while over-boosting invites reviewers to attack claims the data cannot sustain. Reviewers at leading journals are acutely attuned to this calibration, and matching claim strength to evidence strength is among the surest signals of a mature scientific writer. Paragraph-Level Architecture: Topic Sentences and Given-New Progression If the manuscript is a building, the paragraph is the room, and it obeys its own architectural rules. A well-constructed body paragraph is not a collection of related sentences but a single controlled movement of thought, and its most important sentence is its first. The topic sentence states the paragraph's claim, its contribution to the surrounding argument, and everything that follows either supports, elaborates, qualifies, or illustrates that claim. When topic sentences are strong and placed first, the reader can navigate the article at two resolutions: skimming the topic sentences for the argument's spine, or descending into any paragraph for the evidence. When topic sentences are buried, delayed, or absent, the reader must read every sentence at full attention merely to discover what each paragraph is about, a cost that reviewers experience as difficulty and often report as poor writing without being able to name the cause. Beneath the paragraph lies the sentence-to-sentence flow governed by the given-new contract. Readers process a sentence most easily when its subject position holds information already established, connecting the sentence to what precedes it, while new or emphatic material is placed toward the end, in the position of natural stress. Chaining sentences so that the new information of one becomes the given of the next produces a seamless forward pull; this is the linear thematic progression that characterises the most readable scientific prose. The alternative, opening successive sentences with disconnected new subjects, forces the reader to build the connections that the writer should have built, and it is a leading cause of prose that feels choppy despite being grammatically correct. Thematic progression comes in recognisable patterns, and choosing among them consciously is a professional skill. In linear progression, the rheme (new information) of one sentence becomes the theme (starting point) of the next, ideal for building a chain of reasoning. In constant progression, a series of sentences share the same theme, elaborating one entity from several angles, which suits descriptions of a single method or cohort. In derived progression, successive themes are all sub-topics of an overarching hypertheme, useful for surveying the several facets of a complex phenomenon. Skilled writers move between these patterns to match the logical shape of the content, and diagnosing a flat or confusing paragraph often amounts to noticing that its thematic progression is fighting the logic it should express. Cohesion at this level is achieved through concrete devices that repay conscious attention: lexical repetition of key terms rather than elegant variation (calling the same construct by three different names to avoid repetition is a false economy that confuses readers who cannot tell whether the terms are synonyms); reference chains using pronouns and demonstratives that always have an unambiguous antecedent; and a restrained, purposeful use of connectives that name the logical relationship (therefore, however, because) rather than merely gesturing at continuation (additionally, moreover). The goal throughout is that coherence, the deep conceptual unity of the argument, is made audible through cohesion, the surface signals that let the reader feel the unity without having to reconstruct it. Variations of IMRAD Across Disciplines IMRAD is dominant but not universal, and the doctoral writer publishing across or between fields must understand its principal variants. In much of experimental biomedicine and psychology, the structure is close to the canonical form, often with a formally registered protocol and reporting checklists shaping the Methods and Results. In these fields the ordering conventions are strict, and reporting guidelines such as CONSORT for randomised trials, STROBE for observational studies, and PRISMA for systematic reviews effectively prescribe the contents and sometimes the sequence of specific subsections. Writing to the relevant guideline is not merely compliance; it is an efficient way to inherit a battle-tested architecture that reviewers already expect. In many engineering and computer science venues, the structure diverges in instructive ways. Results and Discussion are frequently merged into a single Evaluation or Experiments section that interleaves findings with interpretation, because the argument advances through a sequence of experiments each of which is stated, run, and interpreted before the next begins. Related Work often appears as a distinct section, sometimes after rather than before the technical contribution, so that the reader understands the proposed method before it is positioned against alternatives. A Conclusion section, largely vestigial in biomedical IMRAD, does real work here, summarising contributions and future directions. The underlying rhetorical functions of CARS and the hourglass persist, but they are distributed across a different set of section labels. The humanities and much of qualitative social science depart furthest from IMRAD, often organising by theme or by argument rather than by the empirical cycle, with methodology woven through the analysis rather than quarantined in a labelled section. Even here, the deeper principles hold: the writer must establish a territory, carve a niche, and occupy it; must sequence claims so that each rests on established ground; and must manage scope so that the reader travels from a broad problem to a specific contribution and back. Recognising that IMRAD is one instantiation of these deeper principles, rather than the principles themselves, is what allows a scholar to write persuasively in an unfamiliar genre by reasoning from function to form rather than copying a template. A further contemporary variation concerns the growing prominence of structured abstracts, graphical abstracts, significance statements, and author-contribution declarations recorded through taxonomies such as CRediT. These paratextual elements are not part of IMRAD proper, but they participate in the same architecture: the significance statement is a compressed hourglass, the graphical abstract is a spatial argument, and the structured abstract is IMRAD in miniature. Treating these elements as afterthoughts wastes the disproportionate influence they exert on editors screening submissions and on readers deciding whether to engage. The architecturally aware author drafts them as deliberate distillations of the manuscript's logic, not as boxes to be filled after acceptance. Practical and Real-World Example One: Diagnosing a Broken Introduction Consider a doctoral candidate in environmental science whose manuscript on soil carbon dynamics has been returned by two journals with reviewer comments complaining that the contribution is unclear, despite the underlying dataset being widely praised. Reading the Introduction against the CARS model exposes the problem immediately. The first four paragraphs execute Move 1 with great competence: they establish the centrality of soil carbon to climate mitigation and synthesise a large body of prior measurement studies. The fifth paragraph then states the aim of the study. Move 2 is entirely absent. Nowhere does the text say what the prior measurement studies failed to resolve, and so the reader arrives at the aim unable to see why this study, rather than another entry in the existing series, was needed. The revision is structural, not stylistic. Between the synthesis and the aim, the author inserts a Move 2 paragraph that identifies the specific niche: prior studies measured carbon stocks at single depths and single seasons, leaving the vertical and seasonal dynamics unresolved, which matters because mitigation models assume a stability that has never been directly tested. This gap statement is hedged to avoid disparaging the field (while these studies have established robust surface estimates, the depth-resolved dynamics remain uncharacterised) and it is then answered directly in Move 3, where the aim is restated with an explicit contribution and a one-sentence preview of the principal finding. The dataset did not change; the architecture did, and with it the perceived significance. This example illustrates the general lesson that unclear-contribution critiques are almost always failures of Move 2, and that they are repaired by structural insertion rather than by polishing the surrounding prose. Practical and Real-World Example Two: Re-Sequencing a Discussion A second candidate, in health services research, has a Discussion that reviewers describe as meandering. Mapping the section paragraph by paragraph reveals that it opens with three paragraphs of limitations and methodological caveats before, in the fourth paragraph, finally stating what the study found. The direct answer to the research question is thereby buried beneath qualification, and readers who stop early, as busy readers do, leave without the take-home message. Worse, the paragraphs interpreting the findings are interleaved with unrelated comparisons to prior work in no discernible order, so the argument never accumulates. The re-sequencing applies the principles of dependency ordering and given-new progression at the section scale. The revised Discussion opens with a single, unhedged paragraph that answers the research question in plain language and states the principal finding, satisfying the reader's central demand immediately. The second paragraph situates that finding against the most directly comparable prior study, explaining both where they agree and where they diverge and why. Subsequent paragraphs then extend outward to broader literature and mechanism, each opening with a topic sentence that names its function. Limitations are consolidated into a single, honest paragraph placed after the interpretation but before the implications, where they qualify without smothering. The implications paragraph closes the hourglass by returning to the field-level problem raised in the Introduction. No new content was written; the same paragraphs were reordered and their topic sentences rewritten, and the section that reviewers had called meandering became, in their words on resubmission, focused. The lesson is that Discussion quality is dominated by sequence, and that sequence is a revisable variable independent of the prose within each paragraph. Building the Deliverable: A Manuscript Outline and Structural Blueprint The unit builds toward a concrete artefact: a detailed manuscript outline and structural blueprint for a full research article, produced before substantial drafting rather than after. The blueprint is a working document, typically a table or nested outline, that assigns to every planned paragraph four attributes: its function (the rhetorical move it performs), its core claim (the topic sentence in draft form), the evidence it will marshal (specific results, figures, or citations), and its links (what it depends on from earlier and what later material depends on it). Constructing this map forces the dependency graph into the open, so that forward references, orphaned claims, and unsupported assertions are caught while they are cheap to fix, before a single polished paragraph has been written and become psychologically expensive to cut. A rigorous blueprint integrates the literature at the level of individual paragraphs rather than as a bibliography to be sprinkled in later. For each paragraph that engages prior work, the author records which specific sources are cited and, more importantly, what argumentative role each source plays: establishing the territory, marking the gap, providing a comparison for a finding, or supplying a method. This turns the reference list from a compliance exercise into a structural component and prevents the common pathology of citation clumping, where a dozen references are massed at one point to signal diligence while other claims go unsupported. Tools such as reference managers and, increasingly, structured note systems make this paragraph-level integration tractable, but the discipline is intellectual before it is technical. The blueprint also encodes evidence placement, the decision about where each result, table, and figure appears and what claim it is positioned to support. Because a figure is an argument in visual form, its placement is a structural choice: a result presented too early, before the reader has the framing to interpret it, wastes its force, while the same result placed at the culmination of a build can carry a paper. Mapping figures to claims in the blueprint reveals whether the visual evidence is distributed in proportion to the argument's needs or clustered by convenience. It also exposes redundant displays and, occasionally, claims that the author intends to make but has no evidence positioned to support, the most valuable discovery a blueprint can offer because it can still be remedied by further analysis. Finally, the blueprint should be built against a specific target journal from the outset, because architecture is partly audience-relative. Word limits, the presence or absence of a separate Conclusion, the expected number of display items, whether Results and Discussion are combined, and the applicable reporting guideline all shape the structure and are known in advance from the author instructions. Designing the blueprint for the target venue avoids the costly late-stage restructuring that follows from writing a generic manuscript and only then choosing where to send it. When a paper is rejected and redirected, the blueprint becomes the instrument of efficient adaptation, allowing the author to re-map the same arguments and evidence onto a new structure rather than rewriting from scratch. Paragraph Function (move) Core claim (draft topic sentence) Planned evidence / literature Links Intro 1 Move 1: territory The phenomenon matters to a broad field-level problem. Two to three synthesising reviews establishing importance Sets up return in final Discussion paragraph Intro 2 Move 1 to 2: bridge Prior work has established X but examined it narrowly. Key primary studies to be extended Feeds the gap in Intro 3 Intro 3 Move 2: niche The specific dynamics of Y remain uncharacterised. Hedged gap statement; note absence of prior tests Motivates aim in Intro 4 Intro 4 Move 3: occupy We test Y and report its principal behaviour. Statement of aim, contribution, and preview Promise delivered in Discussion 1 Disc 1 Direct answer Y behaves as follows, answering the question directly. Primary result and its key figure Fulfils Intro 4 promise; anchors hourglass return Table 2.2 — A paragraph-level blueprint fragment for the Introduction and opening Discussion of a research article. Each row specifies function, draft core claim, planned evidence, and structural links. Sample Activities and Assessments Sample Activity • Task: Select a published research article from a leading journal in your discipline and perform a full CARS move analysis of its Introduction, annotating each sentence with the move and step it performs and marking any transitions between them. • Expected output: A marked-up copy of the Introduction plus a one-page commentary identifying how Move 2 (the niche) is established, which rhetorical step the authors chose, and how the strength of the gap statement is calibrated to avoid alienating the field. • Assessment criteria: Accuracy of move and step identification; insight into the linguistic realisation of the niche; and a critical judgement about whether the introduction could be strengthened, justified with reference to the hourglass and promissory symmetry. Sample Activity • Task: Take a single paragraph from a draft of your own writing and diagnose its thematic progression, labelling the theme and rheme of each sentence and identifying whether the progression is linear, constant, or derived. • Expected output: An annotated version of the paragraph, a statement of which progression pattern the content actually requires, and a rewritten paragraph that repairs any mismatch and restores given-new ordering. • Assessment criteria: Correct identification of theme and rheme; sound reasoning about the appropriate progression pattern for the content; and demonstrable improvement in cohesion in the rewrite, with topic sentence placed first and unambiguous reference chains. Sample Assessment • Task: Produce a complete manuscript outline and structural blueprint for a research article of your own, targeting a named journal, in the paragraph-level table format introduced in this unit. • Expected output: A blueprint mapping every planned paragraph to its function, draft core claim, planned evidence and literature, and structural links, accompanied by a schematic of the hourglass scope and a mapping of figures and tables to the claims they support. • Assessment criteria: Completeness and internal consistency of the dependency graph (no forward references or orphaned claims); appropriateness of evidence and figure placement; fidelity to the target journal's structural conventions and reporting guideline; and evidence of promissory symmetry between the Introduction's promises and the Discussion's delivery. Synthesis: From Template to Architecture The through-line of this unit is a single shift in stance: from treating manuscript structure as a template to be filled to treating it as an architecture to be designed. IMRAD, the hourglass, CARS, signposting, given-new progression, and thematic patterning are not disconnected rules but nested expressions of one underlying obligation, which is to build an argument that a sceptical expert reader can follow, evaluate, and ultimately accept, with the minimum of avoidable friction. At the macro scale this means managing scope so that the reader travels from a broad problem to a specific contribution and back, keeping every promise made on the descent. At the section scale it means sequencing claims so that support always precedes the claim it supports, and answering the reader's central question the moment the evidence permits. At the paragraph scale it means leading with topic sentences and chaining information from given to new so that coherence is felt rather than reconstructed. The practical payoff of internalising these principles is that revision becomes tractable and diagnosis becomes precise. When a reviewer reports that a contribution is unclear, the architecturally literate author knows to look for a missing Move 2; when a Discussion is called meandering, the author knows to check whether the direct answer has been buried and whether the paragraphs are in dependency order; when prose reads as choppy despite being correct, the author knows to inspect thematic progression and reference chains. The vague, demoralising feedback that so often accompanies rejection resolves into specific, addressable structural faults. The blueprint that this unit builds toward is the instrument that makes such diagnosis proactive, exposing the architecture before the prose is poured, so that the manuscript is engineered to be coherent rather than repaired into coherence after the fact. This is the discipline that separates writing that merely reports research from writing that reliably persuades the readers and reviewers who guard the highest-impact venues. Hashtags: #ScientificWritingAndHighImpactPublishing #ScientificWriting #AcademicPublishing #HighImpactPublishing #ScholarlyCommunication #ResearchPublishing #JournalSelection #Bibliometrics #JournalImpactFactor #CiteScore #SJR #SNIP #Scopus #WebOfScience #QuartileRankings #PeerReview #PublicationEthics #COPE #ICMJE #DORA #CRediT #PredatoryPublishing #IMRAD #CARSModel #FutureOfScholarlyPublishing

Download the Book (PDF): Introduction Grounded theory is probably the most cited and least consistently practised method in qualitative research. Search almost any journal in nursing, management, education, information systems, public health or engineering education and you will find papers announcing that they "used a grounded theory approach." Read the methods sections closely and a large share of them describe something else: a thematic analysis with the word "coding" in it, a set of interviews summarised under headings, or a conceptual framework chosen in advance and then decorated with quotations. The label has travelled much further than the practice. This matters more than a quarrel over names. Grounded theory makes an unusual promise. It claims that a researcher can begin with messy, unstructured material, such as interview transcripts, field notes, documents or open survey responses, and build from it an explanatory theory that did not exist before the study began. Not a description of what people said, not a list of themes, but a set of related concepts that explains how something happens: how people manage a stigmatised illness, how organisations absorb a new technology, how students become engineers. That promise is what attracts researchers to the method, and it is also what makes the method easy to fake. A study that ends with five tidy themes and a boxes-and-arrows model can look like a grounded theory while having skipped every step that would make its theory trustworthy. Researchers who come to grounded theory from outside sociology face a particular version of the problem. A civil engineer studying how site managers improvise around safety rules, a clinician investigating why patients abandon a treatment, a computer scientist trying to understand how developers adopt a new tool, an economist who has noticed that a model keeps failing to predict household behaviour: each arrives with strong training in some other mode of inquiry and a practical question that numbers have not answered. They open the methods literature and find, almost immediately, that there is not one grounded theory but several, that the founders of the method spent thirty years disagreeing with each other in print, and that the vocabulary shifts from book to book. "Axial coding" is essential in one text and condemned as a distortion in another. Literature review before data collection is required by one school and warned against by another. Saturation is a cornerstone, or a myth, depending on whom you read. The result is predictable. Many mixed-discipline researchers take the most procedural version they can find, usually the coding sequence popularised by Anselm Strauss and Juliet Corbin, and apply its labels without absorbing the logic that made the labels meaningful. Others borrow from all the versions at once, citing Barney Glaser on emergence, Strauss and Corbin on axial coding and Kathy Charmaz on constructivism in the same paragraph, without noticing that the three positions contradict each other on the questions that matter most. Reviewers who know the method spot this at once. Reviewers who do not know it let it through, which is how the label has spread so far. The argument of this book This book makes one argument and organises everything around it. A grounded theory earns its authority from a traceable chain of analytic decisions that runs from raw data, through systematic comparison and memo-writing, to an integrated theory. The competing schools of grounded theory differ chiefly in how they justify that chain, so a researcher's job is to choose one lineage deliberately and then carry out its core moves consistently, rather than to collect coding labels from all of them. Three consequences follow, and they shape the chapters ahead. The first is that the historical split between the schools is not background trivia. It is the most practically useful thing a newcomer can understand, because each school's procedures make sense only in the light of what it believes about data, the researcher and theory. The first two chapters therefore explain where grounded theory came from, why it split, and what each of the three main lineages, usually called Glaserian or classic, Straussian, and constructivist, actually commits you to. The second is that the analytic procedures, open coding, axial or focused coding, selective or theoretical coding, theoretical sampling, saturation and memo-writing, are best understood as instruments for building and documenting that chain of decisions. They are not a checklist that produces theory when completed. Chapters 3 to 7 treat them in the order a study encounters them, with attention to what each move is for, how it differs between the schools, and what goes wrong when it is done mechanically. The third is that the quality of a grounded theory can be judged, and a researcher crossing disciplines needs to know how it will be judged before writing the first memo, not after submitting the paper. Chapter 8 takes up evaluation criteria, the characteristic failures of grounded theory in applied fields, the use of software and, now, of large language models in coding, and the particular difficulties of presenting grounded theory work to audiences trained in hypothesis testing. Who this is for The intended reader is a researcher with real training in some discipline who needs grounded theory to answer a question, and who wants to do it properly. That includes doctoral students in professional and applied fields, academics in quantitative disciplines taking on their first qualitative project, practitioner-researchers in health, education and management, and supervisors who have been asked to guide a grounded theory thesis without having done one themselves. No prior knowledge of qualitative methods is assumed, but the book does assume that you are willing to think about why a procedure exists before adopting it. To make the procedures concrete, several chapters follow a single invented study, clearly identified as illustrative, about how researchers who move between disciplines establish credibility in a new field. The subject was chosen because it will be familiar to most readers, and because it is exactly the kind of process question grounded theory handles well: it concerns what people do over time, under changing conditions, in ways that existing theory only partly explains. The excerpts from this imaginary study are written for teaching purposes; they are not data and no findings are claimed from them. Everywhere else, named studies, books and authors are real, and the Notes list the works cited. What the book leaves out A short book on a large methodological tradition has to decline a great deal. It does not attempt a full account of the philosophy of social science behind the different schools, though Chapter 2 explains enough of it to make the practical choices intelligible. It does not survey every variant of grounded theory; Adele Clarke's situational analysis, Robert Thornberg's informed grounded theory and the Gioia methodology in management research receive brief treatment where they bear on the main argument, and others are omitted. It does not teach interviewing or field observation from scratch, because good general guides exist, and it does not provide software tutorials, which date quickly. What it does attempt is narrower and, for the reader it is written for, more useful: to show what grounded theory actually requires, where the schools genuinely differ, and how to run a study whose theory a sceptical reader from any discipline can follow back to its evidence. Chapter 1. The Discovery and the Split Every research method is an answer to a problem, and a method is easiest to use well when you know what problem it was answering. Grounded theory was answering a specific complaint about American sociology in the 1950s and 1960s, and nearly every procedure in the method, including the ones that seem fussy or counterintuitive, makes sense as a response to that complaint. The later split between the method's founders is also easier to understand once the original problem is clear, because the two men had solved it together from quite different starting points. The problem grounded theory set out to solve By the late 1950s, American sociology had settled into a division of labour that many younger researchers found stifling. At one end sat what C. Wright Mills, in The Sociological Imagination (1959), called "grand theory": sweeping conceptual systems, most famously Talcott Parsons's structural functionalism, that aimed to explain social order in general and were pitched at a level of abstraction that made them hard to connect with anything a researcher could observe. At the other end sat increasingly sophisticated survey research, much of it developed at Columbia University under Paul Lazarsfeld, which was superb at testing hypotheses but had little to say about where good hypotheses came from. Robert Merton, also at Columbia, had argued for "theories of the middle range," explanations modest enough to be tied to evidence and general enough to matter, but the question of how such theories were to be produced remained open. The working assumption of the period was that theory came first, supplied by a small number of gifted theorists, and that empirical researchers then tested it. Barney Glaser and Anselm Strauss attacked that assumption directly in The Discovery of Grounded Theory: Strategies for Qualitative Research (1967). Their argument was that the emphasis on verifying existing theory had left sociology with too few theories that fitted the situations researchers actually studied, and that the generation of theory from systematically gathered data was a legitimate and teachable research task in its own right. They drew a contrast between theory deduced from abstract premises, which might or might not fit any real setting, and theory discovered from data, which would fit because it had been built from it. The book's rhetoric was combative. It cast the established arrangement as one in which a few theorists supplied ideas and a mass of researchers were reduced to testing them, and it invited ordinary fieldworkers to become theorists themselves. Two claims from Discovery remain central to every version of the method. The first is that theory generation requires a particular kind of analysis, which Glaser and Strauss called the constant comparative method. Instead of coding all the data first and analysing later, the researcher compares each new incident with incidents already coded, compares incidents with emerging categories, and compares categories with one another, all while data collection continues. The second is theoretical sampling: decisions about what data to collect next are driven by the emerging theory, not fixed in advance by a sampling frame. Together these two ideas produce the method's defining feature, the interleaving of collection and analysis, so that the study is shaped as it proceeds by what the analysis is discovering. Discovery also distinguished two levels of theory. Substantive theory explains a particular area of inquiry, such as patient care in hospitals, the work of police officers, or the careers of academics. Formal theory is pitched at a more general conceptual level, such as stigma, status passage or organisational careers, and can be built by comparing substantive theories across different areas. Most grounded theory studies, then and now, produce substantive theory. The distinction matters for mixed-discipline researchers because it clarifies what a grounded theory study can reasonably claim: a well-built substantive theory of how, say, nurses handle alarm fatigue in intensive care units is a real contribution, and it does not need to pretend to explain human attention in general. Two founders, two codifications The partnership that produced Discovery joined two very different intellectual traditions, and the later split can be read as those traditions reasserting themselves. Glaser had trained at Columbia, where Lazarsfeld and Merton were the dominant figures. From Lazarsfeld he absorbed a set of analytic habits developed for quantitative work: the careful construction of indices from multiple indicators, the idea that different indicators could be treated as interchangeable measures of the same underlying concept, and a strong sense that analysis should be systematic and disciplined. Glaser later credited this training for the logic of constant comparison, in which many incidents are compared until the concept they share is identified. His instinct was always towards method as a discipline that protected the researcher from imposing preconceived ideas on the data. Strauss came from the University of Chicago, the home of the pragmatist philosophy of John Dewey and George Herbert Mead and of the symbolic interactionism developed by Herbert Blumer. Chicago sociology had a long tradition of fieldwork, from urban ethnographies of the 1920s to Everett Hughes's studies of occupations, and it treated social life as a matter of people acting on the basis of meanings that are constructed and revised in interaction. Strauss brought an interest in action, process and meaning, and a pragmatist's willingness to let method be shaped by the problem at hand. The two met at the University of California, San Francisco, where Strauss had been appointed to build a social and behavioural sciences programme in the School of Nursing. Their joint fieldwork on how hospital staff and dying patients managed the knowledge that a patient was dying produced Awareness of Dying (1965) and Time for Dying (1968). These studies are still the best demonstration of what the method was meant to produce. Awareness of Dying developed the concept of awareness contexts: the combination of what each party in an interaction knows about the patient's condition and what each knows about what the others know. Glaser and Strauss identified four such contexts, closed awareness (the patient does not know), suspected awareness (the patient suspects and tries to confirm), mutual pretence (everyone knows but acts as if they do not) and open awareness, and they showed how each context structured the interaction between patients, families, nurses and doctors, and how contexts shifted over time. This was not a list of themes about dying. It was a set of related concepts that explained variation in behaviour and predicted how interactions would unfold, and it proved useful far beyond hospitals, because the same logic of who knows what about whom applies to many other social situations. Discovery was written partly to explain how those studies had been done. That origin is worth remembering. The method was codified after the fact from a successful piece of research, not designed on paper and then tested. Its early formulations were accordingly loose on procedure and emphatic on attitude: stay close to the data, compare constantly, let the theory emerge, sample for theoretical purposes, write memos. Over the following two decades each founder tried to make the method teachable, and each did so in the image of his own training. Glaser's Theoretical Sensitivity (1978) is the fullest early statement of what is now called classic or Glaserian grounded theory. It set out the distinction between substantive coding, in which the researcher codes the data for the concepts it contains, and theoretical coding, in which the researcher identifies how those substantive codes relate to one another. To help with the second task Glaser offered a set of coding families, groups of generic relationships that a theory might use to integrate its concepts. The best known is what he called the "Six C's": causes, contexts, contingencies, consequences, covariances and conditions. Others included process families (stages, phases, trajectories), degree families (amount, intensity, range), strategy families and many more. The point was not that a theory should use any particular family, but that a theoretically sensitive researcher should know many of them so that the one that fitted the data could emerge rather than be forced. Glaser also emphasised the core category, a concept that accounts for most of the variation in how participants resolve their main concern, and the discipline of memo-writing and sorting. Strauss's Qualitative Analysis for Social Scientists (1987), and much more influentially the textbook he co-wrote with Juliet Corbin, Basics of Qualitative Research: Grounded Theory Procedures and Techniques (1990), took a different path. Strauss and Corbin broke analysis into three named stages: open coding, which opens up the data into concepts and their properties; axial coding, which reassembles the data by relating categories to subcategories around the "axis" of a category; and selective coding, which integrates the categories around a core category. To guide axial coding they proposed a coding paradigm, a standard template that asks of each phenomenon what causal conditions give rise to it, what context and intervening conditions shape it, what action and interaction strategies people use to handle it, and what consequences follow. They added further tools: a conditional matrix for tracing influences from the micro to the macro level, and a battery of analytic techniques such as asking questions of the data, systematic comparison and the "flip-flop" technique of considering opposites. Basics was clear, practical and procedural, and it became the most widely used grounded theory text in the world, particularly in nursing and other applied fields. It gave new researchers what the looser early formulations had not: a sequence of steps they could follow and report. The split and the constructivist turn Glaser regarded Basics as a betrayal of the method. His response, Basics of Grounded Theory Analysis: Emergence vs. Forcing (1992), was a book-length rebuttal that went through Strauss and Corbin's text point by point. His central charge was that the coding paradigm and the other procedures forced data into a preconceived framework. If every category must be examined for causal conditions, context, intervening conditions, strategies and consequences, then the researcher has decided the shape of the theory before looking at the data, which is precisely what grounded theory was supposed to prevent. Glaser argued that what Strauss and Corbin described was a different method, which he called "full conceptual description," valuable perhaps but not grounded theory, and that they should call it something else. The dispute was not simply personal, although it was bitter. It exposed a genuine tension that had been present in Discovery all along. On one side was the aspiration to let theory emerge from data untouched by the researcher's prior commitments. On the other was the practical need to give researchers enough structure that they could actually produce a theory rather than drowning in codes. Glaser resolved the tension by trusting the method's inner discipline, constant comparison and patience, and by teaching theoretical sensitivity through a broad acquaintance with many possible theoretical codes. Strauss and Corbin resolved it by providing a template that made the analytic task tractable, accepting in return that the template shaped what was found. Methodologists such as Udo Kelle (2005) later argued that Glaser's own coding families were not really different in kind: both founders supplied the researcher with prior theoretical tools, and the real question was how explicit and how restrictive those tools should be. Strauss died in 1996. Corbin continued to revise Basics, and the third (2008) and fourth (2015) editions, now titled Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, softened much of the earlier proceduralism. Corbin explicitly acknowledged a more constructivist view of analysis and de-emphasised the rigid use of the coding paradigm. Researchers who cite "Strauss and Corbin" without specifying the edition are therefore often citing a moving target. Glaser, for his part, continued to publish through his own Sociology Press and supported the Grounded Theory Institute and the journal The Grounded Theory Review, defending what he called classic grounded theory until his death in 2022. He also insisted, in books such as Doing Quantitative Grounded Theory (2008), that grounded theory was a general method usable with any kind of data, quantitative included, not a qualitative method as such. The third major lineage came from a student of both founders. Kathy Charmaz completed her doctorate at UCSF under Strauss and Glaser, and her early work on chronic illness, including the article "Loss of self: a fundamental form of suffering in the chronically ill" (1983) and the book Good Days, Bad Days: The Self in Chronic Illness and Time (1991), was among the most admired grounded theory research of its generation. In a chapter for the second edition of the Handbook of Qualitative Research (2000) she distinguished "objectivist" grounded theory, which in her account treated data as real facts waiting to be discovered by a neutral observer, from "constructivist" grounded theory, which treated both data and analysis as constructed through the interaction between researcher and participants. Her book Constructing Grounded Theory (2006; second edition 2014) developed this position into a complete, practical guide. Constructivist grounded theory kept the core tools, coding, constant comparison, memo-writing and theoretical sampling, while changing their meaning. The researcher is no longer a neutral instrument who discovers a theory latent in the data but a participant in producing it, whose background, questions and relationships shape what is seen. Theories are interpretive renderings of a world, not objective reports of it. Charmaz also simplified the coding sequence into initial coding and focused coding, with theoretical coding as an optional later step, and she dropped axial coding as a required stage. Glaser rejected this too, arguing in an article titled "Constructivist grounded theory?" (2002) that the charge of objectivism misread his method and that constructivism reintroduced exactly the researcher-driven interpretation grounded theory was meant to discipline. Other variants followed. Adele Clarke's Situational Analysis: Grounded Theory After the Postmodern Turn (2005) extended the constructivist line by mapping the full situation of inquiry, including discourses, non-human actors and silences, rather than focusing on a single basic social process. Robert Thornberg's "informed grounded theory" (2012) argued for engaging with existing literature throughout a study rather than postponing it. In management research, the approach set out by Dennis Gioia, Kevin Corley and Aimee Hamilton (2013) adapted grounded theory ideas into a structured procedure for presenting first-order concepts, second-order themes and aggregate dimensions. Each of these has its uses, and several reappear later in this book. What the history teaches For a researcher entering the field from another discipline, three lessons follow from this history. First, when a paper says it used grounded theory, the phrase is incomplete. It could mean a study built on Glaser's classic principles, one following an early or late edition of Strauss and Corbin, one following Charmaz's constructivist guide, or, too often, one that has borrowed selectively from all of them. Readers who know the field will want to know which, and a methods section that does not say so will be read as a sign that the author does not know the difference. Second, the disagreements between the schools are not idiosyncratic preferences. They follow from different answers to deep questions: whether theory is discovered or constructed, whether the researcher should approach the field with prior theoretical frameworks, and how much procedural structure analysis needs. These are questions a researcher from any discipline already has views about, often without having stated them. A trained epidemiologist and a trained anthropologist will lean in different directions, and the next chapter is designed to help readers identify where they stand. Third, and most important for the argument of this book, all three lineages agree on more than their quarrels suggest. All of them require that analysis begin while data collection is still under way. All require constant comparison. All require that sampling become progressively theoretical. All require memo-writing as the medium in which the theory is actually developed. All aim at a theory that explains a process or pattern of action, not merely a description of topics. Those shared commitments are what make a study grounded theory rather than some other form of qualitative analysis, and they are what create the traceable chain from data to theory on which the method's credibility depends. The schools differ in how they justify that chain and in which tools they use to build it. They do not differ on whether it must exist. Chapter 2. Three Grounded Theories and What Each Commits You To A researcher choosing a grounded theory lineage is choosing a set of answers to questions that usually stay implicit in their home discipline. What counts as data? Is the theory you produce a discovery or a construction? Should you read the literature before you go into the field? What does the researcher's own background contribute, and what does it contaminate? How structured should analysis be? What does the finished product look like, and how will anyone judge it? The three main lineages give different answers to these questions, and the answers have practical consequences for how a study is designed, conducted and written. This chapter sets out those answers and then addresses the question most mixed-discipline researchers actually face: how to choose, and whether it is ever legitimate to combine. Epistemology without the jargon Methodological writing on grounded theory is dense with philosophical labels, positivism, post-positivism, objectivism, constructivism, pragmatism, critical realism, and many researchers from applied fields find this the most off-putting part of the literature. The labels matter less than the questions they encode, and the questions can be stated plainly. The first question is about the status of the theory. When a grounded theory study concludes that, say, the main concern of family caregivers of people with dementia is sustaining a sense of normal life, and that they resolve this concern through a process the researcher calls "managing the drift," is that a discovery about something that was there before the researcher arrived, or an interpretation that the researcher composed from what participants said and did? Classic grounded theory leans firmly towards discovery. Glaser wrote of concepts and patterns that "emerge" from the data, and of the researcher's job as to allow that emergence by suspending preconceptions. Constructivist grounded theory leans firmly towards construction: the theory is an interpretive portrayal, one of several that could legitimately be built from the same material, shaped by the questions the researcher asked and the relationships formed in the field. The Straussian position has moved over time. The early editions of Basics read, to most commentators, as broadly post-positivist, treating good procedure as the guarantor of valid findings; Strauss's own roots in pragmatism and interactionism, and Corbin's revisions in the later editions, make it more interpretive than that reputation suggests. The second question concerns the researcher. If theory is discovered, the researcher's background is primarily a risk to be managed; prior knowledge threatens to impose categories on data that did not generate them. If theory is constructed, the researcher's background is an inescapable ingredient, to be acknowledged and examined through what Charmaz calls reflexivity, not eliminated. The classic tradition has its own concept here, theoretical sensitivity, meaning the researcher's ability to see what is theoretically significant in data. For Glaser this sensitivity comes from broad familiarity with many theoretical codes and from the discipline of the method itself, not from commitment to any particular prior theory. The third question is about what the output is for. All three schools aim at theory that explains rather than merely describes, but they differ in what they expect that theory to do. Classic grounded theory emphasises a theory organised around a core category that explains how participants continually resolve their main concern, and it values theories that are abstract enough to be modified and applied to new situations. Straussian grounded theory emphasises dense, well-specified categories with clearly identified conditions, actions and consequences, closer to a detailed explanatory model of a phenomenon. Constructivist grounded theory emphasises interpretive understanding, theory that renders a process in terms that resonate with participants and illuminate their situation, and it is comfortable with theory that is explicitly located in time, place and perspective. Literature, coding and the shape of the output No practical difference between the schools causes more trouble for mixed-discipline researchers than their positions on the literature review, because the requirements of doctoral programmes, ethics committees and funders collide with the method's founding principles. Glaser's position was that the researcher should not undertake a review of the literature in the substantive area before data collection and analysis, since doing so would sensitise the researcher to existing concepts and lead to forcing. Literature in the substantive area should be engaged once the theory has emerged, at which point it becomes more data to be compared with the emerging categories. Glaser did not ask researchers to be blank slates; he encouraged broad reading in other areas, precisely to build theoretical sensitivity without pre-empting the substantive categories. Strauss and Corbin took a more permissive line. They distinguished technical literature (research reports and theoretical papers) from non-technical literature (documents, letters, reports) and allowed that both could be used from the start: to stimulate questions, to suggest areas for theoretical sampling, and to provide concepts that could be compared with the data. They were concerned, however, that researchers not let the literature dictate what they found. Charmaz acknowledges that most researchers cannot and should not avoid prior reading, and that the idea of entering the field without preconceptions is neither possible nor, in a constructivist framework, coherent. Her advice is to treat prior theoretical ideas as sensitising concepts, a term borrowed from Herbert Blumer's 1954 article "What is wrong with social theory?" Blumer distinguished definitive concepts, which specify exactly what falls under them, from sensitising concepts, which give the researcher a general sense of reference and direction. Sensitising concepts are points of departure, not frameworks to be confirmed, and a researcher who uses them must be prepared to discard them if the data do not bear them out. Thornberg's informed grounded theory develops this line further, arguing for continuous, critical engagement with the literature as a source of comparison throughout the study. For researchers in applied fields the practical upshot is this. Almost every thesis proposal, grant application and ethics submission will require some review of prior work. That is compatible with all three lineages provided the review's purpose is stated honestly. A proposal can review the literature to establish that a problem exists and that existing theory does not adequately explain it, which is the justification for doing grounded theory at all, without adopting any of that theory as the study's framework. What is not compatible with any lineage is a study that selects a theoretical model in advance, uses its constructs as coding categories, and then reports that the categories "emerged." Each lineage has its own sequence of analytic operations, and later chapters treat these in detail. At this point it is enough to see how the sequences map onto each other, because researchers frequently misread one school's terms through another's. In classic grounded theory, analysis moves from open coding, in which the researcher codes everything for as many concepts as fit, through selective coding, which begins once a core category has been identified and restricts coding to that category and the concepts related to it, and then to theoretical coding, in which the researcher identifies how the substantive codes relate, drawing on coding families. Note that Glaser's "selective coding" is a sampling and delimiting operation that comes relatively early, once the core is found; it is not the same as Strauss and Corbin's selective coding. In Straussian grounded theory, the sequence is open coding, axial coding using the coding paradigm to relate categories and subcategories, and selective coding, which in this school means integrating and refining the theory around a core category, often by writing a storyline. The later editions of Basics loosen the sequence and present these as analytic operations that overlap rather than stages. In constructivist grounded theory, the sequence is initial coding, often line by line and using gerunds to capture action; focused coding, which selects the most significant or frequent initial codes to sift through larger batches of data; and, optionally, theoretical coding, which specifies relationships between categories and may draw on Glaser's families or other analytic frameworks, provided they earn their way in. The same word can therefore refer to different operations, and the same operation can go by different names. A researcher who writes "we used open, axial and selective coding (Glaser, 1978)" has made an error that a knowledgeable reviewer will catch instantly, since Glaser does not use axial coding and rejected it explicitly. The differences can now be set out compactly. Table 1 compares the three lineages on the dimensions that most affect design and execution. It simplifies positions that have evolved over time, especially the Straussian one, and it should be read alongside the discussion above. Table 1. The three main grounded theory lineages compared. Dimension of difference Classic (Glaser) Straussian (Strauss and Corbin) Constructivist (Charmaz) Status of the theory Discovered; emerges from data Built by procedure; later more interpretive Co-constructed with participants Role of prior literature Deferred until theory emerges Allowed from the outset Read early as sensitising concepts Coding sequence Open, selective, theoretical Open, axial, selective Initial, focused, theoretical Relating categories Coding families that fit Coding paradigm Theoretical codes that earn a place Role of the researcher Sensitive but detached Active, technique-guided analyst Co-constructor; reflexive Typical output Core category and main concern Dense model of a phenomenon Situated interpretive theory Key sources Glaser and Strauss 1967; Glaser 1978 Strauss and Corbin 1990; Corbin and Strauss 2015 Charmaz 2006, 2014 How to choose Choosing among the lineages is best treated as a design decision with three inputs: the researcher's own assumptions, the nature of the research question, and the audience that will judge the work. The researcher's assumptions come first, because a lineage that contradicts them will produce a methods section the researcher cannot defend. A useful test is to ask how you would answer a reviewer who says, "A different researcher analysing these same transcripts would have produced a different theory." If your instinct is to reply that a properly conducted analysis should converge on the same core concepts, you are close to the classic or early Straussian positions. If your instinct is to reply that of course they might, that your theory is one well-grounded interpretation, and that its value lies in how illuminating and useful it is, you are closer to the constructivist position. Neither answer is wrong. What is wrong is to hold one view and cite a methodology built on the other. The research question matters second. Classic grounded theory is at its best when the researcher is entering a substantive area with little existing theory and wants to discover what the participants' main concern actually is, which may differ from what the researcher expected. A health services researcher who sets out to study "barriers to medication adherence" may find, on classic principles, that patients' main concern is something else entirely, perhaps protecting their identity as competent adults, and that adherence is one of several ways they handle it. Straussian grounded theory suits questions where a phenomenon has already been identified and the researcher wants a detailed account of its conditions, strategies and outcomes, a common situation in nursing, education and organisational research. Constructivist grounded theory suits questions where meanings, identities and power are central, where the researcher's position relative to participants is significant, or where the aim is to give an interpretive account that participants and practitioners will recognise. The audience matters third, more than methodologists usually admit. In management and information systems, reviewers often expect Straussian or Gioia-style presentation, with explicit data structures. In nursing, all three lineages are well established and reviewers will expect precise citation. In psychology and sociology, constructivist grounded theory has become common and reviewers will expect a reflexive account of the researcher's position. None of this should dictate the choice, but a researcher who chooses a lineage unfamiliar to their target audience should expect to explain it more fully. The differences become concrete when a single research problem is imagined under each lineage. Take the illustrative study that recurs in this book: a researcher wants to understand how academics who move from one discipline into another, a physicist into computational biology, a nurse into health economics, an engineer into education research, come to be treated as credible members of their new field. Under classic grounded theory, the researcher would begin with as open a question as possible, something close to "What is going on for people who cross disciplinary boundaries in their research careers?" The researcher would avoid reading the substantive literature on interdisciplinarity, academic identity and career transitions until well into analysis, would start interviewing people who had made such moves, and would look first for the participants' main concern. It might turn out to be credibility, as the researcher suspects. It might instead be something the researcher had not anticipated, such as protecting a sense of intellectual continuity, or managing the loss of seniority. The study's output would be a theory organised around whatever core category explains how participants continually resolve that concern. Under Straussian grounded theory, the researcher could reasonably begin with credibility as the phenomenon of interest, having identified it from the literature and from experience. Early interviews would still be open, but axial coding would progressively specify the conditions under which credibility becomes problematic (moving from a high-status to a lower-status field, lacking a publication record in the new venue), the strategies crossers use (acquiring a methodological credential, co-authoring with established insiders, reframing their original expertise as a distinctive asset), the intervening conditions that help or hinder those strategies (the openness of the new field, the presence of a sponsor) and their consequences (acceptance, marginalisation, return to the original field). The output would be a dense, well-specified model of establishing credibility. Under constructivist grounded theory, the researcher would also be likely to start with credibility as a sensitising concept and would read the relevant literature beforehand, treating concepts such as boundary work or legitimate peripheral participation as possible points of departure. The researcher would pay close attention to how participants themselves define credibility, whose recognition they seek and why, and to how power, gender and institutional hierarchy shape who is allowed to cross. The researcher, perhaps a boundary crosser, would write reflexive memos about how that position shapes interviews and interpretation. The output would be an interpretive theory of how credibility is negotiated, situated in the particular fields and institutions studied. All three designs are legitimate. Each would produce a different kind of study with a different kind of contribution, and each would be judged by partly different criteria. What would not be legitimate is a fourth design that starts from a published model of academic identity, uses its constructs as coding categories, interviews a fixed sample of twenty people all at once, codes them after collection is complete, and presents the model's constructs as findings that "emerged." That study might be worth doing, but it would be a deductive qualitative study, and calling it grounded theory would be a misdescription. The question of mixing Can a study legitimately combine elements from more than one lineage? The honest answer is yes, within limits, and only when the combination is deliberate and explained. Some borrowings are uncontroversial. Many constructivist studies use Glaser's coding families during theoretical coding; Charmaz herself recommends treating them as possible analytic tools. Many studies in all traditions use Strauss and Corbin's analytic techniques, such as asking systematic questions of data or making comparisons with extreme cases, because these are generic aids to thinking rather than epistemological commitments. Using the coding paradigm as one heuristic among several in a constructivist study is defensible if the researcher explains why and remains willing to abandon it where it does not fit. Other combinations are incoherent. A study cannot claim that its theory emerged untouched by prior frameworks and also that it was co-constructed through reflexive interaction. It cannot claim to follow Glaser while using axial coding as a required stage. It cannot claim constructivist reflexivity while presenting its findings as objective discoveries validated by inter-rater reliability statistics. These combinations signal to informed readers that the researcher has assembled a methods section from citations rather than designed a study. The principle that separates legitimate from illegitimate mixing follows from this book's central argument. What matters is whether the chain of analytic decisions from data to theory is coherent and traceable, and whether the justification offered for it is consistent. A study may borrow a technique from another tradition if that technique can be justified within the study's own epistemological position. It may not borrow a justification from one tradition to cover a practice that only makes sense in another. A researcher who states a lineage, explains any borrowed tools, and shows that the resulting procedure hangs together has done what the method requires. A researcher who lists three founding texts and leaves the reader to reconcile them has not. Chapter 3. Designing a Study That Can Produce Theory Most grounded theory studies that fail do so before the first interview. They fail because the question was one that grounded theory cannot answer, because the design fixed in advance what the method needs to leave open, or because the researcher planned to collect all the data and then analyse it, which removes the mechanism that makes the method work. This chapter is about avoiding those failures. It covers when grounded theory is the right choice, how to frame a question that leaves room for discovery, how to begin collecting data, and how to present an emergent design to the committees, supervisors and funders who want everything specified in advance. Is grounded theory the right tool? Grounded theory is a method for generating explanatory theory about processes, typically social or social-psychological processes, in areas where existing theory is absent, thin or poorly fitted to the setting. Each part of that description is a test a proposed study should pass. Generating theory. If the aim is to test an existing theory, grounded theory is the wrong method. The method's logic is inductive and abductive: it moves from data to concepts and, as Chapter 6 explains, uses the emerging theory to direct further data collection. A researcher who already has a model and wants to know whether it holds should use a design built for confirmation, qualitative or quantitative. Explanatory. If the aim is to describe participants' experiences or the range of views on a topic, grounded theory is more machinery than the question needs. Thematic analysis, as systematised by Virginia Braun and Victoria Clarke in their widely cited 2006 paper in Qualitative Research in Psychology and in later work, is a flexible method for identifying and reporting patterns of meaning across a data set, and it does not require theoretical sampling or theory generation. Interpretative phenomenological analysis suits questions about how individuals make sense of significant lived experiences. Qualitative description suits applied questions where a clear, low-inference account is what practitioners need. There is no shame in choosing one of these. There is a real cost to choosing grounded theory and delivering a thematic description, because the study will be judged against a promise it did not keep. About processes. Grounded theory is at its best when the question concerns what people do over time: how they manage, cope, negotiate, adapt, decide, become, resist or transform. The method's focus on action and interaction, and its common output of a "basic social process" with stages or phases, reflect this. Questions about static attributes or opinions fit it poorly. Where existing theory is inadequate. The method's justification is that current theory does not explain the phenomenon well enough. That does not mean nothing has been written about it. It means that what has been written is too abstract to fit the setting, was developed in different contexts, or leaves the process itself unexplained. A practical test for mixed-discipline researchers is to state the question and then complete the sentence "At the end of this study I will be able to explain how..." If the sentence completes naturally with a process, such as "how software teams decide when a system is safe to release" or "how rural patients decide when a symptom is worth a long journey to a clinic", grounded theory is a candidate. If it completes awkwardly, or if the natural completion is "what the barriers are" or "whether the intervention works," another method probably fits better. Grounded theory should also be distinguished from two neighbours with which it is often confused in applied fields. Kathleen Eisenhardt's approach to building theory from case study research, set out in the Academy of Management Review in 1989, borrows heavily from grounded theory but is built around a small number of cases chosen for theoretical reasons and often combines qualitative and quantitative data with cross-case comparison and formal propositions. It is a legitimate method with its own literature, and researchers doing it should cite it rather than grounded theory. The Gioia methodology, described in Chapter 2, is similarly its own approach, influenced by grounded theory but organised around a particular presentational structure. Framing a question that leaves room for discovery A grounded theory research question is deliberately provisional. It identifies an area of inquiry and a kind of process, and it avoids specifying the concepts that the analysis is meant to discover. Consider three versions of a question about the illustrative study. "What strategies do interdisciplinary researchers use to overcome barriers to acceptance in a new field?" presupposes that there are barriers, that acceptance is the issue, and that the participants' response takes the form of strategies to overcome barriers. Every one of those presuppositions may be right, but the question has done the analysis before any data exist. "How do researchers who move between disciplines establish credibility in their new field?" is better for Straussian or constructivist designs: it names a phenomenon but leaves open its conditions, forms and consequences. "What is going on when researchers move from one discipline to another, and how do they handle it?" is the kind of question a classic grounded theorist would start with, since it does not even assume that credibility is what matters most to participants. The researcher's own interests and background will shape the question in all three cases. Classic grounded theory asks the researcher to hold those interests loosely and to be ready for the participants' main concern to be something else. Constructivist grounded theory asks the researcher to state them openly and examine how they shape the inquiry. What neither permits is a question that locks the study into a particular answer. Research questions in grounded theory also change during a study, and this should be expected rather than concealed. As analysis proceeds, a broad opening question narrows around the concepts that turn out to matter. A researcher who began by asking about credibility may end by theorising something more specific, perhaps how crossers manage the period in which they are expert in neither their old field nor their new one. Documenting that evolution in memos is part of the audit trail described in later chapters, and reporting it honestly strengthens the final paper. Mixed-discipline researchers bring an additional, less visible influence on framing: the habits of seeing that their home discipline has trained into them. These habits are assets, and they are also the most common source of forcing. An engineer tends to see systems, variables and failure modes, and may frame a question about clinical practice as a question about where a process breaks down. A clinician tends to diagnose, and may frame participants' accounts as symptoms of an underlying problem to be identified and treated. An economist tends to see incentives and trade-offs, and may hear every account of a decision as a story about costs and benefits. A computer scientist may reach for states and transitions. None of these lenses is wrong, and each can illuminate aspects of a process that a sociologist would miss. The danger is that the lens operates before the data have been heard, so that the question, the interview guide and the first codes are already shaped by it. The remedy is not to abandon disciplinary training, which is neither possible nor desirable, but to make it explicit early. A useful exercise before finalising the question is to write a short memo answering three prompts: what the researcher expects to find, what their discipline would treat as the obvious explanation, and what kind of finding would surprise them. That memo becomes a reference point. When, months later, the emerging theory looks suspiciously like the obvious explanation from the researcher's home field, it is a signal to check whether the data really carry it, or whether the researcher has found what their training told them to look for. When the theory contains surprises, that is often, though not always, a sign that the method is working. Interdisciplinary research teams face the same issue in a different form. A team composed of, say, a nurse, a data scientist and a sociologist will bring three lenses to the same data. Managed well, this is a strength, since the comparison of readings is itself a form of constant comparison and can surface concepts none of the team would have seen alone. Managed badly, the most senior or most confident discipline dominates, and the analysis becomes an exercise in fitting data to that discipline's categories. Teams should agree at the outset how disagreements about codes and categories will be recorded and resolved, and should treat recorded disagreements as analytic material rather than noise. Data, initial sampling and the first round Glaser's dictum "all is data" captures a principle every lineage shares: anything that helps the researcher understand the area under study may be used, including interviews, observations, documents, informal conversations, published statistics and existing literature. In practice most grounded theory studies rest mainly on interviews, often supplemented by observation or documents, and the choices made about data at the outset matter a great deal. Interviews for grounded theory are typically in-depth and loosely structured. Charmaz calls them intensive interviews: conversations that explore a participant's experience of the area under study in detail, following what the participant treats as significant, with questions that invite accounts of action and process. "Tell me about how you came to start working in this field" and "What happened next?" produce more usable data than "Do you feel accepted by your new colleagues?" An interview guide is useful, particularly for ethics review, but it should be written as a set of possible openings and prompts rather than a questionnaire, and it should be expected to change. By the tenth interview in a well-run study, the guide will typically look different from the first version, because theoretical sampling has begun to direct the questions towards emerging categories. Observation matters when the process under study is enacted in settings the researcher can access: wards, classrooms, meetings, construction sites, software stand-ups. Interviews give participants' accounts of what they do; observation shows what they do, including the things they do not mention because they take them for granted. Documents, such as policies, emails, meeting minutes, grant applications and online forum posts, can provide both data about the process and data about the institutional context. Glaser also insisted that quantitative data could be used for grounded theory, and mixed-discipline researchers with access to administrative or survey data should know that doing so is methodologically respectable when the data are treated as further material for comparison rather than as a test of the emerging theory. The first round of sampling is necessarily different from what follows. Before any analysis has happened there is no theory to guide theoretical sampling, so the researcher begins with what is usually called initial or purposive sampling: choosing people, settings or documents likely to be rich in the process of interest. For the illustrative study, that would mean finding researchers who have made a disciplinary move recently enough to remember it vividly and far enough back to have seen how it turned out. Initial sampling criteria should be broad enough to allow variation to appear. A first round that recruits only physicists who moved into biology will produce a theory of physicists moving into biology. The most important design decision at this stage concerns sequencing. Grounded theory requires that analysis begin after the first few data sources are collected and that it shape what is collected next. A study that schedules thirty interviews over three weeks, transcribes them all, and then begins coding cannot be a grounded theory study, however carefully the coding is done, because the constant comparative method and theoretical sampling have been designed out of it. The minimum practical requirement is a rhythm in which a small batch of data, often two to four interviews, is collected, transcribed promptly and coded, memos are written, and the next batch is planned in light of what the analysis has revealed. Researchers should budget time for this. Transcription delays are the most common practical reason that grounded theory studies drift into collect-then-analyse designs, and it is worth arranging transcription, or at least detailed notes and audio review, so that analysis can begin within days of each interview. The researcher should also start a memo file on the first day, before any data are collected. The earliest memos record the researcher's starting assumptions, expectations and interests. In a constructivist study they form the beginning of a reflexive record; in a classic study they are a way of noticing preconceptions so that they can be set aside. In all traditions they are the first link in the documentary chain that shows how the theory was built. Getting an emergent design past committees The emergent logic of grounded theory sits awkwardly with the review processes most researchers face. Ethics committees want to know in advance who will be recruited, how many, and what they will be asked. Doctoral panels want a literature review and a theoretical framework. Funders want sample sizes and timelines. None of these requirements is unreasonable, and none is fatal to grounded theory if handled openly. For ethics review, the approach that works in most institutions is to describe the emergent design explicitly and to bound it. The application can state that initial recruitment will target a defined population through defined channels; that subsequent recruitment will be guided by the emerging analysis but will remain within that population, or within named additional groups whose inclusion is foreseeable; that the interview guide provides the initial topics and that later interviews will explore emerging concepts within the same subject area; and that any substantial change, such as recruiting a new category of participant or introducing observation in a new setting, will be submitted as an amendment. Committees are usually far more comfortable with an emergent design that is candid about its boundaries than with one that pretends to a fixed protocol and then changes silently. On sample size, the proposal should give a realistic estimated range with a justification, and explain that the final number will be determined by theoretical considerations. Chapter 6 discusses what the evidence on sample sizes does and does not show, but a proposal can reasonably cite prior grounded theory studies of similar scope, and it can make clear that the stopping criterion is conceptual rather than numerical. On literature and theoretical frameworks, the proposal should distinguish between reviewing the literature to establish the problem and gap and adopting a framework to guide analysis. Doctoral panels are often satisfied by a literature review that shows what is known, why it does not explain the process in question, and which sensitising concepts the researcher brings, with an explicit statement that these will be tested against the data and discarded if they do not fit. For a classic grounded theory study, the review can be scoped to the problem and to methodology, with engagement with the substantive literature deferred and planned as a later chapter of the thesis. On timelines, the proposal should plan for the iterative rhythm described above and should include time for analysis during data collection, not only after it. A timeline that shows data collection ending before analysis begins will, rightly, alarm any reviewer who knows the method. One more design element deserves planning from the outset: how the chain from data to theory will be documented. The researcher should decide how codes, memos and analytic decisions will be recorded and stored, whether in qualitative data analysis software, in a structured set of word-processing files, or on paper, and should plan to keep dated memos and successive versions of the emerging theory. This is not bureaucratic tidiness. It is the evidence base on which the finished theory's credibility will rest, and it cannot be reconstructed convincingly after the fact. Hashtags: #GroundedTheoryForMixedDisciplineResearchers #GroundedTheory #QualitativeResearch #TheoryGeneration #ConstantComparativeMethod #TheoreticalSampling #MemoWriting #OpenCoding #AxialCoding #SelectiveCoding #FocusedCoding #TheoreticalCoding #GlaserianGroundedTheory #StraussianGroundedTheory #ConstructivistGroundedTheory #BarneyGlaser #AnselmStrauss #JulietCorbin #KathyCharmaz #TheoreticalSaturation #CoreCategory #Reflexivity #QualitativeMethodology #InterdisciplinaryResearch #FutureOfGroundedTheory