Natural Experiment

A natural experiment is a situation in which an event, institutional rule, or policy creates differences in exposure that researchers use to investigate cause and effect, although they do not control the assignment of that exposure. Unlike a researcher-designed experiment, the variation arises outside the study itself. “Natural” therefore does not mean that the cause must be a natural phenomenon: legislation, administrative decisions, and lotteries can also generate natural experiments. The central question is whether the resulting comparisons support credible causal inference, rather than merely documenting an association. (ukri.org)

Definition and scope

The term has no universally accepted boundary. In a narrow usage, a natural experiment requires an assignment mechanism that is plausibly as if random: exposed and unexposed units differ for reasons unrelated to their potential outcomes. In a broader usage, common in population-health research, it includes externally introduced interventions whose effects are investigated using a defensible comparison design, even when exposure is not approximately randomized. (ukri.org)

Natural experiments consequently overlap with quasi-experiments, but the terms are not exact synonyms. A quasi-experiment may be deliberately organized by researchers without random assignment; a natural experiment emphasizes the external origin of the variation. Both differ from a randomized controlled trial, in which random assignment is deliberately built into the research design. A natural experiment can nevertheless exploit genuine randomization conducted for a nonresearch purpose, such as an administrative lottery. (ukri.org)

The label describes a source of variation, not a guarantee of validity or a particular statistical technique. A policy change is not automatically a useful natural experiment: it must create a comparison capable of separating its effect from other influences. (nber.org)

Logic of causal identification

The fundamental difficulty is counterfactual: the same person, organization, or region cannot simultaneously be observed both exposed and unexposed under otherwise identical circumstances. Researchers therefore seek other units, or observations from other periods, that can represent what would have happened without the exposure. Differences in outcomes are causally interpretable only when the comparison has a justified relationship to that missing counterfactual. (nobelprize.org)

Ordinary correlation may be misleading because of confounding. For example, people with more schooling may differ from people with less schooling in family resources or other characteristics that also affect earnings. A natural experiment seeks variation in schooling generated by an external rule or event, rather than relying on all observed differences in educational attainment. (nber.org)

A persuasive analysis explains:

what generated the exposure differences;
which units were affected and which provide the comparison;
why the assignment mechanism supports that comparison;
which causal effect can be identified;
what alternative explanations could invalidate the result.

This emphasis on explicit, scrutinizable assumptions distinguishes the natural-experiment approach from treating a fitted statistical model as sufficient evidence of causality. (nber.org)

Principal analytical designs

Externally administered lotteries

Lotteries used to allocate access, eligibility, or obligations can create randomized comparisons without researchers arranging the allocation. Assignment and actual participation must be distinguished: some eligible people may not participate, while some ineligible people may obtain the exposure through another route. The causal effect of eligibility is therefore not necessarily the causal effect of participation. (nber.org)

Difference-in-differences

Difference-in-differences compares the change in an exposed group with the change in a comparison group:

[ \widehat{\tau}_{\mathrm{DiD}}

(\bar Y_{T,\mathrm{after}}-\bar Y_{T,\mathrm{before}})

(\bar Y_{C,\mathrm{after}}-\bar Y_{C,\mathrm{before}}). ]

Here, (T) denotes the treated group, (C) the comparison group, and (\bar Y) their mean outcomes. The central parallel-trends assumption is that, without treatment, their average outcomes would have changed similarly. Their initial outcome levels need not be equal. When different groups adopt a policy at different times, conventional regression estimates can become difficult to interpret, particularly when treatment effects change over time. (nber.org)

Regression discontinuity

A regression discontinuity design exploits a threshold determining treatment or eligibility, such as an admissions score. Under appropriate continuity assumptions, units immediately above and below the threshold provide a local comparison. Important threats include precise manipulation of the assignment variable and other changes occurring at the same cutoff. The resulting effect ordinarily applies at or near the threshold, rather than to the entire population. (arxiv.org)

Instrumental variables

Instrumental variables use an external variable that changes exposure to identify an exposure’s causal effect. A valid instrument must affect treatment, have an appropriate independence relationship with potential outcomes, and affect the outcome only through treatment—the exclusion restriction. These conditions require substantive justification; they do not follow merely because an instrument originates outside the study. (nber.org)

With a binary instrument, imperfect participation, and additional assumptions including monotonicity, the analysis can identify a local average treatment effect: the average effect for people whose participation changes because of the instrument. It need not equal the effect for everyone exposed or for the population as a whole. (nber.org)

Historical development and examples

John Snow’s investigation of cholera in nineteenth-century London is frequently cited as an early natural-experiment study. The relevant tradition uses externally generated differences in environmental exposure to investigate disease causes, rather than deliberately assigning harmful exposures. Natural experiments subsequently became important in epidemiology and public health, including evaluations of environmental regulation and population-wide interventions. (ukri.org)

In economics, influential work used changes in state laws, military draft mechanisms, and other institutional arrangements to obtain plausibly exogenous variation. This approach placed increasing emphasis on the assignment process and on transparent assumptions supporting causal interpretation. (nber.org)

A prominent example is the study by David Card and Alan Krueger of New Jersey’s minimum-wage increase on April 1, 1992. They surveyed 410 fast-food restaurants in New Jersey and Pennsylvania before and after the change, using Pennsylvania, where the minimum wage remained unchanged, as a comparison. The case illustrates a policy-generated difference-in-differences design rather than researcher-controlled assignment. (nber.org)

The 2021 Nobel Memorial Prize in Economic Sciences recognized Card’s empirical contributions to labour economics and Joshua Angrist and Guido Imbens’s methodological contributions to the analysis of causal relationships. The award highlighted both the use of natural experiments and the clarification of which causal conclusions they permit. (nobelprize.org)

Strengths, limitations, and interpretation

Natural experiments expand the range of questions that can be investigated when deliberate randomization is impractical, unethical, or incompatible with how an intervention is implemented. Applications include national legislation, education policies, transport infrastructure, and environmental exposures. Their usefulness depends on the particular opportunity and available evidence, not simply on the absence of a feasible randomized trial. (ukri.org)

Several limitations recur:

Selective exposure: affected and comparison units may differ in outcome-relevant ways.
Concurrent changes: another intervention or event may explain the observed difference.
Measurement problems: inconsistent records or changes in data collection may resemble treatment effects.
Local interpretation: threshold-based and instrumental-variable estimates may concern only a particular subgroup.
Limited generalizability: a credible effect in one institutional setting need not transfer unchanged to another. (nber.org)

Assessment therefore combines institutional knowledge with empirical checks: examining preintervention patterns, considering alternative comparison groups, testing outcomes not expected to respond, and comparing results across methods with different assumptions. Such checks can expose weaknesses but cannot establish every identifying assumption. Studies should distinguish uncertainty about the assignment mechanism from sampling uncertainty and state precisely which population and intervention the estimate describes. (ukri.org)