{
  "id": "external-validity",
  "title": "External Validity",
  "lang": "en",
  "source_lang": "en",
  "summary": "External validity is the extent to which a study’s findings support inferences about specified populations, settings, interventions, outcomes, or times beyond those directly studied.",
  "category": "mathematics",
  "subcategory": "probability-statistics",
  "type": "concept",
  "aliases": [
    "External research validity"
  ],
  "generated_at": "2026-11-05",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/external-validity/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/external-validity/",
      "json": "https://aiwiki.page/zh/external-validity.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "External validity is the extent to which findings from a study support inferences beyond the particular people, conditions, and procedures examined. In statistics, experimental design, and causal inference, it concerns whether a result applies to a specified target population or context. External validity is not an intrinsic, all-or-nothing property of a study: the same finding may justify an inference about one target and provide little evidence about another.\n\nFor example, an educational intervention evaluated among volunteers at a single university may establish an effect for those participants without establishing the same effect for all university students, younger pupils, or institutions with different teaching resources. Assessing external validity requires specifying both the original study conditions and the intended scope of the conclusion."
    },
    {
      "heading": "Distinctions from other forms of validity",
      "text": "External validity is commonly distinguished from several related concepts:\n\n- Internal validity concerns whether the study supports its conclusions within the circumstances investigated. For a causal study, this includes whether the estimated effect can reasonably be attributed to the intervention rather than to confounding, differential attrition, or other biases.\n- Construct validity concerns whether an intervention, measurement, or operational definition adequately represents the theoretical concept of interest. Generalizing from a particular test to “learning,” for example, depends partly on what that test measures.\n- Statistical conclusion validity concerns the adequacy of statistical evidence for an association or effect, including precision, model assumptions, and error control.\n- Ecological validity concerns correspondence with everyday conditions or behavior. The term has several uses and is not an exact synonym for external validity. A realistic study can still have a narrowly selected population, while an artificial task can reveal a mechanism that operates across many settings.\n\nThese dimensions are related but not interchangeable. A precise estimate from a poorly identified causal comparison does not become externally valid simply because its participants resemble the target population. Conversely, a tightly controlled experiment may identify a causal effect convincingly while leaving its applicability elsewhere uncertain."
    },
    {
      "heading": "Historical development",
      "text": "The modern distinction between internal and external validity was developed prominently in twentieth-century experimental methodology. Donald T. Campbell’s work in the 1950s and the subsequent work of Campbell and Julian C. Stanley helped establish it as a central framework for evaluating experimental and quasi-experimental research.\n\nLater methodological work differentiated statistical conclusion validity and construct validity from internal and external validity. Contemporary approaches increasingly formulate external-validity questions in terms of explicit target populations, causal assumptions, and differences between the processes that generate study and target data.\n\nIn this literature, generalizability often refers to extending findings from a study sample to a population of which it is a part, whereas transportability often refers to extending findings to a distinct population or setting. Usage varies across disciplines, so the intended target and assumptions are more informative than the terminology alone."
    },
    {
      "heading": "Dimensions of generalization",
      "text": "External validity can concern several dimensions at once.\n\nPeople and populations\n\nStudy participants may differ from the target population in age, prior knowledge, resources, motivation, or other characteristics. These differences matter especially when they modify the effect being estimated.\n\nRandom sampling and random assignment address different problems. Random sampling can support population inference through a known selection mechanism. Random assignment supports causal comparison among study participants by making treatment allocation independent of their pretreatment characteristics, subject to the design. A randomized controlled trial does not automatically constitute a representative sample of a wider population.\n\nSettings and implementation\n\nAn intervention may depend on institutional resources, staff experience, infrastructure, incentives, or background practices. Its effect under intensive supervision may differ from its effect under routine implementation.\n\nThe relevant intervention also includes its comparator. An improvement over no intervention need not imply an improvement over an established alternative. Changes in delivery, adherence, or available alternatives can therefore change the causal question rather than merely test the same effect elsewhere.\n\nOutcomes and measurements\n\nA result for one outcome does not automatically establish a result for another. Performance on a short-term assessment, for example, is not identical to long-term retention or practical application.\n\nMeasurement procedures may also behave differently across languages, groups, or institutions. Apparent differences in effects can arise from changes in what is measured, not only from changes in the underlying process.\n\nTime and scale\n\nEffects can change as technologies, institutions, or participant expectations change. Short-term effects may diminish, accumulate, or reverse over longer periods.\n\nScaling an intervention can introduce mechanisms absent from a small study. An employment program that helps a limited group obtain jobs may have different aggregate effects when expanded enough to affect competition for available positions. Such changes involve interactions and system responses, not simply sampling error."
    },
    {
      "heading": "A formal account of population generalization",
      "text": "External-validity questions can be expressed using potential outcomes. Let \\(A\\in\\{0,1\\}\\) denote treatment, \\(Y^a\\) the outcome that would occur under treatment \\(a\\), and \\(S\\) study participation, with \\(S=1\\) for participants and \\(S=0\\) for a target population of nonparticipants.\n\nA target average treatment effect is\n\n\\[\\tau_T=\\mathbb{E}[Y^1-Y^0\\mid S=0].\\]\n\nThe study may instead directly identify\n\n\\[\\tau_S=\\mathbb{E}[Y^1-Y^0\\mid S=1].\\]\n\nThese quantities need not coincide. Suppose \\(X\\) is a set of pretreatment characteristics and define the conditional study effect as\n\n\\[\\tau_S(x)= \\mathbb{E}[Y^1-Y^0\\mid X=x,S=1].\\]\n\nIf conditional average effects are the same in study and target populations,\n\n\\[\\mathbb{E}[Y^1-Y^0\\mid X=x,S=0]=\\tau_S(x),\\]\n\nthen the target effect can be written as\n\n\\[\\tau_T=\\mathbb{E}[\\tau_S(X)\\mid S=0].\\]\n\nThis expression averages the conditional study effects over the target population’s distribution of \\(X\\). It illustrates why the same conditional effects can produce different population-average effects when the populations have different compositions.\n\nUsing this formula requires more than algebra:\n\n- Identification within the study. The conditional causal effects must be estimable. In an observational study, this may require adequate control of treatment confounding.\n- Conditional effect comparability. After conditioning on \\(X\\), no remaining study–target differences may alter the conditional average effect relevant to this formula.\n- Population overlap. Characteristics occurring in the target must have adequate representation in the study.\n- Consistent intervention and outcome definitions. The treatment and outcome must retain the meaning required by the comparison.\n- Appropriate treatment-overlap and interference assumptions. Treatment comparisons must be supported within relevant study groups, and any effects between units must be accommodated by the causal specification.\n\nStronger assumptions, such as equality of each conditional potential-outcome mean across populations, are used in other formulations. The assumptions needed depend on the target quantity and available data.\n\nDifferences in average effects are closely related to effect modification. Whether effects are constant also depends on the scale: a constant risk ratio does not generally imply a constant risk difference."
    },
    {
      "heading": "Methods of assessment and estimation",
      "text": "Comparing study and target populations\n\nResearchers can compare eligibility criteria, participant characteristics, recruitment processes, settings, and implementation conditions. These comparisons are informative when guided by knowledge of plausible effect modifiers.\n\nDemographic similarity alone is insufficient. Two populations can resemble each other on recorded variables while differing on unmeasured determinants of treatment response. Conversely, some visible population differences may be irrelevant to the effect of interest.\n\nStandardization and weighting\n\nStandardization estimates conditional effects in the study and averages them over the target population’s characteristics. Age standardization is a familiar related technique for separating population composition from comparisons of rates.\n\nWeighting methods instead give study participants different contributions so that the weighted distribution of measured characteristics resembles the target distribution. Depending on the sampling arrangement, weights can involve participation probabilities, inverse odds of participation, or ratios of population densities.\n\nThese methods can address relevant measured differences, but they cannot manufacture evidence for groups absent from the study. Limited overlap can produce extreme weights, increased variance, and dependence on modeling assumptions.\n\nReplication and evidence synthesis\n\nStudies conducted across different populations, sites, and periods can reveal the stability and boundaries of a result. Meta-analysis can synthesize such evidence and examine variation in effect sizes, although the collection of available studies need not represent every setting of interest.\n\nA multisite study broadens observed conditions but does not establish universal applicability. Findings can still depend on how sites were selected, how interventions were delivered, and which contexts remain unobserved.\n\nSensitivity analysis\n\nSensitivity analysis examines how conclusions change under alternative assumptions about unmeasured effect modifiers, participation mechanisms, model specification, or implementation differences. It can clarify the strength of assumptions required for a generalization without proving those assumptions correct."
    },
    {
      "heading": "Interpretation and limitations",
      "text": "External validity has no universal numerical score. It is a relation between evidence, a target claim, and the assumptions connecting them.\n\nSeveral common interpretations are therefore misleading:\n\n- Increasing sample size improves precision under appropriate conditions but does not by itself correct selection bias or contextual mismatch.\n- A narrow confidence interval may describe sampling uncertainty while omitting uncertainty about whether an effect transports.\n- Statistical significance in one population and nonsignificance in another do not, by themselves, demonstrate different effects.\n- A representative sample does not automatically make a causal comparison valid.\n- A successful replication supports the conditions actually examined, not every possible extension.\n\nIn machine learning, related questions arise when predictions are used on data that differ from training data. Ordinary cross-validation mainly evaluates performance under the data-generating conditions represented by its splits; it does not by itself establish performance under a new institutional, geographic, or temporal distribution. For both prediction and causal inference, external validity ultimately depends on which features of the original setting remain stable and which change."
    }
  ],
  "links": [
    "statistics",
    "experimental-design",
    "causal-inference",
    "confounding",
    "random-sampling",
    "randomization",
    "randomized-controlled-trial",
    "statistical-interaction",
    "age-standardization",
    "variance",
    "meta-analysis",
    "effect-size",
    "sensitivity-analysis",
    "sample-size",
    "selection-bias",
    "confidence-interval",
    "machine-learning",
    "training-data",
    "cross-validation"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}