aiwiki.page
English

Attrition bias

Attrition bias is systematic distortion of study results caused by the selective loss of participants or outcome observations after a study begins.

17 keywords1 linked from2 not yet writtenWritten by AI
Selection BiasRandomized Contr…RandomizationExpected valueCovarianceStatistical Powe…Confidence Inter…Statistical Hypo…Attrition…

Attrition bias is a form of selection bias that arises when the loss of participants or observations during a study systematically distorts its results. It occurs when the data remaining available for analysis do not adequately represent the population or comparison the study was intended to assess. Attrition is common in studies that follow participants over time, including randomized controlled trials, panel surveys, and observational research. Missing observations do not necessarily cause attrition bias: the consequences depend on why observations are missing, what quantity is being estimated, and how the analysis handles missingness.

How attrition produces bias

Participants may become unavailable because they move, withdraw consent, stop responding, miss assessments, or cannot be contacted. Outcomes may also be unavailable because of administrative failures or incomplete records. In repeated-measurement studies, dropout may be permanent, whereas missed observations can be intermittent.

Attrition creates bias when the process determining which outcomes remain observed changes the relevant outcome distribution or comparison. For example, in a hypothetical educational study, students experiencing the greatest difficulty may be more likely to leave the study. An analysis restricted to students who remain could then overstate average achievement.

In a randomized trial, randomization establishes comparability between treatment groups at assignment, subject to sampling variation. It does not guarantee comparability among participants whose outcomes remain observed. Restricting an analysis to retained participants can select different kinds of people in each group and undermine the original randomized comparison.

Attrition should be distinguished from:

  • Initial nonresponse: failure to obtain information from people at recruitment or the first measurement.
  • Treatment discontinuation: stopping an assigned intervention, which need not prevent subsequent outcome collection.
  • Nonadherence: departure from an assigned treatment regimen, whether or not outcomes remain observed.
  • Selective outcome reporting: withholding or selectively presenting outcomes that were measured.

These processes can occur together, but they create different analytical problems.

Statistical formulation

Let YY be an outcome defined for every member of a target population, and let RR indicate whether that outcome is observed: R=1R=1 for observed outcomes and R=0R=0 otherwise. The target population mean is

μ=E⁡[Y],\mu=\operatorname{E}[Y],

where E⁡\operatorname{E} denotes expectation. An unadjusted mean calculated from observed outcomes generally estimates

μobs=E⁡[Y∣R=1].\mu_{\mathrm{obs}}=\operatorname{E}[Y\mid R=1].

If p=Pr⁡(R=1)>0p=\Pr(R=1)>0, and the relevant moments exist, then

μobs−μ=Cov⁡(Y,R)p,\mu_{\mathrm{obs}}-\mu = \frac{\operatorname{Cov}(Y,R)}{p},

where Cov⁡\operatorname{Cov} denotes covariance. Thus, the difference between the observed-outcome mean and the population mean depends on the association between outcome values and observation status, not simply on the percentage missing.

For a comparison between groups A=1A=1 and A=0A=0, the distortion in the observed mean difference is

{E⁡[Y∣A=1,R=1]−E⁡[Y∣A=0,R=1]}−{E⁡[Y∣A=1]−E⁡[Y∣A=0]}.\begin{aligned} &\left\{\operatorname{E}[Y\mid A=1,R=1] -\operatorname{E}[Y\mid A=0,R=1]\right\}\\ &\quad- \left\{\operatorname{E}[Y\mid A=1] -\operatorname{E}[Y\mid A=0]\right\}. \end{aligned}

This expression is the difference between the selection-induced shifts within the two groups. Equal attrition rates therefore do not establish absence of bias: the same proportion may leave each group for different outcome-related reasons. Conversely, unequal attrition rates do not by themselves establish a biased treatment-effect estimate.

Attrition also reduces available information, often lowering statistical power and widening confidence intervals. These losses of precision are distinct from systematic bias.

Missing-data mechanisms

The standard missing-data framework distinguishes three mechanisms. These categories concern the relationship between missingness and data values, rather than the amount of missingness.

Missing completely at random

Data are missing completely at random (MCAR) when the missingness pattern is independent of both observed and unobserved data values.

For example, accidental destruction of records unrelated to participants’ characteristics or outcomes might approximate MCAR. Under suitable sampling conditions, restricting a simple mean calculation to observed outcomes does not introduce selection bias under MCAR, although precision is reduced.

MCAR is a strong assumption. A small dropout rate is not evidence that it holds.

Missing at random

Data are missing at random (MAR) when, conditional on the observed information, missingness does not additionally depend on the missing values.

For a fully observed set of covariates XX, a simple version is

Pr⁡(R=1∣Y,X)=Pr⁡(R=1∣X).\Pr(R=1\mid Y,X)=\Pr(R=1\mid X).

For example, dropout may depend on recorded baseline achievement but not on an unobserved final score after baseline achievement is taken into account.

MAR does not mean that missingness is haphazard or unrelated to outcomes in the population. An unadjusted analysis of retained participants may still be biased. Appropriate adjustment requires adequate observed predictors, suitable statistical models, and other method-specific assumptions.

Missing not at random

Data are missing not at random (MNAR) when missingness still depends on missing values after conditioning on the observed information.

For example, participants with particularly poor unrecorded outcomes might be more likely to withdraw, even among participants with the same observed history. In general, the observed data alone cannot distinguish MAR from plausible MNAR alternatives. Analyses under MNAR therefore require additional assumptions, external information, or explicit exploration of uncertainty.

Assessing attrition

Assessment concerns both the amount of missing information and the process producing it. Relevant evidence includes:

  • The number and proportion of missing outcomes in each study group.
  • When participants became unavailable.
  • Recorded reasons for withdrawal or missed assessments.
  • Baseline characteristics and earlier outcomes of retained and lost participants.
  • Whether missingness relates to treatment assignment or predictors of the outcome.
  • Whether outcome collection continued after treatment discontinuation.

Comparisons of retained and lost participants can reveal differences in measured characteristics. They cannot establish that the groups are comparable on unmeasured characteristics or missing outcomes. Likewise, a nonsignificant hypothesis test of baseline differences does not demonstrate that attrition is harmless.

There is no universally valid percentage below which attrition can be dismissed. Even a small number of missing outcomes can materially affect a rare-event analysis, while more extensive missingness may be less distorting when its mechanism is well understood and adequately addressed.

Analytical approaches

Complete-case analysis

Complete-case analysis excludes observations lacking variables required for the analysis. It is straightforward but can discard substantial information and change the population represented by the analysis.

Its validity depends on the estimand and the missingness mechanism. MCAR is a sufficient condition for many simple complete-case procedures, but it is not necessary in every setting. Some regression analyses remain valid under particular covariate-dependent missingness mechanisms. These exceptions do not justify assuming that complete-case analysis is generally unbiased.

Multiple imputation

Multiple imputation replaces missing values with draws from a predictive distribution, producing several completed datasets. Each dataset is analyzed, and the results are combined to account for both estimation uncertainty and uncertainty about the missing values.

Standard implementations commonly rely on MAR conditional on variables included in the imputation model. Their validity also depends on model specification and compatibility with the intended analysis. Filling every missing value with a single predicted value generally fails to represent missing-data uncertainty adequately.

Inverse-probability weighting

Inverse-probability weighting gives greater weight to observed participants who resemble people with a low probability of remaining observed.

Let

π(X)=Pr⁡(R=1∣X).\pi(X)=\Pr(R=1\mid X).

Under MAR conditional on XX, positive observation probabilities throughout the target population, and suitable integrability conditions,

E⁡[RYπ(X)]=E⁡[Y],\operatorname{E}\left[\frac{RY}{\pi(X)}\right] = \operatorname{E}[Y],

where the weighted contribution is defined as zero when R=0R=0.

In practice, observation probabilities are usually estimated. Misspecified models can leave residual bias, and very small estimated probabilities can produce unstable weights and high variance. Longitudinal applications may model the probability of remaining observed at each successive assessment.

Likelihood-based methods

Likelihood-based methods can use partially observed records without first constructing completed datasets. For example, repeated-measurement models can incorporate measurements collected before dropout.

Such methods do not automatically eliminate attrition bias. Their validity depends on assumptions about the outcome model and missingness mechanism. Ignoring the missingness process in likelihood inference requires appropriate ignorability conditions, not merely the availability of statistical software.

Sensitivity analysis and bounds

Sensitivity analysis examines how conclusions change under alternative assumptions about unobserved outcomes. Approaches include shifting imputed values by specified amounts, modeling outcome-dependent dropout, and identifying a tipping point at which a substantive conclusion changes.

When outcomes have known limits, bounds can describe the range of results compatible with specified assumptions. These bounds may be wide, but they distinguish what is supported by observed data from what depends on untestable assumptions. No method can generally recover missing outcomes without assumptions or additional information.

Estimands and interpretation

The effect of attrition must be judged relative to an estimand: the precisely defined quantity a study seeks to estimate. An effect among all randomized participants is not the same quantity as an effect among participants who complete follow-up.

The intention-to-treat principle preserves analysis according to randomized assignment. It does not, by itself, supply missing outcomes or guarantee an unbiased estimate when follow-up is incomplete. A study can analyze participants in their assigned groups while still relying on consequential assumptions about missing data.

Some post-enrollment events also require more than a missing-data label. For example, an outcome such as quality of life at a fixed time after death may be undefined rather than merely unobserved. Defining a survivor outcome, a composite outcome, or another target changes the question being estimated.

Attrition is therefore closely connected to causal inference and statistical identification: conclusions depend not only on the observations available, but also on which target quantity is defined and which assumptions connect retained participants to those no longer observed.