Effective sample size (ESS) is a measure in statistics that translates the precision of an estimate into an equivalent number of observations under a specified reference sampling scheme. It is commonly used when observations are correlated, carry unequal weights, or arise from a complex sampling design. A dataset containing observations may provide substantially less information than independent, equally weighted observations, although some designs and dependence structures can produce an effective sample size greater than . ESS is not a single universal quantity: its definition depends on the estimator, the reference experiment, and the aspect of precision being measured.
Variance-based definition
For independent observations from the same distribution, each with variance , the sample mean has variance
If an estimator estimates the same population mean under a different sampling scheme, a natural variance-matching definition is
Thus, if an estimator has variance , its effective sample size is 100 relative to independent sampling from that population.
More generally, suppose the reference estimator based on independent observations has variance approximately . Then
Here depends on the target quantity and the reference estimator. Consequently, the same observations can have different effective sample sizes for a mean, a proportion, a regression coefficient, or a quantile.
This interpretation concerns sampling precision, not every aspect of information. In particular, it does not account directly for bias. A highly precise estimate can still be systematically wrong, so a large ESS does not by itself imply small mean squared error.
Unequal observation weights
Kish effective sample size
Suppose observations have a common mean, common variance , and are independent. For fixed, nonnegative weights , with at least one positive weight, consider the weighted mean
Its variance is
Matching this to gives the Kish effective sample size:
For normalized weights
the same expression becomes
Multiplying all weights by the same positive constant leaves ESS unchanged. Equal weights give ; a single nonzero weight gives . For nonnegative weights,
with the upper bound following from the Cauchy–Schwarz inequality.
For example, normalized weights give
Although four observations are present, their weighted average has the same variance as approximately 1.92 independent, equally weighted observations under the stated assumptions.
Relationship to weight variability
Define the coefficient of variation of the weights using divisor :
Then
This identity expresses how concentration of weight reduces the effective number of contributing observations.
Its interpretation requires care. If weights depend on observed outcomes, observations have different variances, or observations are dependent, the weight-only formula need not equal a variance-matched ESS. For fixed normalized weights and covariance matrix , the general variance is
which cannot generally be inferred from weight dispersion alone.
Complex survey sampling
In survey sampling, precision depends on features such as clustering, stratification, unequal selection probabilities, and weighting adjustments.
The design effect for an estimator is
When the reference variance for a nominal sample of size scales as , a corresponding effective sample size is
The reference is commonly simple random sampling, but whether sampling is with or without replacement, and whether a finite-population correction is included, matters. If the reference variance does not scale exactly as , the expression is an approximation or a reporting convention rather than an exact variance match.
Clustered observations
Under a simple model with equal cluster size , equal individual variances, common within-cluster correlation , and independent clusters, the design effect for the overall mean is
Hence
For example, 1,000 observations arranged in independent clusters of 10, with within-cluster correlation , have
The result does not extend unchanged to unequal cluster sizes or arbitrary correlation structures.
Clustering with positive within-cluster correlation usually reduces precision relative to independent sampling. Effective stratification can instead reduce variance below the reference variance, giving and . This is greater efficiency relative to the chosen reference, not an increase in the actual number of observations.
Correlated sequences
For observations from a stationary process, the variance of a mean depends on correlations across time. Let each observation have variance , and define the lag- autocorrelation as
The exact variance of the mean of consecutive observations is
The corresponding effective sample size is therefore
This expression shows that correlations at many lags can matter; adjusting only for the correlation between neighboring observations is generally insufficient.
When the autocorrelations are absolutely summable and an appropriate central limit theorem holds, the large-sample approximation is
Here is the integrated autocorrelation time under a convention in which independent observations have .
Positive serial correlation generally reduces ESS. Negative correlation can make , producing ESS greater than the number of observations.
First-order autoregressive example
For a stationary first-order autoregressive model with , where ,
so
If , the effective sample size is approximately . This is a large-sample approximation; the finite-sample formula retains the factors .
Markov chain Monte Carlo
In Markov chain Monte Carlo (MCMC), successive simulation draws are usually dependent. ESS describes the precision of a Monte Carlo estimate relative to independent draws from the target distribution.
For a target function , define
Under suitable stationarity and limit-theorem conditions,
where is the autocorrelation of the transformed sequence . The Monte Carlo standard error is then approximately
This dependence on is essential. A chain may estimate a mean efficiently while estimating tail probabilities poorly.
Practical diagnostics distinguish bulk ESS, commonly calculated from rank-normalized draws, from tail ESS, which assesses sampling efficiency in distributional tails. These diagnostics need not equal the ESS for the original-scale mean.
Autocorrelations must be estimated from finite runs. Summing noisy estimates over many lags is unstable, so software uses truncation, paired-sequence methods, or other long-run variance estimators. Multi-chain methods can also account for disagreement between chains.
ESS alone does not establish convergence to the posterior distribution or another target. A chain trapped in one region can yield an apparently favorable autocorrelation-based ESS while failing to explore important regions. Convergence diagnostics and ESS address related but distinct questions.
Importance sampling and particle methods
In importance sampling, draws from a proposal distribution are assigned weights to represent a target distribution. For normalized weights , a common diagnostic is
This measures weight concentration and is algebraically identical to the Kish formula. It approaches for nearly uniform weights and approaches 1 when one weight dominates.
However, it is not generally the exact ESS for an importance-sampling estimate. For independent proposal draws , normalized target density , and weights , the self-normalized estimate of
has, under suitable moment and support conditions, asymptotic variance
Its variance-matched ESS therefore depends on both the weights and the integrand . A weight-only diagnostic cannot capture that dependence.
In sequential Monte Carlo, weight-based ESS is often used to trigger resampling. Resampling can restore equal weights, but it may create repeated particles sharing the same ancestors. An ESS equal to the particle count immediately after resampling does not imply that all particles contain independent information.
Interpretation and limitations
An ESS value is meaningful only with its definition and reference specified.
- It is estimator-specific. Means, quantiles, tail probabilities, and model parameters can have different effective sample sizes.
- It is not necessarily an integer. ESS is an equivalence measure, not a literal count of retained observations.
- It need not be bounded by nominal sample size. The nonnegative-weight Kish formula is bounded by , whereas variance-based definitions can exceed .
- It does not correct systematic error. Selection bias, model misspecification, and incomplete simulation exploration are not removed by reporting ESS.
- It is itself estimated. Short sequences, slowly decaying dependence, and unstable weights can make ESS estimates unreliable.
- It is not automatically a degrees-of-freedom adjustment. Replacing nominal sample size with ESS in a test or confidence interval does not by itself establish the correct reference distribution or coverage.
- Its standard interpretation can fail without finite variances. Heavy-tailed importance weights or dependence that prevents an ordinary central limit theorem may invalidate conventional variance-based ESS calculations.
A reproducible report identifies the estimator or diagnostic being assessed, the nominal sample size, the reference sampling scheme, and the method used to calculate ESS.