The empirical distribution function is the cumulative distribution function obtained by assigning equal probability to every observation in a sample. At any real number , it gives the proportion of observations less than or equal to . In statistics, it is a fundamental nonparametric estimator of a population distribution: it requires no assumption that the population belongs to a particular family, such as the normal or exponential distributions.
Definition and construction
For real-valued observations , where , the empirical distribution function is
where is an indicator function equal to if , and otherwise.
The definition applies to any finite sample. Its usual statistical properties assume that the observations are independent and identically distributed, with common population distribution function
Before the data are observed, is a random function; after observation, it is a fixed step function.
Example
For the sample ,
The jump at has size , because two observations equal . In particular, : observations equal to the threshold are included.
Structural properties
Every empirical distribution function is:
- nondecreasing;
- right-continuous;
- equal to below the smallest observation;
- equal to at and above the largest observation.
At a distinct observed value , its jump is
where denotes the left-hand limit. Thus, an observation appearing times produces a jump of size .
If the sorted observations, or order statistics, are denoted by
then, in the absence of ties,
With ties, some intermediate levels are skipped.
The empirical distribution is discrete even when the population distribution is continuous. Consequently, the ordinary derivative of the ECDF is zero between jumps and undefined at jumps; it is not a probability density function representing the empirical distribution.
Empirical measure and sample summaries
The distribution underlying the ECDF is the empirical measure
where assigns probability to the single point . The ECDF is its cumulative representation:
For a real-valued function that is finite at the observations,
Thus, integration against the empirical distribution turns population expectations into sample averages. For example, its expected value is the sample mean,
Its variance is
This is the variance of the empirical probability distribution, not the usual unbiased estimator of population variance, which uses denominator .
The ECDF also determines empirical interval probabilities:
The interval endpoints matter because the empirical distribution places positive probability on observed values.
Sampling properties
Assume throughout this section that the observations are independent and identically distributed with distribution function .
Pointwise expectation and variance
For each fixed , the count has a binomial distribution:
It follows that
Hence the ECDF is pointwise unbiased. Its standard error at is at most , with the maximum attained when .
Values of the ECDF at different thresholds are dependent. Their covariance is
Pointwise convergence and normal approximation
By the law of large numbers, for each fixed ,
When , the central limit theorem gives
This is a statement about a fixed threshold. It does not by itself provide simultaneous uncertainty control across all thresholds.
Uniform convergence and uncertainty bands
Glivenko–Cantelli theorem
The Glivenko–Cantelli theorem, established in 1933, strengthens pointwise convergence to uniform convergence:
Thus, with probability one, the largest discrepancy anywhere between the empirical and population distribution functions tends to zero. The result applies to continuous, discrete, and mixed distributions.
Dvoretzky–Kiefer–Wolfowitz inequality
The Dvoretzky–Kiefer–Wolfowitz inequality, with its sharp constant, provides a finite-sample bound:
For , define
With probability at least , the inequalities
hold simultaneously for every . These are simultaneous confidence bands, rather than separate pointwise confidence intervals.
Empirical-process limit
At the process level, the centered and rescaled ECDF
converges in distribution, in the standard empirical-process formulation, to
where is a Brownian bridge: a mean-zero Gaussian process on with covariance
This functional limit describes fluctuations of the entire estimated distribution and underlies many distributional tests.
Applications
Distribution comparison and goodness-of-fit testing
An ECDF plot displays location, spread, ties, and tail behavior without choosing histogram bins. Multiple ECDFs can be plotted together to compare samples.
The one-sample Kolmogorov–Smirnov test uses the maximum discrepancy from a specified distribution function :
For an independent sample under a fully specified continuous null distribution, its null distribution does not depend on the particular form of . This distribution-free property generally does not hold unchanged for discrete null distributions or when parameters of are estimated from the tested sample.
The two-sample version compares two ECDFs, and , through
Quantile estimation
A natural empirical quantile function is the generalized inverse
It satisfies
Software also uses interpolated quantile definitions. These may return values between observations and need not coincide with the generalized inverse of the ECDF.
Bootstrap resampling
In the ordinary nonparametric bootstrap, observations are sampled independently with replacement from the empirical distribution. Conditional on the original data, each indexed observation receives probability . Repeated observed values therefore receive probability equal to their relative frequency.
This makes the empirical distribution a substitute for the unknown population distribution when approximating the sampling behavior of many statistics. Bootstrap validity still depends on the statistic and the sampling assumptions.
Variants and limitations
A weighted empirical distribution function has the form
for nonnegative weights with positive total weight. Such functions arise with unequal sampling probabilities and importance weighting. Their uncertainty is not generally described by the ordinary unweighted formulas.
For vector-valued observations, a multivariate ECDF counts observations whose every coordinate is at most the corresponding threshold. It estimates a joint distribution, but its geometry and distributional theory differ from the univariate case.
Several limitations are important:
- Dependence: serially or otherwise dependent observations invalidate the simple binomial variance formula and can change convergence rates and limiting distributions.
- Sampling bias: the ECDF describes the observed sample; it does not automatically correct selection bias, measurement error, or missing data.
- Tail resolution: it assigns no probability below the observed minimum or above the observed maximum. Reliable extrapolation beyond the sample range requires additional assumptions.
- Censoring and truncation: applying the ordinary ECDF to incompletely observed values generally does not estimate the distribution of the underlying uncensored variable.
- Lack of smoothness: the ECDF estimates cumulative probabilities directly, not a smooth density. Density estimation requires a different construction, often with additional smoothing choices.
Computationally, sorting the sample permits a compact representation by distinct observed values and their cumulative counts. Subsequent evaluations can use binary search, while empirical quantiles can be obtained directly from the sorted observations.