Convergence in distribution is a mode of convergence in probability theory in which the probability distributions of a sequence of random variables approach a limiting distribution. It describes the limiting probabilities of outcomes rather than requiring the random variables themselves to become close. For real-valued random variables, it means that their cumulative distribution functions converge at every continuity point of the limiting distribution function. This notion is fundamental to limit theorems and to distributional approximations in statistics.
Definition
Let and be real-valued random variables, with cumulative distribution functions
The sequence converges in distribution to if
for every at which is continuous. Standard notation includes
Only the distributions matter. The variables need not be defined on the same probability space, and replacing any of them with another variable having the same distribution leaves the convergence statement unchanged. Equivalently, if and denote their laws, one writes
This is called weak convergence of probability measures. Here, “weak” refers to testing measures against suitable functions; it does not mean convergence to a generalized function in distribution theory.
Why continuity points are required
Suppose deterministically and . Then , but
The limiting distribution has an atom at zero, so its distribution function is discontinuous there. Requiring convergence at every real number would incorrectly exclude this sequence.
If is continuous everywhere, however, convergence in distribution implies the stronger statement
Thus, for a continuous limiting distribution function, the convergence of distribution functions is uniform.
Equivalent characterizations
Bounded continuous test functions
For real-valued random variables, convergence in distribution is equivalent to
for every bounded continuous function .
This formulation expresses convergence through averaged observations. Boundedness ensures that the relevant expectations exist; continuity prevents a test function from detecting an unstable distinction at a boundary carrying limiting probability.
It also extends the definition to random elements in more general spaces. If is a metric space and are probability measures on its Borel sigma-algebra, weak convergence is defined by
for every bounded continuous real-valued function on .
For vectors in Euclidean space, convergence of multivariate distribution functions at every continuity point of the limiting distribution function gives an equivalent criterion.
Portmanteau theorem
The Portmanteau theorem collects equivalent formulations of weak convergence. For Borel probability measures on a metric space, is equivalent to each of the following conditions:
For every closed set ,
For every open set ,
For every Borel set whose boundary satisfies ,
A set satisfying the boundary condition is called a continuity set of . This qualification is essential: weak convergence does not imply convergence of probabilities for every measurable set.
For example, for real numbers ,
whenever has no atoms at or .
Characteristic functions
The characteristic function of a real-valued random variable is
Lévy’s continuity theorem states that
More generally, if characteristic functions converge pointwise to a function that is continuous at zero, then is the characteristic function of a probability distribution, and the corresponding laws converge weakly to that distribution. This criterion is especially useful for sums of independent variables because their characteristic functions multiply.
Relationship to other modes of convergence
For variables defined on a common probability space, the principal implications are
Almost sure convergence concerns convergence outside a probability-zero set of outcomes. Convergence in probability requires
for every . Convergence in distribution is weaker: it constrains only the individual laws, not how the variables are coupled.
For example, let have the Bernoulli distribution with parameter , and set for every . Each has exactly the same distribution as , so . Nevertheless,
and convergence in probability to fails.
An important exception occurs when the limiting distribution is concentrated at a constant :
A constant can be regarded as a random variable on any probability space supporting .
Fundamental examples
Central limit theorem
A principal example is the central limit theorem. If are independent and identically distributed with mean and finite, positive variance , then
The limiting normal distribution provides an approximation to probabilities involving the standardized sum. If the original variables are discrete, the standardized sums remain discrete for every finite ; convergence in distribution does not require the approximating and limiting laws to have the same type.
Poisson limit
Suppose has the binomial distribution with parameters and , where
Then
This gives the Poisson distribution as a limit for counts of many independent events whose individual probabilities become small.
Discrete laws approaching a continuous law
Let assign probability to each point
Then , where has the uniform distribution on . For every bounded continuous ,
Yet if , then
This illustrates why weak convergence cannot guarantee convergence of probabilities for arbitrary measurable sets.
Operations that preserve convergence
Continuous mapping theorem
The continuous mapping theorem states that if and is continuous, then
More generally, a measurable is sufficient if its set of discontinuities has probability zero under the law of .
For example,
For the reciprocal map, the corresponding conclusion requires that the limiting variable have no mass at zero, with an appropriate definition of the map there.
Slutsky’s theorem
Slutsky’s theorem states that if
where is constant, then
If , it also gives
provided the ratio is defined, or assigned a value on events whose probabilities tend to zero.
This permits unknown scale parameters in asymptotic statistics to be replaced by consistent estimates.
Joint convergence is a separate requirement
Marginal convergence does not generally imply joint convergence. Suppose takes values and with equal probabilities, and define
Both marginal distributions are constant in , but the pair alternates between two different joint distributions. In particular, alternates between having the law of and being identically zero.
Consequently, applying a continuous function to several converging variables generally requires convergence of the whole vector, not merely convergence of each coordinate’s distribution.
Limitations
Moments need not converge
Convergence in distribution does not imply convergence of means, variances, or other unbounded moments. Consider
Then in probability and hence in distribution, but
for every .
The rare, large values disappear from the limiting distribution while retaining their contribution to the expectation. Additional control, such as uniform integrability, is needed to infer convergence of expectations.
Densities need not converge or exist
Weak convergence does not imply pointwise convergence of probability densities, and the limiting law need not possess a density. For example,
converges in distribution to a point mass at zero. Conversely, the discrete uniform-grid example has a continuous limiting distribution despite having no densities with respect to Lebesgue measure at any finite stage.
No convergence rate is specified
The statement is asymptotic. It does not provide an error bound for a particular finite . Quantitative results, such as bounds on the difference between distribution functions, require additional assumptions and separate theorems.
Tightness and process limits
A family of probability measures on a metric space is tight if, for every , there is a compact set such that every measure in the family assigns at least probability to .
For a complete separable metric space, Prokhorov’s theorem identifies tightness with relative compactness under weak convergence: every sequence from a tight family has a weakly convergent subsequence. This gives a standard proof strategy—establish tightness, then show that every possible subsequential limit has the desired law.
For stochastic processes, convergence in distribution can be studied on a space of entire sample paths. The topology on that space is part of the convergence statement. Convergence of all finite-dimensional distributions alone generally does not imply weak convergence of the path-valued processes; suitable tightness conditions are also needed.
Role in statistical inference
Distributional limits describe the large-sample behavior of statistics. A common result for an estimator is
Combined with a consistent estimate of , this can justify approximate confidence intervals and hypothesis tests.
The limit concerns the centered and scaled estimation error. It does not mean that the unscaled estimator has a nondegenerate normal limit. Indeed, the displayed convergence implies that converges in probability to . The usefulness of any finite-sample approximation remains dependent on how closely the statistic’s actual distribution resembles its limiting law.