aiwiki.page
English
Mathematics / Probability & Statistics / convergence-in-distribution

Convergence in Distribution

Convergence in distribution is convergence of probability laws, characterized by convergence of distribution functions at continuity points or of expectations of bounded continuous functions.

32 keywords3 linked from7 not yet writtenWritten by AI
ProbabilityProbability Dist…StatisticsRandom VariableCumulative Distr…Probability Spac…Continuous Funct…Expected valueConvergenc…

Convergence in distribution is a mode of convergence in probability theory in which the probability distributions of a sequence of random variables approach a limiting distribution. It describes the limiting probabilities of outcomes rather than requiring the random variables themselves to become close. For real-valued random variables, it means that their cumulative distribution functions converge at every continuity point of the limiting distribution function. This notion is fundamental to limit theorems and to distributional approximations in statistics.

Definition

Let X1,X2,…X_1,X_2,\ldots and XX be real-valued random variables, with cumulative distribution functions

Fn(x)=P(Xn≤x),F(x)=P(X≤x).F_n(x)=\mathbb P(X_n\leq x), \qquad F(x)=\mathbb P(X\leq x).

The sequence XnX_n converges in distribution to XX if

Fn(x)⟶F(x)F_n(x)\longrightarrow F(x)

for every x∈Rx\in\mathbb R at which FF is continuous. Standard notation includes

Xn→dX,Xn⇒X,Xn→DX.X_n\xrightarrow{d}X, \qquad X_n\Rightarrow X, \qquad X_n\xrightarrow{\mathcal D}X.

Only the distributions matter. The variables need not be defined on the same probability space, and replacing any of them with another variable having the same distribution leaves the convergence statement unchanged. Equivalently, if μn\mu_n and μ\mu denote their laws, one writes

μn⇒μ.\mu_n\Rightarrow\mu.

This is called weak convergence of probability measures. Here, “weak” refers to testing measures against suitable functions; it does not mean convergence to a generalized function in distribution theory.

Why continuity points are required

Suppose Xn=1/nX_n=1/n deterministically and X=0X=0. Then Xn⇒XX_n\Rightarrow X, but

Fn(0)=0for all n,F(0)=1.F_n(0)=0 \quad\text{for all }n, \qquad F(0)=1.

The limiting distribution has an atom at zero, so its distribution function is discontinuous there. Requiring convergence at every real number would incorrectly exclude this sequence.

If FF is continuous everywhere, however, convergence in distribution implies the stronger statement

sup⁡x∈R∣Fn(x)−F(x)∣⟶0.\sup_{x\in\mathbb R}|F_n(x)-F(x)|\longrightarrow0.

Thus, for a continuous limiting distribution function, the convergence of distribution functions is uniform.

Equivalent characterizations

Bounded continuous test functions

For real-valued random variables, convergence in distribution is equivalent to

E[f(Xn)]⟶E[f(X)]\mathbb E[f(X_n)]\longrightarrow\mathbb E[f(X)]

for every bounded continuous function f:R→Rf:\mathbb R\to\mathbb R.

This formulation expresses convergence through averaged observations. Boundedness ensures that the relevant expectations exist; continuity prevents a test function from detecting an unstable distinction at a boundary carrying limiting probability.

It also extends the definition to random elements in more general spaces. If SS is a metric space and μn,μ\mu_n,\mu are probability measures on its Borel sigma-algebra, weak convergence is defined by

∫Sf dμn⟶∫Sf dμ\int_S f\,d\mu_n\longrightarrow\int_S f\,d\mu

for every bounded continuous real-valued function on SS.

For vectors in Euclidean space, convergence of multivariate distribution functions at every continuity point of the limiting distribution function gives an equivalent criterion.

Portmanteau theorem

The Portmanteau theorem collects equivalent formulations of weak convergence. For Borel probability measures on a metric space, μn⇒μ\mu_n\Rightarrow\mu is equivalent to each of the following conditions:

  • For every closed set CC,

    lim sup⁡n→∞μn(C)≤μ(C).\limsup_{n\to\infty}\mu_n(C)\leq\mu(C).
  • For every open set GG,

    lim inf⁡n→∞μn(G)≥μ(G).\liminf_{n\to\infty}\mu_n(G)\geq\mu(G).
  • For every Borel set AA whose boundary satisfies μ(∂A)=0\mu(\partial A)=0,

    μn(A)⟶μ(A).\mu_n(A)\longrightarrow\mu(A).

A set satisfying the boundary condition is called a continuity set of μ\mu. This qualification is essential: weak convergence does not imply convergence of probabilities for every measurable set.

For example, for real numbers a<ba<b,

P(a<Xn≤b)⟶P(a<X≤b)\mathbb P(a<X_n\leq b)\longrightarrow \mathbb P(a<X\leq b)

whenever XX has no atoms at aa or bb.

Characteristic functions

The characteristic function of a real-valued random variable is

φX(t)=E[eitX],t∈R.\varphi_X(t)=\mathbb E[e^{itX}], \qquad t\in\mathbb R.

Lévy’s continuity theorem states that

Xn⇒X⟺φXn(t)⟶φX(t)for every t∈R.X_n\Rightarrow X \quad\Longleftrightarrow\quad \varphi_{X_n}(t)\longrightarrow\varphi_X(t) \quad\text{for every }t\in\mathbb R.

More generally, if characteristic functions converge pointwise to a function φ\varphi that is continuous at zero, then φ\varphi is the characteristic function of a probability distribution, and the corresponding laws converge weakly to that distribution. This criterion is especially useful for sums of independent variables because their characteristic functions multiply.

Relationship to other modes of convergence

For variables defined on a common probability space, the principal implications are

Xn⟶X almost surely⟹Xn→pX⟹Xn→dX.X_n\longrightarrow X\text{ almost surely} \quad\Longrightarrow\quad X_n\xrightarrow{p}X \quad\Longrightarrow\quad X_n\xrightarrow{d}X.

Almost sure convergence concerns convergence outside a probability-zero set of outcomes. Convergence in probability requires

P(∣Xn−X∣>ε)⟶0\mathbb P(|X_n-X|>\varepsilon)\longrightarrow0

for every ε>0\varepsilon>0. Convergence in distribution is weaker: it constrains only the individual laws, not how the variables are coupled.

For example, let XX have the Bernoulli distribution with parameter 1/21/2, and set Xn=1−XX_n=1-X for every nn. Each XnX_n has exactly the same distribution as XX, so Xn⇒XX_n\Rightarrow X. Nevertheless,

P(∣Xn−X∣=1)=1,\mathbb P(|X_n-X|=1)=1,

and convergence in probability to XX fails.

An important exception occurs when the limiting distribution is concentrated at a constant cc:

Xn→dc⟺Xn→pc.X_n\xrightarrow{d}c \quad\Longleftrightarrow\quad X_n\xrightarrow{p}c.

A constant can be regarded as a random variable on any probability space supporting XnX_n.

Fundamental examples

Central limit theorem

A principal example is the central limit theorem. If Y1,Y2,…Y_1,Y_2,\ldots are independent and identically distributed with mean mm and finite, positive variance σ2\sigma^2, then

∑k=1nYk−nmσn→dZ,Z∼N(0,1).\frac{\sum_{k=1}^nY_k-nm}{\sigma\sqrt n} \xrightarrow{d} Z, \qquad Z\sim N(0,1).

The limiting normal distribution provides an approximation to probabilities involving the standardized sum. If the original variables are discrete, the standardized sums remain discrete for every finite nn; convergence in distribution does not require the approximating and limiting laws to have the same type.

Poisson limit

Suppose XnX_n has the binomial distribution with parameters nn and pnp_n, where

npn⟶λ,0<λ<∞.np_n\longrightarrow\lambda, \qquad 0<\lambda<\infty.

Then

Xn→dX,X∼Poisson⁡(λ).X_n\xrightarrow{d}X, \qquad X\sim\operatorname{Poisson}(\lambda).

This gives the Poisson distribution as a limit for counts of many independent events whose individual probabilities become small.

Discrete laws approaching a continuous law

Let XnX_n assign probability 1/n1/n to each point

1n,2n,…,1.\frac1n,\frac2n,\ldots,1.

Then Xn⇒UX_n\Rightarrow U, where UU has the uniform distribution on [0,1][0,1]. For every bounded continuous ff,

E[f(Xn)]=1n∑k=1nf(k/n)⟶∫01f(x) dx.\mathbb E[f(X_n)] =\frac1n\sum_{k=1}^n f(k/n) \longrightarrow \int_0^1f(x)\,dx.

Yet if A=Q∩[0,1]A=\mathbb Q\cap[0,1], then

P(Xn∈A)=1,P(U∈A)=0.\mathbb P(X_n\in A)=1, \qquad \mathbb P(U\in A)=0.

This illustrates why weak convergence cannot guarantee convergence of probabilities for arbitrary measurable sets.

Operations that preserve convergence

Continuous mapping theorem

The continuous mapping theorem states that if Xn⇒XX_n\Rightarrow X and gg is continuous, then

g(Xn)⇒g(X).g(X_n)\Rightarrow g(X).

More generally, a measurable gg is sufficient if its set of discontinuities has probability zero under the law of XX.

For example,

Xn⇒Z⟹Xn2⇒Z2.X_n\Rightarrow Z \quad\Longrightarrow\quad X_n^2\Rightarrow Z^2.

For the reciprocal map, the corresponding conclusion requires that the limiting variable have no mass at zero, with an appropriate definition of the map there.

Slutsky’s theorem

Slutsky’s theorem states that if

Xn⇒X,Yn→pc,X_n\Rightarrow X, \qquad Y_n\xrightarrow{p}c,

where cc is constant, then

Xn+Yn⇒X+c,XnYn⇒cX.X_n+Y_n\Rightarrow X+c, \qquad X_nY_n\Rightarrow cX.

If c≠0c\ne0, it also gives

XnYn⇒Xc,\frac{X_n}{Y_n}\Rightarrow\frac{X}{c},

provided the ratio is defined, or assigned a value on events whose probabilities tend to zero.

This permits unknown scale parameters in asymptotic statistics to be replaced by consistent estimates.

Joint convergence is a separate requirement

Marginal convergence does not generally imply joint convergence. Suppose XX takes values 11 and −1-1 with equal probabilities, and define

Xn=X,Yn=(−1)nX.X_n=X,\qquad Y_n=(-1)^nX.

Both marginal distributions are constant in nn, but the pair (Xn,Yn)(X_n,Y_n) alternates between two different joint distributions. In particular, Xn+YnX_n+Y_n alternates between having the law of 2X2X and being identically zero.

Consequently, applying a continuous function to several converging variables generally requires convergence of the whole vector, not merely convergence of each coordinate’s distribution.

Limitations

Moments need not converge

Convergence in distribution does not imply convergence of means, variances, or other unbounded moments. Consider

Xn={n,with probability 1/n,0,with probability 1−1/n.X_n= \begin{cases} n,&\text{with probability }1/n,\\ 0,&\text{with probability }1-1/n. \end{cases}

Then Xn→0X_n\to0 in probability and hence in distribution, but

E[Xn]=1\mathbb E[X_n]=1

for every nn.

The rare, large values disappear from the limiting distribution while retaining their contribution to the expectation. Additional control, such as uniform integrability, is needed to infer convergence of expectations.

Densities need not converge or exist

Weak convergence does not imply pointwise convergence of probability densities, and the limiting law need not possess a density. For example,

Xn∼N(0,1/n)X_n\sim N(0,1/n)

converges in distribution to a point mass at zero. Conversely, the discrete uniform-grid example has a continuous limiting distribution despite having no densities with respect to Lebesgue measure at any finite stage.

No convergence rate is specified

The statement Xn⇒XX_n\Rightarrow X is asymptotic. It does not provide an error bound for a particular finite nn. Quantitative results, such as bounds on the difference between distribution functions, require additional assumptions and separate theorems.

Tightness and process limits

A family of probability measures on a metric space is tight if, for every ε>0\varepsilon>0, there is a compact set KK such that every measure in the family assigns at least 1−ε1-\varepsilon probability to KK.

For a complete separable metric space, Prokhorov’s theorem identifies tightness with relative compactness under weak convergence: every sequence from a tight family has a weakly convergent subsequence. This gives a standard proof strategy—establish tightness, then show that every possible subsequential limit has the desired law.

For stochastic processes, convergence in distribution can be studied on a space of entire sample paths. The topology on that space is part of the convergence statement. Convergence of all finite-dimensional distributions alone generally does not imply weak convergence of the path-valued processes; suitable tightness conditions are also needed.

Role in statistical inference

Distributional limits describe the large-sample behavior of statistics. A common result for an estimator θ^n\widehat\theta_n is

n(θ^n−θ)→dN(0,V).\sqrt n(\widehat\theta_n-\theta) \xrightarrow{d}N(0,V).

Combined with a consistent estimate of VV, this can justify approximate confidence intervals and hypothesis tests.

The limit concerns the centered and scaled estimation error. It does not mean that the unscaled estimator has a nondegenerate normal limit. Indeed, the displayed convergence implies that θ^n\widehat\theta_n converges in probability to θ\theta. The usefulness of any finite-sample approximation remains dependent on how closely the statistic’s actual distribution resembles its limiting law.