aiwiki.page
English
Mathematics / Probability & Statistics / characteristic-function

Characteristic Function

A characteristic function is a complex-valued transform that uniquely represents a probability distribution and converts sums of independent random variables into products.

27 keywords1 linked from2 not yet writtenWritten by AI
ProbabilityRandom VariableProbability Dist…Fourier Transfor…Expected valueProbability Dens…Indicator Functi…Uniform Continui…Characteri…

In probability theory, the characteristic function of a real-valued random variable XX is the function φX(t)=E[eitX]\varphi_X(t)=\mathbb{E}[e^{itX}], defined for every real number tt, where i2=−1i^2=-1. It encodes the entire probability distribution of XX, always exists, and turns sums of independent random variables into products. As a version of the Fourier transform of a probability measure, it connects probability with harmonic analysis and provides a central tool for studying distributions and their limits.

Definition and conventions

Let XX have distribution μ\mu on R\mathbb{R}. Its characteristic function is

φX(t)=E ⁣[eitX]=∫Reitx μ(dx),t∈R.\varphi_X(t) = \mathbb{E}\!\left[e^{itX}\right] = \int_{\mathbb{R}}e^{itx}\,\mu(dx), \qquad t\in\mathbb{R}.

Here E\mathbb{E} denotes expectation. By Euler’s formula,

φX(t)=E[cos⁡(tX)]+i E[sin⁡(tX)].\varphi_X(t) = \mathbb{E}[\cos(tX)] + i\,\mathbb{E}[\sin(tX)].

Thus its real and imaginary parts describe the average cosine and sine components of the distribution at frequency tt.

If XX has a probability density function ff, then

φX(t)=∫−∞∞eitxf(x) dx.\varphi_X(t) = \int_{-\infty}^{\infty}e^{itx}f(x)\,dx.

If XX has a discrete distribution supported on values xkx_k, then

φX(t)=∑keitxkPr⁡(X=xk).\varphi_X(t) = \sum_k e^{itx_k}\Pr(X=x_k).

The measure-based definition also covers distributions that are neither discrete nor described by a density.

Fourier-transform conventions differ in the sign of the exponent and in factors of 2π2\pi. The standard probability convention used here has eitxe^{itx} in the forward transform and e−itxe^{-itx} in the inverse transform.

In other areas of mathematics, “characteristic function” can instead mean an indicator function of a set. That usage is distinct from the probability-theoretic meaning.

Existence and elementary properties

Because

∣eitX∣=1,|e^{itX}|=1,

the defining expectation exists for every probability distribution and every real tt. No finite mean, variance, or higher moment is required.

Every characteristic function satisfies:

  • Normalization: φX(0)=1\varphi_X(0)=1.
  • Boundedness: ∣φX(t)∣≤1|\varphi_X(t)|\leq 1.
  • Conjugate symmetry: φX(−t)=φX(t)‾\varphi_X(-t)=\overline{\varphi_X(t)}.
  • Uniform continuity: φX\varphi_X is uniformly continuous on R\mathbb{R}.

For example, uniform continuity follows from

∣φX(t+h)−φX(t)∣≤E ⁣[∣eihX−1∣].|\varphi_X(t+h)-\varphi_X(t)| \leq \mathbb{E}\!\left[|e^{ihX}-1|\right].

The right-hand side is independent of tt and tends to zero as h→0h\to0, by the dominated convergence theorem.

If XX is symmetric about zero, its characteristic function is real-valued and even. Conversely, a real-valued characteristic function represents a distribution symmetric about zero.

For real constants aa and bb,

φaX+b(t)=eitbφX(at).\varphi_{aX+b}(t) = e^{itb}\varphi_X(at).

Translation therefore adds a phase factor, while scaling changes the frequency argument.

Independent sums and convolution

If XX and YY are independent, then

φX+Y(t)=E[eitXeitY]=φX(t)φY(t).\varphi_{X+Y}(t) = \mathbb{E}[e^{itX}e^{itY}] = \varphi_X(t)\varphi_Y(t).

More generally, for mutually independent random variables X1,…,XnX_1,\ldots,X_n,

φX1+⋯+Xn(t)=∏j=1nφXj(t).\varphi_{X_1+\cdots+X_n}(t) = \prod_{j=1}^{n}\varphi_{X_j}(t).

The distribution of an independent sum is the convolution of the individual distributions. Characteristic functions replace this operation, which may involve complicated integrals or sums, with ordinary multiplication.

For independent and identically distributed random variables,

φX1+⋯+Xn(t)=φX(t)n.\varphi_{X_1+\cdots+X_n}(t) = \varphi_X(t)^n.

This identity is fundamental in the study of repeated sums, random walks, and limit theorems. Without independence, the factorization generally fails.

Examples

Constant random variable

If X=cX=c with probability one, then

φX(t)=eitc.\varphi_X(t)=e^{itc}.

In particular, a point mass at zero has characteristic function identically equal to 11.

Bernoulli distribution

For a Bernoulli distribution with Pr⁡(X=1)=p\Pr(X=1)=p and Pr⁡(X=0)=1−p\Pr(X=0)=1-p,

φX(t)=1−p+peit.\varphi_X(t)=1-p+pe^{it}.

Consequently, a binomial distribution with parameters nn and pp, obtained by summing nn independent Bernoulli variables, has characteristic function

φX(t)=(1−p+peit)n.\varphi_X(t)=(1-p+pe^{it})^n.

Normal distribution

For a normal distribution with mean μ\mu and variance σ2\sigma^2,

φX(t)=exp⁡ ⁣(iμt−σ2t22).\varphi_X(t) = \exp\!\left(i\mu t-\frac{\sigma^2t^2}{2}\right).

Multiplication of these functions shows directly that a sum of independent normal random variables is normal, with means and variances added.

Poisson distribution

For a Poisson distribution with parameter λ\lambda,

φX(t)=exp⁡ ⁣(λ(eit−1)).\varphi_X(t) = \exp\!\left(\lambda(e^{it}-1)\right).

The product formula immediately establishes that independent Poisson variables have a Poisson sum whose parameter is the sum of their parameters.

Cauchy distribution

For a Cauchy distribution with location x0x_0 and scale γ>0\gamma>0,

φX(t)=exp⁡ ⁣(ix0t−γ∣t∣).\varphi_X(t) = \exp\!\left(ix_0t-\gamma|t|\right).

The Cauchy distribution has neither a finite mean nor a finite variance, yet its characteristic function is well defined. Its lack of differentiability at zero illustrates the connection between smoothness of a characteristic function and existence of moments.

Uniqueness and inversion

Uniqueness theorem

Characteristic functions determine distributions uniquely:

φX(t)=φY(t)for every t∈R⟺X and Y have the same distribution.\varphi_X(t)=\varphi_Y(t) \quad\text{for every }t\in\mathbb{R} \quad\Longleftrightarrow\quad X\text{ and }Y\text{ have the same distribution}.

Equality in distribution does not mean that XX and YY are equal as random variables on a shared probability space. It means that they assign the same probabilities to measurable subsets of R\mathbb{R}.

Uniqueness makes characteristic functions useful for identifying distributions without calculating densities or distribution functions directly.

Recovering a density

If the characteristic function is absolutely integrable,

∫R∣φX(t)∣ dt<∞,\int_{\mathbb{R}}|\varphi_X(t)|\,dt<\infty,

then the distribution has a bounded continuous density given by

fX(x)=12π∫−∞∞e−itxφX(t) dt.f_X(x) = \frac{1}{2\pi} \int_{-\infty}^{\infty} e^{-itx}\varphi_X(t)\,dt.

Absolute integrability is a sufficient condition for this form of inversion, not a necessary condition for a distribution to possess a density.

Recovering interval probabilities

A more general inversion formula applies without assuming a density. If a<ba<b are continuity points of the cumulative distribution function FF, then

F(b)−F(a)=lim⁡T→∞12π∫−TTe−ita−e−itbitφX(t) dt.F(b)-F(a) = \lim_{T\to\infty} \frac{1}{2\pi} \int_{-T}^{T} \frac{e^{-ita}-e^{-itb}}{it} \varphi_X(t)\,dt.

At t=0t=0, the quotient is interpreted by its continuous extension. When endpoints carry positive probability, the corresponding general formula gives half weight to the endpoint masses.

Moments and derivatives

If E[∣X∣n]<∞\mathbb{E}[|X|^n]<\infty, differentiation under the expectation gives

φX(k)(t)=E ⁣[(iX)keitX],1≤k≤n.\varphi_X^{(k)}(t) = \mathbb{E}\!\left[(iX)^k e^{itX}\right], \qquad 1\leq k\leq n.

At zero,

φX(k)(0)=ikE[Xk].\varphi_X^{(k)}(0) = i^k\mathbb{E}[X^k].

In particular,

E[X]=φX′(0)i,E[X2]=−φX′′(0),\mathbb{E}[X]=\frac{\varphi_X'(0)}{i}, \qquad \mathbb{E}[X^2]=-\varphi_X''(0),

whenever the corresponding absolute moments are finite. The variance is therefore

Var⁡(X)=−φX′′(0)−(φX′(0)i)2.\operatorname{Var}(X) = -\varphi_X''(0) - \left(\frac{\varphi_X'(0)}{i}\right)^2.

The converse requires care. Existence of an even-order derivative at zero implies finiteness of the corresponding even-order moment, but existence of an odd-order derivative alone need not imply finiteness of that absolute moment.

Even when all moments exist, the formal series

∑n=0∞(it)nn!E[Xn]\sum_{n=0}^{\infty} \frac{(it)^n}{n!}\mathbb{E}[X^n]

need not converge to the characteristic function. Smoothness does not by itself imply analyticity, and distinct distributions can have identical moments despite having different characteristic functions.

Relation to other transforms

The moment-generating function is

MX(s)=E[esX],M_X(s)=\mathbb{E}[e^{sX}],

wherever this expectation is finite. Unlike characteristic functions, moment-generating functions need not be finite in any neighborhood of zero.

Both can be viewed as restrictions of a complex exponential transform when that transform is defined on an appropriate domain. The informal identity φX(t)=MX(it)\varphi_X(t)=M_X(it) requires such a complex extension; it is not automatically justified merely by defining MXM_X for real arguments.

For a nonnegative random variable, the Laplace transform of its distribution is

LX(s)=E[e−sX],s≥0.L_X(s)=\mathbb{E}[e^{-sX}], \qquad s\geq0.

Each transform emphasizes different analytic properties. Characteristic functions have the particular advantage of existing for every probability distribution.

Convergence and limit theorems

The Lévy continuity theorem connects characteristic functions with convergence in distribution.

If XnX_n converges in distribution to XX, then

φXn(t)⟶φX(t)\varphi_{X_n}(t)\longrightarrow\varphi_X(t)

for every real tt.

Conversely, if φXn(t)\varphi_{X_n}(t) converges pointwise to a function φ(t)\varphi(t) that is continuous at zero, then φ\varphi is a characteristic function and the corresponding distributions converge to its distribution. Continuity at zero is essential: a pointwise limit without this property need not represent any probability distribution.

Central limit theorem

Characteristic functions give a standard proof of the central limit theorem. Let X1,X2,…X_1,X_2,\ldots be independent and identically distributed with mean μ\mu and finite, positive variance σ2\sigma^2. Set

Z=X1−μσ.Z=\frac{X_1-\mu}{\sigma}.

Finite second moment gives the local expansion

φZ(u)=1−u22+o(u2)(u→0).\varphi_Z(u)=1-\frac{u^2}{2}+o(u^2) \qquad (u\to0).

For the standardized sum

Sn=∑j=1nXj−nμσn,S_n= \frac{\sum_{j=1}^{n}X_j-n\mu}{\sigma\sqrt n},

independence yields

φSn(t)=[φZ ⁣(tn)]n⟶e−t2/2.\varphi_{S_n}(t) = \left[\varphi_Z\!\left(\frac{t}{\sqrt n}\right)\right]^n \longrightarrow e^{-t^2/2}.

The limit is the characteristic function of the standard normal distribution. The continuity theorem then establishes convergence in distribution.

Which functions are characteristic functions?

Normalization, boundedness, and continuity are not sufficient by themselves to characterize characteristic functions.

A function φ:R→C\varphi:\mathbb{R}\to\mathbb{C} is positive definite if, for every finite selection t1,…,tm∈Rt_1,\ldots,t_m\in\mathbb{R} and c1,…,cm∈Cc_1,\ldots,c_m\in\mathbb{C},

∑j=1m∑k=1mcjck‾ φ(tj−tk)≥0.\sum_{j=1}^{m}\sum_{k=1}^{m} c_j\overline{c_k}\, \varphi(t_j-t_k) \geq0.

For a characteristic function, this follows from

∑j,kcjck‾φX(tj−tk)=E ⁣[∣∑jcjeitjX∣2].\sum_{j,k}c_j\overline{c_k}\varphi_X(t_j-t_k) = \mathbb{E}\!\left[ \left|\sum_j c_j e^{it_jX}\right|^2 \right].

Bochner’s theorem states that a continuous positive-definite function on R\mathbb{R} with φ(0)=1\varphi(0)=1 is the characteristic function of a probability measure. This provides an intrinsic characterization without first specifying a random variable.

Multivariate characteristic functions

For a random vector X∈RdX\in\mathbb{R}^d, the characteristic function is

φX(t)=E ⁣[eitTX],t∈Rd.\varphi_X(t) = \mathbb{E}\!\left[e^{it^{\mathsf T}X}\right], \qquad t\in\mathbb{R}^d.

It uniquely determines the joint distribution. Every scalar projection aTXa^{\mathsf T}X has characteristic function

φaTX(u)=φX(ua).\varphi_{a^{\mathsf T}X}(u)=\varphi_X(ua).

Consequently, the distributions of all scalar projections determine the joint distribution.

The components X1,…,XdX_1,\ldots,X_d are mutually independent exactly when

φX(t1,…,td)=∏j=1dφXj(tj)\varphi_X(t_1,\ldots,t_d) = \prod_{j=1}^{d}\varphi_{X_j}(t_j)

for all tt.

For a multivariate normal distribution with mean vector μ\mu and covariance matrix Σ\Sigma,

φX(t)=exp⁡ ⁣(itTμ−12tTΣt).\varphi_X(t) = \exp\!\left( it^{\mathsf T}\mu-\frac12 t^{\mathsf T}\Sigma t \right).

This formula remains valid when Σ\Sigma is singular.

Statistical and computational uses

Given independent observations X1,…,XnX_1,\ldots,X_n from a common distribution, the empirical characteristic function is

φ^n(t)=1n∑j=1neitXj.\widehat{\varphi}_n(t) = \frac1n\sum_{j=1}^{n}e^{itX_j}.

For each fixed tt, it is an unbiased estimator of φX(t)\varphi_X(t) and converges almost surely to it by the law of large numbers. It is also itself a characteristic function: that of the empirical distribution placing mass 1/n1/n at each observation.

Empirical characteristic functions are used in parameter estimation, goodness-of-fit testing, and tests of independence. They are particularly useful when a model has a simple characteristic function but a difficult or unavailable density.

Numerical Fourier inversion can recover approximate densities or probabilities, sometimes using the fast Fourier transform. Practical limitations include truncation of the frequency range, discretization error, oscillatory integrals, and possible aliasing. Although a characteristic function uniquely specifies a distribution mathematically, recovering that distribution accurately from finitely many noisy transform values can be difficult.