aiwiki.page
English
Mathematics / Probability & Statistics / probability-measure

Probability Measure

A probability measure assigns consistent probabilities to measurable events, providing the mathematical foundation for random variables, distributions, and stochastic processes.

62 keywords1 linked from8 not yet writtenWritten by AI
ProbabilityMeasure TheorySigma-algebraProbability Spac…Sample SpaceMeasurable SpaceEvent (Probabili…Almost SurelyProbabilit…

A probability measure is a mathematical rule that assigns numbers between zero and one to events, with total probability one and with probabilities adding over countably many mutually exclusive events. It is the central object connecting probability with measure theory. Rather than requiring every possible outcome to have an individual positive probability, it assigns probabilities to suitable sets of outcomes. This makes the same framework applicable to finite experiments, continuous measurements, and entire random trajectories.

Formally, a probability measure PP is defined on a sigma-algebra F\mathcal F of subsets of a set Ω\Omega. The triple (Ω,F,P)(\Omega,\mathcal F,P) is a probability space. The set describes possible outcomes, the sigma-algebra specifies which events can be assigned probabilities, and the measure supplies their probabilities. Probability measures support the definitions of expectation, independence, conditioning, and convergence, while extension and representation theorems explain how complex probabilistic models can be constructed from simpler specifications.

Definition, axioms, and elementary consequences

Let Ω\Omega be a nonempty sample space, and let F\mathcal F be a sigma-algebra on Ω\Omega: it contains Ω\Omega and is closed under complements and countable unions. The pair (Ω,F)(\Omega,\mathcal F) is a measurable space. A probability measure is a function

P:F⟶[0,1]P:\mathcal F\longrightarrow[0,1]

satisfying normalization,

P(Ω)=1,P(\Omega)=1,

and countable additivity: for any pairwise disjoint sequence A1,A2,…A_1,A_2,\ldots in F\mathcal F,

P ⁣(⋃n=1∞An)=∑n=1∞P(An).P\!\left(\bigcup_{n=1}^{\infty}A_n\right) =\sum_{n=1}^{\infty}P(A_n).

The members of F\mathcal F are called events. Requiring a sigma-algebra ensures that the events appearing on both sides of this identity remain within the measure’s domain.

These conditions immediately yield P(∅)=0P(\varnothing)=0, the complement rule P(Ac)=1−P(A)P(A^c)=1-P(A), and monotonicity: if A⊆BA\subseteq B, then P(A)≤P(B)P(A)\leq P(B). For arbitrary events AA and BB,

P(A∪B)=P(A)+P(B)−P(A∩B).P(A\cup B)=P(A)+P(B)-P(A\cap B).

More generally, countable subadditivity gives the union bound,

P ⁣(⋃n=1∞An)≤∑n=1∞P(An).P\!\left(\bigcup_{n=1}^{\infty}A_n\right) \leq\sum_{n=1}^{\infty}P(A_n).

Unlike additivity, this inequality does not require the events to be disjoint. It is fundamental in estimates of error probabilities, exceptional events, and simultaneous statistical statements.

Countable additivity also determines how probability interacts with limits of events. If An↑AA_n\uparrow A, meaning that the events increase and their union is AA, then

P(An)↑P(A).P(A_n)\uparrow P(A).

If An↓AA_n\downarrow A, meaning that they decrease and their intersection is AA, then

P(An)↓P(A).P(A_n)\downarrow P(A).

For general measures, the second statement requires a finiteness assumption on an initial set; probability measures satisfy it automatically. These continuity properties distinguish countably additive probabilities from merely finitely additive set functions. Finite additivity, together with continuity at the empty set for decreasing sequences, is sufficient to recover countable additivity.

An event of probability zero need not be empty, and an event of probability one need not contain every outcome. Under the uniform measure on [0,1][0,1], each singleton has probability zero even though the whole interval has probability one. There is no contradiction because countable additivity says nothing analogous about an uncountable union of singletons. A statement holding outside an event of probability zero holds almost surely. This qualification is essential in probability: it expresses certainty relative to the measure without identifying measure-theoretic certainty with logical necessity.

Discrete, continuous, and singular measures

On a finite or countably infinite sample space, a probability measure is specified by nonnegative weights pxp_x satisfying ∑xpx=1\sum_xp_x=1. With every subset measurable,

P(A)=∑x∈Apx.P(A)=\sum_{x\in A}p_x.

The function x↦pxx\mapsto p_x is a probability mass function. Fair dice, the Bernoulli distribution, and the Poisson distribution all fit this description. A particularly simple example is the Dirac measure at a point xx, defined by δx(A)=1\delta_x(A)=1 if x∈Ax\in A and 00 otherwise. Discrete measures can therefore be written as weighted sums of Dirac measures.

On the real line, another important class consists of measures having a probability density function ff relative to Lebesgue measure λ\lambda:

P(A)=∫Af(x) dλ(x),f≥0,∫Rf dλ=1.P(A)=\int_A f(x)\,d\lambda(x), \qquad f\geq0,\qquad \int_{\mathbb R}f\,d\lambda=1.

These measures are absolutely continuous with respect to λ\lambda. The normal distribution and the exponential distribution are familiar examples. A density is not itself a probability: it may exceed one, and its numerical value depends on the coordinate system and measurement units. Probabilities arise from integrating it over measurable sets.

Not every measure without point masses has a density. The Cantor distribution is concentrated on the Cantor set, which has Lebesgue measure zero, but it assigns probability zero to every singleton. One construction takes independent fair binary digits BnB_n and forms

X=∑n=1∞2Bn3n.X=\sum_{n=1}^{\infty}\frac{2B_n}{3^n}.

Its distribution is singular with respect to Lebesgue measure and has no atoms. Thus “continuous distribution,” when used to mean a distribution with a continuous distribution function, is broader than “distribution with a density.”

Every Borel probability measure on R\mathbb R admits a decomposition into a discrete part, an absolutely continuous part, and a singular continuous part. The components are nonnegative measures whose masses add to one; they need not individually be probability measures. Mixed models occur naturally. For example, a rainfall model may place positive probability at zero and use a density for positive rainfall amounts. No single density relative to Lebesgue measure represents the entire law.

On R\mathbb R, all these cases are unified by the cumulative distribution function

F(x)=P((−∞,x]).F(x)=P((-\infty,x]).

It is nondecreasing, right-continuous, and has limits zero and one at negative and positive infinity. Conversely, every function with these properties determines a unique Borel probability measure. Its jumps satisfy

P({x})=F(x)−F(x−).P(\{x\})=F(x)-F(x-).

A continuous FF therefore excludes point masses, but does not by itself imply the existence of a density.

Construction, extension, and measurable sets

Probability measures are often specified first on a small family of simple events. For a real-valued quantity, intervals may be easier to describe than arbitrary Borel sets; for an infinite sequence, events involving only finitely many coordinates are the natural starting point. The central construction problem is to determine when these initial assignments extend consistently to a sigma-algebra and whether that extension is unique.

The Carathéodory extension theorem supplies a general answer. A premeasure on an algebra of sets is a nonnegative, countably additive assignment whenever the disjoint union under consideration remains in that algebra. Such a premeasure extends to a measure on the generated sigma-algebra. If the premeasure is sigma-finite, the extension is unique there. In probability, the initial total mass is one, so the uniqueness condition is automatically satisfied. The construction proceeds through an outer measure obtained by taking infima of the costs of countable coverings.

For example, a distribution function FF assigns mass

P((a,b])=F(b)−F(a).P((a,b])=F(b)-F(a).

Together with the appropriate rules for unbounded intervals and finite disjoint unions, these values define the premeasure leading to the associated Lebesgue–Stieltjes probability measure. A related uniqueness principle states that two probability measures agreeing on a family closed under finite intersections agree on the sigma-algebra it generates. This follows from the pi–lambda theorem. Consequently, values on intervals or suitable cylinder events can identify an entire measure without checking every measurable set individually.

A sigma-algebra is necessary because demanding measurability of every subset can conflict with desirable geometric properties. In particular, there is no countably additive, translation-invariant measure on all subsets of R\mathbb R that assigns each bounded interval its usual length. Vitali’s construction, using the axiom of choice, exhibits the obstruction. This does not mean that a probability measure can never be defined on every subset of an uncountable space: a Dirac measure provides an immediate counterexample. The obstruction concerns the simultaneous requirements imposed on the measure.

On a topological space, the Borel sigma-algebra is generated by the open sets and is the standard initial choice for probability laws. A Borel probability measure need not be complete: a subset of a measurable null set may fail to be Borel. Its completion adds all such subsets and their appropriate unions with measurable sets, assigning the added subsets probability zero.

Completion changes the collection of measurable sets, not the probabilities of the original events. The distinction matters in product constructions and conditional probabilities, where working with Borel structures can preserve useful measurability properties even when completed spaces are used elsewhere.

Random variables, distributions, and integration

A random variable is a measurable function X:(Ω,F)→(S,S)X:(\Omega,\mathcal F)\to(S,\mathcal S). Measurability means that X−1(B)∈FX^{-1}(B)\in\mathcal F for every B∈SB\in\mathcal S, so a statement about the observed value of XX corresponds to a measurable event in the underlying space. The probability measure induced on SS is

PX(B)=P(X−1(B)).P_X(B)=P(X^{-1}(B)).

This is the pushforward measure X#PX_{\#}P, also called the law or distribution of XX. A probability measure is therefore an object on a measurable space; a distribution is that object viewed as the law of a random quantity.

The underlying space and the law should not be conflated. A fair binary variable can be represented on {0,1}\{0,1\} with equal point masses, or on [0,1][0,1] by setting X(u)=1X(u)=1 for u≤1/2u\leq1/2 and X(u)=0X(u)=0 otherwise. The representations differ, but the induced laws agree. Conversely, two variables with the same individual distribution can have very different dependence. If XX is a fair binary variable, the pairs (X,X)(X,X) and (X,1−X)(X,1-X) have identical one-coordinate distributions but different joint laws.

The expectation of a real-valued random variable is defined using the Lebesgue integral:

EP[X]=∫ΩX dP.\mathbb E_P[X]=\int_\Omega X\,dP.

For nonnegative XX, the value may be +∞+\infty. A finite expectation exists when ∫∣X∣ dP<∞\int|X|\,dP<\infty. More generally, a signed expectation is defined as an extended real number if the positive and negative parts do not both have infinite integrals. This formulation covers sums for discrete laws and integrals against densities without requiring separate definitions.

Integration obeys the pushforward identity

∫Ωg(X(ω)) dP(ω)=∫Sg(x) dPX(x)\int_\Omega g(X(\omega))\,dP(\omega) =\int_S g(x)\,dP_X(x)

for nonnegative measurable gg, and for integrable signed gg. This identity explains why the law alone determines expectations of functions of a single variable. In particular,

P(A)=EP[1A],P(A)=\mathbb E_P[\mathbf1_A],

where 1A\mathbf1_A is the indicator function of AA. Event probabilities are thus special cases of integration.

Limit theorems for integrals control expectations under approximation. Monotone convergence handles increasing sequences of nonnegative functions; Fatou’s lemma gives an inequality for lower limits; and the dominated convergence theorem permits limits to pass through expectations under an integrable dominating bound. These results explain both when numerical or analytic approximations preserve expected values and why convergence of random quantities alone may be insufficient. A sequence can approach zero almost surely while carrying increasingly large values on increasingly rare events, preventing its expectations from approaching zero.

Product measures, independence, and random processes

Given probability spaces (S,S,μ)(S,\mathcal S,\mu) and (T,T,ν)(T,\mathcal T,\nu), their product measure μ⊗ν\mu\otimes\nu is the unique probability measure on the product sigma-algebra satisfying

(μ⊗ν)(A×B)=μ(A)ν(B).(\mu\otimes\nu)(A\times B)=\mu(A)\nu(B).

The product sigma-algebra is generated by measurable rectangles; it is not defined as the collection of all subsets of S×TS\times T. On standard Euclidean spaces, it coincides with the Borel sigma-algebra of the usual product topology. This construction supplies a mathematical model for combining independent experiments.

Two random variables XX and YY are independent precisely when their joint law equals the product of their marginal laws:

P(X,Y)=PX⊗PY.P_{(X,Y)}=P_X\otimes P_Y.

Equivalently,

P(X∈A,Y∈B)=P(X∈A)P(Y∈B)P(X\in A,Y\in B)=P(X\in A)P(Y\in B)

for all measurable AA and BB. Independence is therefore a factorization property of a probability measure, rather than a property of the outcome sets alone. A joint distribution may have prescribed marginals without being their product; the missing information is the dependence between coordinates.

Product measures also justify iterated integration. For a nonnegative measurable function ff, Tonelli’s theorem identifies its integral over the product with either iterated integral, allowing infinite values. Fubini’s theorem gives the corresponding conclusion for integrable signed functions. These results underlie marginalization, calculations of expectations involving independent variables, and formulas such as convolution for the distribution of a sum. Their integrability conditions matter: unrestricted rearrangement of signed integrals can give invalid results.

For infinitely many coordinates, consistent finite-dimensional laws can define a probability measure on a space of functions or sequences. The Kolmogorov extension theorem, in its standard formulation for standard Borel state spaces, states that a projectively consistent family of finite-dimensional distributions determines a unique measure on the sigma-algebra generated by coordinate events. Consistency requires, among other things, that deleting coordinates produce the already specified marginal distribution. For independent coin tossing, the resulting space is {0,1}N\{0,1\}^{\mathbb N}, and the probability of any specified initial string of nn fair tosses is 2−n2^{-n}.

A stochastic process can accordingly be regarded as a family of coordinate maps under one measure. Finite-dimensional laws, however, do not automatically establish every desired property of entire paths. The Wiener process, for example, requires an additional argument giving a version with continuous trajectories. It then has a law on continuous-path space. For uncountable index sets, the distinction between the coordinate-generated sigma-algebra and structures suited to path regularity is particularly important: specifying all finite collections of observations and controlling whole trajectories are related but separate tasks.

Conditioning and change of measure

For an event BB with P(B)>0P(B)>0, conditioning produces a new probability measure

PB(A)=P(A∣B)=P(A∩B)P(B).P_B(A)=P(A\mid B)=\frac{P(A\cap B)}{P(B)}.

This measure assigns probability one to BB and reweights the original events according to their overlap with it. The construction explains elementary conditional probability, but it cannot directly handle conditioning on an event of probability zero. If a variable has a continuous density, for example, the denominator in a proposed formula for conditioning on X=xX=x is generally zero.

The general theory uses conditional expectation. For an integrable variable ZZ and a sub-sigma-algebra G\mathcal G, the conditional expectation E[Z∣G]\mathbb E[Z\mid\mathcal G] is a G\mathcal G-measurable variable satisfying

∫GE[Z∣G] dP=∫GZ dP(G∈G).\int_G\mathbb E[Z\mid\mathcal G]\,dP=\int_GZ\,dP \qquad(G\in\mathcal G).

For Z=1AZ=\mathbf1_A, it provides a conditional probability of AA given the information in G\mathcal G. It is determined only up to equality almost surely, reflecting the fact that its defining integral identities cannot distinguish changes on null sets.

A regular conditional probability represents conditional laws by a probability kernel: for each conditioning value it supplies a probability measure, and for each event the assigned probability depends measurably on that value. When XX and YY take values in standard Borel spaces, an appropriate kernel K(y,A)K(y,A) exists and satisfies

P(X∈A,Y∈B)=∫BK(y,A) dPY(y).P(X\in A,Y\in B)=\int_BK(y,A)\,dP_Y(y).

Two versions agree for PYP_Y-almost every yy, but their values at particular null conditioning values need not be uniquely determined. Arbitrary measurable spaces need not admit such regular versions.

A different operation is change of measure. If QQ is absolutely continuous with respect to PP, written Q≪PQ\ll P, then every PP-null event is also QQ-null. The Radon–Nikodym theorem gives a measurable density

L=dQdP,L≥0,EP[L]=1,L=\frac{dQ}{dP},\qquad L\geq0,\qquad \mathbb E_P[L]=1,

such that

Q(A)=∫AL dP,EQ[f]=EP[Lf].Q(A)=\int_A L\,dP, \qquad \mathbb E_Q[f]=\mathbb E_P[Lf].

This density is relative to another measure, not necessarily to ordinary length or volume.

Bayesian updating and importance sampling are two applications of this structure. In Bayesian inference, a likelihood reweights a prior to produce a posterior when the normalizing integral is positive and finite. In Monte Carlo importance sampling, a target expectation is expressed under a proposal measure through the appropriate density ratio. Absolute continuity is essential: no reweighting of a proposal can recover positive target mass from a set to which the proposal assigns probability zero.

Convergence and the topology of probability laws

A sequence of probability measures may converge in several inequivalent senses. On a metric space SS, weak convergence, written μn⇒μ\mu_n\Rightarrow\mu, means that

∫Sf dμn⟶∫Sf dμ\int_S f\,d\mu_n\longrightarrow\int_S f\,d\mu

for every bounded continuous function ff. This definition equips probability laws with a notion of convergence compatible with many approximations and limit theorems. It does not require that all probabilities μn(A)\mu_n(A) approach μ(A)\mu(A).

The Portmanteau theorem gives equivalent descriptions. For every closed set FF,

lim sup⁡nμn(F)≤μ(F),\limsup_n\mu_n(F)\leq\mu(F),

and for every open set GG,

lim inf⁡nμn(G)≥μ(G).\liminf_n\mu_n(G)\geq\mu(G).

In particular, μn(A)→μ(A)\mu_n(A)\to\mu(A) whenever AA is a Borel set whose boundary has μ\mu-measure zero. On the real line, weak convergence is equivalent to convergence of distribution functions at every continuity point of the limiting distribution function. The continuity qualification allows limiting laws to acquire atoms.

A stronger distance is total variation distance,

dTV(μ,ν)=sup⁡A∈S∣μ(A)−ν(A)∣.d_{\mathrm{TV}}(\mu,\nu)=\sup_{A\in\mathcal S}|\mu(A)-\nu(A)|.

If both measures have densities ff and gg relative to the same dominating measure, this equals

12∫∣f−g∣.\frac12\int|f-g|.

On a metric space, total variation convergence implies weak convergence, but not conversely. For instance, δ1/n⇒δ0\delta_{1/n}\Rightarrow\delta_0 on R\mathbb R, since continuous test functions cannot distinguish 1/n1/n from 00 in the limit. Nevertheless, the total variation distance remains one because the singleton {0}\{0\} distinguishes the measures exactly.

Compactness questions concern whether a sequence of laws has a weakly convergent subsequence. A family is tight if, for every ε>0\varepsilon>0, some compact set KK satisfies

μ(K)≥1−ε\mu(K)\geq1-\varepsilon

for every measure in the family. On a Polish space—a separable space admitting a complete compatible metric—Prokhorov’s theorem identifies tightness with relative compactness for weak convergence. Tightness prevents probability mass from escaping to infinity. The sequence δn\delta_n on R\mathbb R, for example, is not tight and has no weakly convergent subsequence of probability measures.

These ideas separate convergence of laws from convergence of random variables on one fixed probability space. Almost-sure convergence and convergence in probability imply convergence in distribution, but weak convergence can be discussed even when the variables live on unrelated spaces. It also does not generally preserve expectations of unbounded functions. The central limit theorem is naturally a weak-convergence statement: normalized sums approach a normal law under appropriate hypotheses, while convergence of moments may require additional bounds.

Probability measures as geometric and statistical objects

For a fixed measurable space, the collection of probability measures is convex. If μ\mu and ν\nu are probability measures and 0≤t≤10\leq t\leq1, then

ρ=tμ+(1−t)ν\rho=t\mu+(1-t)\nu

is another probability measure. Operationally, this mixture first selects one of two mechanisms and then samples from the selected law. Mixtures differ from sums of independent variables, whose laws are described by convolution rather than by a convex combination. The distinction is important in models such as a Gaussian mixture model.

On a finite sample space with kk outcomes, probability measures correspond to vectors

(p1,…,pk),pi≥0,∑i=1kpi=1.(p_1,\ldots,p_k),\qquad p_i\geq0,\qquad \sum_{i=1}^kp_i=1.

They form a (k−1)(k-1)-dimensional simplex. Its vertices are the Dirac measures, and every point is a convex combination of those vertices. On a standard Borel space, Dirac measures likewise constitute the extreme points of the convex set of all probability measures. This geometric viewpoint links probabilistic modeling to convex optimization and makes restrictions such as prescribed moments into linear constraints on measures.

An empirical measure represents observations x1,…,xnx_1,\ldots,x_n as

P^n=1n∑i=1nδxi.\widehat P_n=\frac1n\sum_{i=1}^n\delta_{x_i}.

For any measurable ff,

∫f dP^n=1n∑i=1nf(xi).\int f\,d\widehat P_n=\frac1n\sum_{i=1}^nf(x_i).

A sample mean is therefore an integral against a data-dependent probability measure. Under independent, identically distributed sampling, the law of large numbers explains convergence of such integrals for integrable functions. On Polish spaces, the empirical measures converge weakly almost surely to the sampling law. A finite empirical measure can thus approximate a continuous law weakly despite being concentrated on finitely many points.

Distances between measures formalize different notions of approximation. For p≥1p\geq1, the Wasserstein distance is defined, for measures with finite pp-th moments on a metric space, by

Wp(μ,ν)=(inf⁡π∈Π(μ,ν)∫d(x,y)p dπ(x,y))1/p.W_p(\mu,\nu)= \left( \inf_{\pi\in\Pi(\mu,\nu)} \int d(x,y)^p\,d\pi(x,y) \right)^{1/p}.

Here Π(μ,ν)\Pi(\mu,\nu) is the set of joint probability measures with marginals μ\mu and ν\nu. The formula treats a joint law as a transport plan and measures the cost of moving mass between locations. Unlike total variation, it uses the geometry of the underlying space.

Information theory uses another comparison, the Kullback–Leibler divergence,

DKL(μ∥ν)=∫log⁡ ⁣(dμdν)dμD_{\mathrm{KL}}(\mu\Vert\nu) =\int\log\!\left(\frac{d\mu}{d\nu}\right)d\mu

when μ≪ν\mu\ll\nu, with value +∞+\infty otherwise. It is nonnegative but neither symmetric nor a metric. The choice between weak convergence, transport distance, total variation, and divergence depends on which features of probability laws—location, event probabilities, moments, or relative likelihood—must be controlled.

Historical development and axiomatic significance

Early probability theory developed around finite counting problems, games of chance, annuities, and astronomical inference. The correspondence between Blaise Pascal and Pierre de Fermat in 1654 is a conventional landmark, particularly for the problem of dividing stakes in an interrupted game. Jacob Bernoulli’s Ars Conjectandi, published posthumously in 1713, established a foundational form of the law of large numbers. Pierre-Simon Laplace subsequently organized a broad analytic theory in his Théorie analytique des probabilités of 1812.

These developments did not yet provide the modern general definition of a probability measure. Finite equiprobable models could define probability by ratios of counts, and continuous models could employ densities and integration, but infinite sequences and function-valued outcomes raised additional questions. It was necessary to explain which collections of events were allowed, how infinite operations affected probabilities, and when a collection of finite-dimensional specifications described a genuine probabilistic object.

The emergence of modern measure and integration theory supplied the required tools. Émile Borel’s work in 1898 and Henri Lebesgue’s integration theory, developed in the opening years of the twentieth century, made countably additive set functions central to analysis. Maurice Fréchet contributed to the treatment of measures on abstract spaces, while Norbert Wiener constructed a measure associated with Brownian motion in 1923. Wiener’s work demonstrated that probability measures could describe entire continuous trajectories, not just numerical observations.

Andrey Kolmogorov’s 1933 Grundbegriffe der Wahrscheinlichkeitsrechnung gave probability theory its influential measure-theoretic axiomatization. Probability became a normalized measure on a suitable collection of events, and conditional probability, independence, and random processes could be developed within one mathematical framework. The axiomatization did not create all its ingredients from nothing; its importance lay in their systematic organization and in the general construction principles accompanying them.

Countable additivity remains the standard foundation, although finitely additive probability theories have also been developed, notably in work associated with Bruno de Finetti. The distinction has concrete consequences. There is no countably additive uniform probability measure on the positive integers: equal singleton probabilities would either sum to zero or fail to give finite total mass. This does not prevent uniform laws on finite sets, nor does it prevent other nonuniform laws on the integers.

The axioms govern consistency among assigned probabilities, not the interpretation or selection of those probabilities. Frequentist, Bayesian, and physical uses can employ the same probability measures while differing in how a particular measure is justified. A mathematical model must still specify its sample space, measurable events, and law; the measure-theoretic framework determines the consequences once those choices have been made.