Almost surely is the expression used in probability theory for a property that holds with probability one. It permits exceptional outcomes, provided their total probability is zero. The term therefore differs from “surely,” understood as holding for every possible outcome. It is the probabilistic form of almost everywhere in measure theory, with the relevant measure being a probability measure. The abbreviation a.s. commonly accompanies equations, inequalities, and convergence statements. (statslab.cam.ac.uk)
Definition and interpretation
Let ((\Omega,\mathcal F,\mathbb P)) be a probability space, where (\Omega) is the sample space, (\mathcal F) is a sigma-algebra of measurable events, and (\mathbb P) is the probability measure. A measurable event (A) occurs almost surely if
[ \mathbb P(A)=1, ]
or, equivalently, (\mathbb P(\Omega\setminus A)=0). Its complement is a null set. For a property expressed in terms of an outcome (\omega), saying that it holds almost surely means that it holds outside a measurable set of probability zero. (statslab.cam.ac.uk)
“Almost” does not mean “very likely”: probability (0.999999) is not probability one. Nor does probability zero necessarily mean that an event is empty. These distinctions arise naturally in continuous probability models. An almost-sure assertion is always relative to the specified probability measure; changing the measure can change which properties hold almost surely. (terrytao.wordpress.com)
Probability one without pointwise certainty
Consider the interval ([0,1]), equipped with its Borel sets and the uniform distribution. The probability of each singleton is zero, although every point belongs to the sample space. If (U(\omega)=\omega), then (U\ne 1/2) almost surely, but the equality (U=1/2) remains a possible sample-space outcome. This illustrates why zero probability and impossibility are distinct mathematical notions. (terrytao.wordpress.com)
A related example uses the rational numbers. Since those in ([0,1]) form a countable set, their union has probability zero. Consequently, (U) is an irrational number almost surely. This conclusion follows from countable additivity, not from removing rational points from the sample space. In contrast, on a finite sample space where every individual outcome has positive probability, a probability-one event must include every outcome. (souravchatterjee.su.domains)
Almost-sure equality
Two random variables (X) and (Y), defined on the same probability space, are equal almost surely when
[ \mathbb P(X=Y)=1. ]
They need not be identical as functions: their values may differ on a null set. Almost-sure equality is an equivalence relation, allowing random variables to be grouped into equivalence classes. Equal-almost-surely variables have the same probability distribution, and the same expected value whenever that expectation exists. The converse is false: equal distributions do not require equality on the underlying sample space. (terrytao.wordpress.com)
This distinction is important in Lp spaces, whose elements identify functions that agree almost everywhere. It also explains the usual uniqueness statement for conditional expectation. For an integrable variable and a specified conditioning sigma-algebra, any two versions of its conditional expectation agree almost surely, rather than necessarily at every outcome. (tropp.caltech.edu)
Almost-sure convergence
A sequence (X_1,X_2,\ldots) converges almost surely to (X) if
[ \mathbb P!\left( \left{\omega: \lim_{n\to\infty}X_n(\omega)=X(\omega) \right}\right)=1. ]
The notation is (X_n\xrightarrow{\mathrm{a.s.}}X). Thus, outside one exceptional null set, the numerical sequence associated with each fixed outcome converges. This is pointwise convergence on a probability-one set, not necessarily uniform convergence over that set. The index beyond which an error becomes small can depend on the outcome. (ocw.mit.edu)
Almost-sure convergence implies convergence in probability, meaning that, for every (\varepsilon>0),
[ \mathbb P(|X_n-X|>\varepsilon)\longrightarrow0. ]
The reverse implication generally fails. Nevertheless, every sequence converging in probability has a subsequence converging almost surely to the same limit. Almost-sure convergence also does not, by itself, justify convergence of expectations; additional conditions are needed, such as those in the dominated convergence theorem. (ocw.mit.edu)
Countability and limiting theorems
If each event (A_n) has probability one, then their countable intersection also has probability one. Equivalently, a countable union of null events is null. The restriction to countable families matters: for a uniform variable (U), each assertion (U\ne t), for fixed (t\in[0,1]), holds almost surely, but they cannot all hold simultaneously over every (t), because (U) takes some value in that interval. (souravchatterjee.su.domains)
The Borel–Cantelli lemmas provide useful criteria for almost-sure assertions. If (\sum_n\mathbb P(A_n)<\infty), then almost surely only finitely many (A_n) occur. If the events are mutually independent and the sum diverges, infinitely many occur almost surely. (ocw.mit.edu)
A central application is the strong law of large numbers. For independent, identically distributed real random variables with (\mathbb E|X_1|<\infty), the sample mean satisfies
[ \frac1n\sum_{k=1}^{n}X_k \xrightarrow{\mathrm{a.s.}} \mathbb E[X_1]. ]
The theorem concerns the limiting behavior of entire infinite sequences of observations. It does not assert that every conceivable sequence has this limit, nor that any finite sample mean equals the expectation exactly. (ocw.mit.edu)