The law of total expectation is a theorem in probability theory stating that the expected value of a random variable equals the expected value of its conditional expectation. For an integrable random variable and another random variable ,
Informally, an overall mean can be obtained by calculating the mean within each group and then averaging those group means using the groups’ probabilities. The law requires no independence assumption. Its more general form, called the tower property, describes successive conditioning on nested collections of information.
Basic statement and interpretation
Suppose is real-valued and integrable, meaning
The expression is itself a random variable: its value depends on the observed value of . The outer expectation averages this conditional mean over the distribution of .
If denotes a suitable version of , the law becomes
Thus, replacing each outcome of by the average associated with its value of preserves the overall expectation.
The theorem also holds for nonnegative , allowing either side to equal . For a signed random variable that is not integrable, additional care is necessary: an expectation involving the undefined expression is not meaningful.
Discrete and continuous forms
Conditioning on a partition
Let be a finite or countable partition of the sample space, with events of probability zero omitted. Then
Here
where is the indicator function of .
For a discrete conditioning variable , this gives
Only needs to be discrete; may be discrete, continuous, or mixed.
Conditioning on a continuous variable
If has a probability density function , then
If and have a joint density, a version of can be written as
where . Substituting this expression yields
For continuous , the event ordinarily has probability zero. Accordingly, is not defined by dividing by . It is interpreted through a conditional distribution or a measurable representation of , determined up to sets of -probability zero.
More generally, writing for the distribution of ,
This formulation also covers distributions that are neither purely discrete nor purely continuous.
Measure-theoretic formulation
On a probability space , let be a sigma-algebra representing available information. For integrable , the conditional expectation
is an integrable, -measurable random variable satisfying
It is unique almost surely.
Taking immediately gives
The basic law of total expectation is therefore a direct consequence of the defining property of conditional expectation.
Conditioning on means conditioning on , the sigma-algebra generated by . This measure-theoretic formulation does not require a density or a countable set of possible values.
Tower property
If
then
Averaging an estimate based on finer information over coarser information gives the estimate based directly on the coarser information.
To prove this, set
For every , the inclusion implies
Together with -measurability, this identifies as a version of .
The reverse order also satisfies
because a -measurable random variable is already -measurable.
An important random-variable form is
The nesting matters: arbitrary successive conditioning on unrelated information sets does not obey the same simplification.
Examples
Weighted group means
Suppose a production process uses machine A for of items and machine B for . The mean processing times are 4 minutes and 9 minutes, respectively. If is the processing time and identifies the machine, then
The unweighted average, minutes, is not the overall mean because the machines are not used equally often.
A random success probability
Let take values in , and suppose
where is fixed. The conditional mean of this binomial distribution is , so
This remains valid regardless of the particular distribution of .
A random number of summands
Let be a nonnegative integer-valued random variable with finite mean. Let be identically distributed, integrable random variables with mean , and suppose the entire sequence is independent of . Define
with when . Since
the law gives
Independence among the is not needed for this mean calculation. Independence from , however, is a sufficient condition that justifies the stated conditional mean.
Consequences and applications
Total probability
Applying the theorem to the indicator of an event gives
For discrete ,
Thus, the law of total probability is the indicator-variable case of total expectation.
Variance decomposition
If , applying total expectation to squared deviations gives the law of total variance:
This separates variance into average within-group variability and variability of the group means.
Conditional Monte Carlo
In the Monte Carlo method, replacing by a computable conditional expectation preserves its mean. Under square integrability,
A simulation based on conditional means can therefore have lower variance, although the computational cost of obtaining those means also matters.
Bayesian prediction and stochastic processes
In Bayesian inference, predictive expectations average parameter-conditional expectations over the parameter’s posterior distribution. For observed data , parameter , and future observation ,
whenever the required expectations exist.
For a filtration and an integrable random variable , the process
is a martingale. Indeed, for , the tower property gives
Conditions and common pitfalls
Conditional means must receive the correct weights. Equally weighting groups generally gives a different result from averaging over their population probabilities.
Independence is unnecessary. If and are statistically independent, the conditional mean is constant almost surely; the theorem also applies when they are dependent.
Integrability cannot be ignored. For example, a Cauchy-distributed random variable has no ordinary expectation, so the finite-mean identity cannot be invoked for it.
Conditioning must be nested for the tower simplification. Without an inclusion relation between information sets, replacing repeated conditioning by a single conditional expectation is generally invalid.
Expectation does not generally commute with nonlinear functions. The theorem implies
when integrable, but does not generally imply
For convex , the corresponding inequality is governed by Jensen’s inequality.