The law of total probability is a theorem in probability theory that expresses the probability of an event as a weighted sum of its probabilities under different conditions. When those conditions form mutually exclusive and exhaustive cases, each conditional probability is weighted by the probability of its case. The law provides a basic method for calculating probabilities indirectly, underlies the denominator in Bayes’ theorem, and extends to integration over continuously varying conditions.
Statement
Let be a probability space, and let be an event. Suppose that are measurable events forming a partition of the sample space:
If for every , then
Here denotes the conditional probability of given .
The same formula holds for a countably infinite partition:
The terms are nonnegative, and the series converges to a value in . The cases may also cover the sample space only up to an event of probability zero; the result is unchanged.
An equivalent statement, which does not require positive probabilities for the partition events, is
If some has probability zero, its intersection with also has probability zero. Such terms can be omitted from the conditional-probability formula, rather than attempting to define the ratio .
Two-case form
A common special case uses an event and its complement . When ,
This separates the calculation into two possibilities: occurs, or it does not.
Proof and interpretation
Because the partition events are mutually exclusive and exhaustive,
and the events are mutually exclusive. Countable additivity therefore gives
By the definition of conditional probability,
whenever . Substitution proves the formula.
The law is thus a consequence of the probability axioms and the definition of conditioning, rather than an additional assumption about a model.
For a finite partition, the formula expresses as a convex combination of the conditional probabilities: the weights are nonnegative and sum to one. Consequently,
The weights matter. An ordinary arithmetic average of the conditional probabilities is generally incorrect unless the cases are equally likely.
No assumption of statistical independence is required. If is independent of every partition event, then for each positive-probability case, and the formula reduces to
Example: production from several factories
Suppose a distributor receives products from three factories:
| Factory | Share of products | Probability of a defect |
|---|---|---|
| 1 | ||
| 2 | ||
| 3 |
Let be the event that a randomly selected product is defective, and let identify its factory. Then
The overall defect probability is therefore . The unweighted average of the three factory defect rates is approximately , but that calculation ignores their different contributions to the distributor’s supply.
The relevant weights depend on the sampling procedure. Choosing a factory uniformly and then choosing one of its products would describe a different experiment from choosing a product uniformly from the distributor’s inventory.
Conditional form
The law can itself be applied within a conditioning event. If , then
where terms with are omitted.
This is the ordinary law applied to the probability measure . Both the case probabilities and the within-case probabilities must be evaluated under the additional information . In general, the unconditional weights cannot be substituted for .
Random-variable and continuous forms
Conditioning on a discrete random variable
If a random variable takes finitely or countably many values, the events form a partition. Hence
This formulation is useful when the cases are values of a count, category, state, or unobserved variable.
Conditioning on a continuous random variable
If has a probability density function , the corresponding expression is
with interpreted through an appropriate version of conditional probability.
For a continuously distributed , each individual event has probability zero. The conditional probability in this integral therefore cannot generally be defined by the elementary ratio
Instead, it is defined using conditional distributions or conditional expectation. Under standard regularity conditions, including the usual real-valued settings, suitable conditional distributions exist. Their values need only be specified for -almost every , where is the distribution of .
More generally, the integral form is
This formulation also covers discrete and mixed distributions.
Marginal distributions
Applied to events involving another random variable , total probability yields marginalization formulas. If and have a joint density and is a compatible conditional density, then
for almost every .
Thus, eliminating an auxiliary variable by summing or integrating over its possible values is a distributional form of the law.
Connection with conditional expectation
The most general formulation uses conditional expectation. For a sub-σ-algebra , define
where is the indicator of . Then
This is the law of iterated expectations applied to an indicator. More generally, for any integrable random variable ,
When is generated by a finite or countable partition, conditional probability is constant on each positive-probability partition event, and the expectation becomes the familiar weighted sum. This connects total probability with the broader theory of expectation without requiring conditioning events to have positive probability.
Applications
Bayesian inference
For hypotheses forming a partition, Bayes’ theorem becomes
Total probability supplies the denominator, which normalizes the updated probabilities.
In Bayesian inference, the corresponding continuous operation integrates a likelihood against a prior distribution. The resulting marginal likelihood is
where is the prior probability measure.
Mixture distributions
In a model with a latent category , the distribution of an observation is obtained by averaging its category-specific distributions:
For a Gaussian mixture model, this produces a weighted sum of normal densities. The component label supplies the partition, and the mixture weights supply its probabilities.
Markov chains
For a discrete-state Markov chain, conditioning on the state at an intermediate time gives
for relevant positive-probability conditioning events. The simplification of the first conditional probability uses the Markov property; the sum over intermediate states uses total probability. For time-homogeneous chains, this leads to the Chapman–Kolmogorov equations.
Conditions and common errors
- Overlapping cases cause double counting. The simple sum requires mutually exclusive cases. Overlapping events must first be replaced by a suitable partition or handled with another identity.
- Incomplete cases omit probability. If the selected cases do not cover the sample space up to probability zero, a residual case is needed.
- Conditional probabilities require weights. Adding alone does not generally produce .
- Zero-probability conditioning needs care. Elementary conditional probabilities are undefined when the conditioning event has probability zero; continuous conditioning requires a more general construction.
- Uncountable partitions require integration. Countable additivity does not justify an uncountable sum over individual outcomes. In particular, summing zero-probability singleton events is not a valid way to calculate a continuous probability.
- Conditioning is not intervention. The formula concerns observational conditional probabilities. Its terms do not, by themselves, describe the effects of changing or intervening on the conditioning variable; that distinction belongs to causal inference.