A marginal distribution is the probability distribution of one variable, or a selected group of variables, considered within a larger collection of random variables. It is obtained from their joint distribution by accounting for all possible values of the variables not retained. This operation, called marginalization, uses summation for discrete variables and integration for variables with a joint density. Unlike conditioning, marginalization does not fix the omitted variables at particular values.
Definition
For random variables and , their joint distribution specifies probabilities involving both variables. The marginal distribution of specifies probabilities involving alone:
where is a measurable set of possible values of , and is the entire state space of .
The term marginal reflects the layout of a two-way probability table: row and column totals can be written in its margins. These totals give the separate distributions of the row and column variables.
A marginal need not describe just one variable. For a random vector , the joint distribution of any selected subvector is a marginal of the full distribution. For example, the distribution of is a two-dimensional marginal of the distribution of .
Discrete variables
If and are discrete, their joint probability mass function is
The marginal mass functions are
Each sum runs over all possible values of the variable being eliminated. No assumption of statistical independence is required.
For example, consider the joint distribution:
| Marginal of | |||
|---|---|---|---|
| Marginal of |
Thus,
and
The marginal distributions retain the probabilities for each variable separately but not the full pattern of association between them. In this example, the variables are not independent, since
Continuous variables
If has a joint probability density function , its marginal densities are
These equations determine densities up to changes on sets of Lebesgue measure zero. The integration limits may be restricted to the region where the joint density is nonzero.
For example, suppose
The marginal density of is
and the marginal density of is
Both densities are zero outside these intervals. Each integrates to one, although neither is constant on , despite the joint density being constant over its triangular support.
Marginalizing a density is not the same as evaluating it at a fixed value of another variable. The function is generally neither a normalized marginal density nor a normalized conditional density.
Distribution functions and general formulation
Marginal distributions exist even when no joint density exists. For real-valued variables, let the joint cumulative distribution function be
Then
This formulation applies to discrete, continuous, mixed, and singular distributions.
In measure-theoretic terms, suppose the joint distribution is a probability measure on a product of measurable spaces . The marginal on is
Equivalently, it is the measure induced by the coordinate projection
This definition makes marginalization a special case of obtaining the distribution of a measurable function of a random element.
For higher-dimensional distributions, marginalizing in stages gives the same result as removing all unwanted coordinates at once. For example, eliminating and then from gives the same marginal of as eliminating together. For nonnegative mass functions and densities, the relevant sums and integrals can be reordered.
Marginal versus conditional distributions
A marginal distribution describes a variable without restricting the values of the other variables. A conditional distribution describes it under a specified condition.
For discrete variables, conditional probability gives
By contrast,
Thus, a marginal distribution is a probability-weighted average of conditional distributions, not usually their equally weighted average.
When a joint density exists, the analogous formula is
where the conditional density is defined for relevant values of with .
In the discrete table above,
whereas the marginal probability is .
If and are independent, conditioning on one does not change the distribution of the other, apart from qualifications concerning conditioning on null sets. Conversely, if the conditional distribution of equals its marginal distribution for -almost every , then the variables are independent.
What marginals do and do not determine
A joint distribution determines its marginals uniquely. Marginal distributions alone generally do not determine the joint distribution.
For example, let have a uniform distribution on . Each of the following constructions gives and the same uniform marginals:
- and ;
- and ;
- and , where is an independent uniform variable.
Nevertheless, the first construction has correlation , the second has correlation , and the third has independent coordinates.
Independence supplies one particular joint distribution compatible with given marginals:
For variables with a joint density, this becomes
almost everywhere. Multiplying marginal densities is therefore justified only when independence is assumed or established.
Even complete knowledge of all one-dimensional marginals does not generally determine a multivariate distribution. Higher-order dependence may remain unspecified.
Expectations and multivariate normal distributions
Any expectation involving only can be calculated from its marginal distribution. For an integrable function ,
When a joint density exists, this is equivalently
Consequently, the mean and variance of depend only on its marginal distribution. In contrast, covariance between and generally requires information about their joint distribution.
An important closed-form example is the multivariate normal distribution. If
then
Here and may themselves be vectors. Their marginal parameters are obtained by selecting the corresponding mean entries and covariance matrix blocks. The cross-covariance blocks affect their dependence and conditional distributions, but not these marginals.
Applications and computation
In Bayesian inference, marginalization removes parameters or latent variables not currently of interest. If is a target parameter, represents other unknown quantities, and denotes observed data, the marginal posterior distribution is
This preserves uncertainty about by averaging over its posterior distribution. Substituting a single estimate of generally gives a different result.
The marginal likelihood similarly integrates a likelihood over a prior distribution:
In probabilistic graphical models, computing marginal probabilities is a central inference task. Factorization and conditional independence can make some calculations substantially more efficient.
Although the mathematical operation is straightforward, exact marginalization can be computationally expensive. Direct summation over many discrete variables may require exponentially many terms, while high-dimensional integration can encounter the curse of dimensionality. Practical approaches include numerical integration, exact elimination methods, Monte Carlo methods, Markov chain Monte Carlo, and variational inference.
Marginalization should also be distinguished from optimization. Integrating over an unwanted parameter accounts for its possible values with probability weights; maximizing over it selects a value giving the largest objective. The resulting functions generally answer different statistical questions.