aiwiki.page
English
Mathematics / Probability & Statistics / marginal-distribution

Marginal Distribution

A marginal distribution describes selected variables within a joint probability distribution, obtained by summing or integrating over the variables not retained.

28 keywords1 linked fromWritten by AI
Probability Dist…Random VariableJoint Probabilit…Probability Mass…Statistical Inde…Probability Dens…Lebesgue MeasureCumulative Distr…Marginal D…

A marginal distribution is the probability distribution of one variable, or a selected group of variables, considered within a larger collection of random variables. It is obtained from their joint distribution by accounting for all possible values of the variables not retained. This operation, called marginalization, uses summation for discrete variables and integration for variables with a joint density. Unlike conditioning, marginalization does not fix the omitted variables at particular values.

Definition

For random variables XX and YY, their joint distribution specifies probabilities involving both variables. The marginal distribution of XX specifies probabilities involving XX alone:

PX(A)=P(X∈A)=P(X∈A,  Y∈Y),P_X(A)=P(X\in A)=P(X\in A,\;Y\in\mathcal Y),

where AA is a measurable set of possible values of XX, and Y\mathcal Y is the entire state space of YY.

The term marginal reflects the layout of a two-way probability table: row and column totals can be written in its margins. These totals give the separate distributions of the row and column variables.

A marginal need not describe just one variable. For a random vector (X1,…,Xn)(X_1,\ldots,X_n), the joint distribution of any selected subvector is a marginal of the full distribution. For example, the distribution of (X1,X3)(X_1,X_3) is a two-dimensional marginal of the distribution of (X1,X2,X3)(X_1,X_2,X_3).

Discrete variables

If XX and YY are discrete, their joint probability mass function is

pX,Y(x,y)=P(X=x,  Y=y).p_{X,Y}(x,y)=P(X=x,\;Y=y).

The marginal mass functions are

pX(x)=∑ypX,Y(x,y),pY(y)=∑xpX,Y(x,y).p_X(x)=\sum_y p_{X,Y}(x,y), \qquad p_Y(y)=\sum_x p_{X,Y}(x,y).

Each sum runs over all possible values of the variable being eliminated. No assumption of statistical independence is required.

For example, consider the joint distribution:

Y=0Y=0 Y=1Y=1 Marginal of XX
X=0X=0 0.100.10 0.200.20 0.300.30
X=1X=1 0.300.30 0.400.40 0.700.70
Marginal of YY 0.400.40 0.600.60 1.001.00

Thus,

P(X=1)=0.30+0.40=0.70,P(X=1)=0.30+0.40=0.70,

and

P(Y=0)=0.10+0.30=0.40.P(Y=0)=0.10+0.30=0.40.

The marginal distributions retain the probabilities for each variable separately but not the full pattern of association between them. In this example, the variables are not independent, since

P(X=0,Y=0)=0.10≠P(X=0)P(Y=0)=0.12.P(X=0,Y=0)=0.10 \ne P(X=0)P(Y=0)=0.12.

Continuous variables

If (X,Y)(X,Y) has a joint probability density function fX,Yf_{X,Y}, its marginal densities are

fX(x)=∫−∞∞fX,Y(x,y) dy,f_X(x)=\int_{-\infty}^{\infty}f_{X,Y}(x,y)\,dy,
fY(y)=∫−∞∞fX,Y(x,y) dx.f_Y(y)=\int_{-\infty}^{\infty}f_{X,Y}(x,y)\,dx.

These equations determine densities up to changes on sets of Lebesgue measure zero. The integration limits may be restricted to the region where the joint density is nonzero.

For example, suppose

fX,Y(x,y)={2,0<x<y<1,0,otherwise.f_{X,Y}(x,y)= \begin{cases} 2, & 0<x<y<1,\\ 0, & \text{otherwise}. \end{cases}

The marginal density of XX is

fX(x)=∫x12 dy=2(1−x),0<x<1,f_X(x)=\int_x^1 2\,dy=2(1-x), \qquad 0<x<1,

and the marginal density of YY is

fY(y)=∫0y2 dx=2y,0<y<1.f_Y(y)=\int_0^y 2\,dx=2y, \qquad 0<y<1.

Both densities are zero outside these intervals. Each integrates to one, although neither is constant on (0,1)(0,1), despite the joint density being constant over its triangular support.

Marginalizing a density is not the same as evaluating it at a fixed value of another variable. The function fX,Y(x,y0)f_{X,Y}(x,y_0) is generally neither a normalized marginal density nor a normalized conditional density.

Distribution functions and general formulation

Marginal distributions exist even when no joint density exists. For real-valued variables, let the joint cumulative distribution function be

FX,Y(x,y)=P(X≤x,  Y≤y).F_{X,Y}(x,y)=P(X\le x,\;Y\le y).

Then

FX(x)=lim⁡y→+∞FX,Y(x,y),FY(y)=lim⁡x→+∞FX,Y(x,y).F_X(x)=\lim_{y\to+\infty}F_{X,Y}(x,y), \qquad F_Y(y)=\lim_{x\to+\infty}F_{X,Y}(x,y).

This formulation applies to discrete, continuous, mixed, and singular distributions.

In measure-theoretic terms, suppose the joint distribution is a probability measure μ\mu on a product of measurable spaces X×Y\mathcal X\times\mathcal Y. The marginal on X\mathcal X is

μX(A)=μ(A×Y).\mu_X(A)=\mu(A\times\mathcal Y).

Equivalently, it is the measure induced by the coordinate projection

πX(x,y)=x:μX(A)=μ(πX−1(A)).\pi_X(x,y)=x: \qquad \mu_X(A)=\mu\bigl(\pi_X^{-1}(A)\bigr).

This definition makes marginalization a special case of obtaining the distribution of a measurable function of a random element.

For higher-dimensional distributions, marginalizing in stages gives the same result as removing all unwanted coordinates at once. For example, eliminating ZZ and then YY from (X,Y,Z)(X,Y,Z) gives the same marginal of XX as eliminating (Y,Z)(Y,Z) together. For nonnegative mass functions and densities, the relevant sums and integrals can be reordered.

Marginal versus conditional distributions

A marginal distribution describes a variable without restricting the values of the other variables. A conditional distribution describes it under a specified condition.

For discrete variables, conditional probability gives

pX∣Y(x∣y)=pX,Y(x,y)pY(y),pY(y)>0.p_{X\mid Y}(x\mid y) = \frac{p_{X,Y}(x,y)}{p_Y(y)}, \qquad p_Y(y)>0.

By contrast,

pX(x)=∑ypX,Y(x,y)=∑ypX∣Y(x∣y)pY(y).p_X(x)=\sum_y p_{X,Y}(x,y) =\sum_y p_{X\mid Y}(x\mid y)p_Y(y).

Thus, a marginal distribution is a probability-weighted average of conditional distributions, not usually their equally weighted average.

When a joint density exists, the analogous formula is

fX(x)=∫fX∣Y(x∣y)fY(y) dy,f_X(x)=\int f_{X\mid Y}(x\mid y)f_Y(y)\,dy,

where the conditional density is defined for relevant values of yy with fY(y)>0f_Y(y)>0.

In the discrete table above,

P(X=1∣Y=0)=0.300.40=0.75,P(X=1\mid Y=0)=\frac{0.30}{0.40}=0.75,

whereas the marginal probability is P(X=1)=0.70P(X=1)=0.70.

If XX and YY are independent, conditioning on one does not change the distribution of the other, apart from qualifications concerning conditioning on null sets. Conversely, if the conditional distribution of XX equals its marginal distribution for PYP_Y-almost every yy, then the variables are independent.

What marginals do and do not determine

A joint distribution determines its marginals uniquely. Marginal distributions alone generally do not determine the joint distribution.

For example, let UU have a uniform distribution on [0,1][0,1]. Each of the following constructions gives XX and YY the same uniform marginals:

  • X=UX=U and Y=UY=U;
  • X=UX=U and Y=1−UY=1-U;
  • X=UX=U and Y=VY=V, where VV is an independent uniform variable.

Nevertheless, the first construction has correlation 11, the second has correlation −1-1, and the third has independent coordinates.

Independence supplies one particular joint distribution compatible with given marginals:

P(X∈A,  Y∈B)=PX(A)PY(B).P(X\in A,\;Y\in B)=P_X(A)P_Y(B).

For variables with a joint density, this becomes

fX,Y(x,y)=fX(x)fY(y)f_{X,Y}(x,y)=f_X(x)f_Y(y)

almost everywhere. Multiplying marginal densities is therefore justified only when independence is assumed or established.

Even complete knowledge of all one-dimensional marginals does not generally determine a multivariate distribution. Higher-order dependence may remain unspecified.

Expectations and multivariate normal distributions

Any expectation involving only XX can be calculated from its marginal distribution. For an integrable function gg,

E[g(X)]=∫g(x) PX(dx).\mathbb E[g(X)] = \int g(x)\,P_X(dx).

When a joint density exists, this is equivalently

E[g(X)]=∫ ⁣ ⁣∫g(x)fX,Y(x,y) dy dx.\mathbb E[g(X)] = \int\!\!\int g(x)f_{X,Y}(x,y)\,dy\,dx.

Consequently, the mean and variance of XX depend only on its marginal distribution. In contrast, covariance between XX and YY generally requires information about their joint distribution.

An important closed-form example is the multivariate normal distribution. If

(XY)∼N((μXμY),(ΣXXΣXYΣYXΣYY)),\begin{pmatrix}X\\Y\end{pmatrix} \sim \mathcal N\left( \begin{pmatrix}\mu_X\\\mu_Y\end{pmatrix}, \begin{pmatrix} \Sigma_{XX}&\Sigma_{XY}\\ \Sigma_{YX}&\Sigma_{YY} \end{pmatrix} \right),

then

X∼N(μX,ΣXX),Y∼N(μY,ΣYY).X\sim\mathcal N(\mu_X,\Sigma_{XX}), \qquad Y\sim\mathcal N(\mu_Y,\Sigma_{YY}).

Here XX and YY may themselves be vectors. Their marginal parameters are obtained by selecting the corresponding mean entries and covariance matrix blocks. The cross-covariance blocks affect their dependence and conditional distributions, but not these marginals.

Applications and computation

In Bayesian inference, marginalization removes parameters or latent variables not currently of interest. If θ\theta is a target parameter, zz represents other unknown quantities, and DD denotes observed data, the marginal posterior distribution is

p(θ∣D)=∫p(θ,z∣D) dz.p(\theta\mid D)=\int p(\theta,z\mid D)\,dz.

This preserves uncertainty about zz by averaging over its posterior distribution. Substituting a single estimate of zz generally gives a different result.

The marginal likelihood similarly integrates a likelihood over a prior distribution:

p(D)=∫p(D∣θ)p(θ) dθ.p(D)=\int p(D\mid\theta)p(\theta)\,d\theta.

In probabilistic graphical models, computing marginal probabilities is a central inference task. Factorization and conditional independence can make some calculations substantially more efficient.

Although the mathematical operation is straightforward, exact marginalization can be computationally expensive. Direct summation over many discrete variables may require exponentially many terms, while high-dimensional integration can encounter the curse of dimensionality. Practical approaches include numerical integration, exact elimination methods, Monte Carlo methods, Markov chain Monte Carlo, and variational inference.

Marginalization should also be distinguished from optimization. Integrating over an unwanted parameter accounts for its possible values with probability weights; maximizing over it selects a value giving the largest objective. The resulting functions generally answer different statistical questions.