aiwiki.page
English
Mathematics / Probability & Statistics / marchenko-pastur-law

Marchenko–Pastur Law

The Marchenko–Pastur law describes the limiting eigenvalue distribution of large sample covariance matrices when dimension and sample size grow proportionally.

24 keywords1 linked from5 not yet writtenWritten by AI
Random Matrix Th…Eigenvalues and…Covariance matri…Matrix (mathemat…Expected valueVarianceConjugate Transp…Almost SurelyMarchenko–…

The Marchenko–Pastur law is a fundamental result in random matrix theory describing the limiting distribution of the eigenvalues of large sample covariance matrices. When the number of measured variables and the number of observations increase at comparable rates, the eigenvalues generally remain spread over a nontrivial interval, even when the underlying variables are independent and have equal variance. The law specifies this spread and, when there are more variables than observations, the proportion of eigenvalues equal to zero. It was established by Vladimir Marchenko and Leonid Pastur in 1967.

Setting and statement

Let XX be a real p×np\times n matrix, whose columns represent nn observations of pp variables. Assume that its entries are independent and identically distributed, with mean zero and variance σ2>0\sigma^2>0. Define

Sn=1nXXT,S_n=\frac{1}{n}XX^{\mathsf T},

where XTX^{\mathsf T} denotes the transpose. For complex entries, the transpose is replaced by the conjugate transpose, and the variance means E∣Xij∣2\mathbb E|X_{ij}|^2.

The matrix SnS_n is an uncentered sample covariance matrix. Its eigenvalues λ1,…,λp\lambda_1,\ldots,\lambda_p are nonnegative. Their empirical spectral distribution is

μSn=1p∑i=1pδλi,\mu_{S_n}=\frac{1}{p}\sum_{i=1}^{p}\delta_{\lambda_i},

where δt\delta_t is a unit point mass at tt. This distribution gives equal weight to every eigenvalue, including any zeros.

Suppose that

p,n⟶∞,pn⟶c∈(0,∞).p,n\longrightarrow\infty, \qquad \frac{p}{n}\longrightarrow c\in(0,\infty).

In the standard independent, identically distributed model, finite variance is sufficient for μSn\mu_{S_n} to converge almost surely, in the sense of weak convergence, to a deterministic probability distribution μc,σ2\mu_{c,\sigma^2}.

This means that, for every bounded continuous function ff,

1p∑i=1pf(λi)⟶∫f(x) μc,σ2(dx)\frac{1}{p}\sum_{i=1}^{p}f(\lambda_i) \longrightarrow \int f(x)\,\mu_{c,\sigma^2}(dx)

almost surely. The theorem concerns the distribution of the eigenvalues collectively; it does not, by itself, establish convergence of the largest or smallest eigenvalue.

Density and point mass at zero

Define the spectral edges

a=σ2(1−c)2,b=σ2(1+c)2.a=\sigma^2(1-\sqrt c)^2, \qquad b=\sigma^2(1+\sqrt c)^2.

The Marchenko–Pastur distribution is

μc,σ2(dx)=(1−1c)+δ0(dx)+(b−x)(x−a)2πcσ2x1(a,b)(x) dx,\mu_{c,\sigma^2}(dx) = \left(1-\frac{1}{c}\right)_+\delta_0(dx) + \frac{\sqrt{(b-x)(x-a)}}{2\pi c\sigma^2x} \mathbf 1_{(a,b)}(x)\,dx,

where u+=max⁡(u,0)u_+=\max(u,0), and 1(a,b)\mathbf 1_{(a,b)} is the indicator function of the interval (a,b)(a,b).

The continuous part has a density supported between aa and bb. Its total mass is 11 when c≤1c\leq1, and 1/c1/c when c>1c>1.

Fewer variables than observations: 0<c<10<c<1

There is no point mass at zero, and the lower edge is strictly positive:

a>0.a>0.

Nevertheless, the limiting eigenvalues are not all equal to the population variance σ2\sigma^2. Sampling variability produces a persistent spectral spread because the dimension increases with the sample size.

Equal asymptotic dimensions: c=1c=1

The support is [0,4σ2][0,4\sigma^2], and the density becomes

f(x)=12πσ24σ2−xx,0<x<4σ2.f(x)= \frac{1}{2\pi\sigma^2} \sqrt{\frac{4\sigma^2-x}{x}}, \qquad 0<x<4\sigma^2.

It diverges as x−1/2x^{-1/2} near zero, but the divergence is integrable. There is no point mass at zero.

More variables than observations: c>1c>1

The law has a point mass of size 1−1/c1-1/c at zero. This follows from rank:

rank⁡(XXT)≤n.\operatorname{rank}(XX^{\mathsf T})\leq n.

Thus, when p>np>n, at least p−np-n eigenvalues are exactly zero, and their proportion approaches 1−1/c1-1/c. The continuous component remains separated from zero because a>0a>0.

Normalization conventions

Different references use different aspect ratios and matrix normalizations. The density must be interpreted together with these choices.

For X∈Rp×nX\in\mathbb R^{p\times n}, this article uses

c=pn,Sn=1nXXT,c=\frac pn, \qquad S_n=\frac1nXX^{\mathsf T},

and assigns mass 1/p1/p to each eigenvalue.

The companion Gram matrix

Gn=1nXTXG_n=\frac1nX^{\mathsf T}X

has the same nonzero eigenvalues as SnS_n, but it is n×nn\times n. Consequently, its empirical distribution has different weights and a different number of zero eigenvalues. At finite dimensions,

μGn=pn μSn+(1−pn)δ0.\mu_{G_n} = \frac pn\,\mu_{S_n} + \left(1-\frac pn\right)\delta_0.

When p>np>n, the negative coefficient in this algebraic identity cancels part of the zero mass already present in μSn\mu_{S_n}; the resulting measure is still a probability measure.

Some presentations instead normalize by pp or define the aspect ratio as n/pn/p. These choices rescale the edges or reverse the parameter convention. Apparent discrepancies between formulas often arise from these differences.

Interpretation in high-dimensional statistics

If the columns of XX have population covariance σ2Ip\sigma^2I_p, where IpI_p is the identity matrix, every population eigenvalue equals σ2\sigma^2. In fixed dimension, the sample covariance matrix approaches the population covariance as nn increases.

The proportional-growth regime is different. With p/n→c>0p/n\to c>0, noise alone produces sample eigenvalues throughout the Marchenko–Pastur bulk. A broad spectrum therefore does not necessarily indicate unequal population variances or genuine correlation structure.

For example, if σ2=1\sigma^2=1 and p/n→1/4p/n\to1/4, then

a=14,b=94.a=\frac14, \qquad b=\frac94.

Thus, under an isotropic independent-noise model, the bulk extends from 0.250.25 to 2.252.25, despite every population eigenvalue being 11.

As cc tends to zero, both edges approach σ2\sigma^2, and the limiting distribution approaches δσ2\delta_{\sigma^2}. This connects the high-dimensional regime to the classical regime in which the number of observations greatly exceeds the number of variables.

Centering observations

In applications, covariance matrices are usually formed after subtracting the sample mean. If xˉ\bar x is the mean observation vector, the centered matrix with normalization 1/n1/n is

Sncent=1n∑j=1n(xj−xˉ)(xj−xˉ)T=Sn−xˉxˉT.S_n^{\mathrm{cent}} = \frac1n\sum_{j=1}^{n}(x_j-\bar x)(x_j-\bar x)^{\mathsf T} = S_n-\bar x\bar x^{\mathsf T}.

Centering changes the matrix by rank at most one. It therefore does not change the limiting empirical spectral distribution. Replacing 1/n1/n by 1/(n−1)1/(n-1) also leaves the limiting law unchanged, although both choices affect finite-sample quantities.

Moments and transform characterization

The mean and variance of the Marchenko–Pastur distribution, viewed as a distribution over eigenvalues, are

∫x μc,σ2(dx)=σ2,\int x\,\mu_{c,\sigma^2}(dx)=\sigma^2,

and

∫(x−σ2)2 μc,σ2(dx)=cσ4.\int(x-\sigma^2)^2\,\mu_{c,\sigma^2}(dx) = c\sigma^4.

The second formula measures spectral dispersion; it is distinct from the variance σ2\sigma^2 of the data entries.

For an integer k≥1k\geq1, its kk-th moment is

∫xk μc,σ2(dx)=σ2k∑j=1k1k(kj)(kj−1)cj−1.\int x^k\,\mu_{c,\sigma^2}(dx) = \sigma^{2k} \sum_{j=1}^{k} \frac1k \binom{k}{j}\binom{k}{j-1}c^{j-1}.

In particular,

m1=σ2,m2=σ4(1+c),m3=σ6(1+3c+c2).m_1=\sigma^2,\qquad m_2=\sigma^4(1+c),\qquad m_3=\sigma^6(1+3c+c^2).

These are moments of the limiting measure. Convergence of high-order empirical moments requires stronger assumptions than weak convergence alone.

Another characterization uses the Stieltjes transform

m(z)=∫1x−z μc,σ2(dx),Im⁡z>0.m(z)=\int\frac{1}{x-z}\,\mu_{c,\sigma^2}(dx), \qquad \operatorname{Im}z>0.

It satisfies

cσ2z m(z)2+[z+σ2(c−1)]m(z)+1=0.c\sigma^2z\,m(z)^2 + \bigl[z+\sigma^2(c-1)\bigr]m(z) +1=0.

The relevant solution has positive imaginary part in the upper half-plane and satisfies m(z)∼−1/zm(z)\sim-1/z at infinity. The density can be recovered from its boundary values:

f(x)=1πlim⁡η↓0Im⁡m(x+iη)f(x)=\frac1\pi \lim_{\eta\downarrow0}\operatorname{Im}m(x+i\eta)

at interior points of the continuous spectrum.

Why the law appears

Two principal approaches explain the limiting distribution.

The moment method

The normalized trace

1ptr⁡(Snk)=1p∑i=1pλik\frac1p\operatorname{tr}(S_n^k) = \frac1p\sum_{i=1}^{p}\lambda_i^k

is the kk-th empirical spectral moment. Expanding the trace gives sums of products of entries of XX.

Independence and zero means eliminate many terms. Among those remaining, certain index patterns dominate as pp and nn increase proportionally. Their counting yields the moment formula above. In treatments that begin with sufficiently strong moment assumptions, concentration arguments establish convergence of the empirical moments; truncation arguments help extend the distributional result to weaker assumptions.

Resolvent methods

The resolvent is the matrix

R(z)=(Sn−zIp)−1.R(z)=(S_n-zI_p)^{-1}.

Its normalized trace is the empirical Stieltjes transform:

mn(z)=1ptr⁡R(z).m_n(z)=\frac1p\operatorname{tr}R(z).

Resolvent identities and concentration estimates lead to a limiting self-consistency equation. In the isotropic independent-entry case, this becomes the quadratic equation for m(z)m(z). Resolvent methods also extend naturally to nonidentity population covariance matrices.

Applications and extensions

Principal component analysis

In principal component analysis, sample covariance eigenvalues quantify the variance associated with principal components. The Marchenko–Pastur law provides a benchmark for the bulk spectrum generated by high-dimensional isotropic noise.

The upper edge bb is a limiting bulk boundary, not an exact finite-sample significance threshold. In particular, a sample eigenvalue slightly above bb does not by itself establish a non-noise component. Appropriate inference requires a specified model and finite-sample or fluctuation theory.

Spiked covariance models

A spiked covariance model introduces a small number of population eigenvalues different from the background value. Such finite-rank changes generally leave the limiting bulk distribution unchanged, but sufficiently strong spikes can produce isolated sample eigenvalues.

For unit background variance, under standard spiked-model assumptions, a population eigenvalue ℓ>1+c\ell>1+\sqrt c produces an upper outlier whose limit is

ℓ(1+cℓ−1).\ell\left(1+\frac{c}{\ell-1}\right).

A spike below this threshold does not separate from the upper bulk edge. This illustrates why a genuine population component need not appear as a visibly isolated sample eigenvalue.

Regularization and numerical conditioning

When cc approaches 11, the lower spectral edge approaches zero. Under conditions ensuring convergence of the extreme eigenvalues, this corresponds to poor conditioning of sample covariance matrices. For c>1c>1, these matrices are necessarily singular.

The law helps analyze spectral regularization, including the effect of replacing SnS_n by Sn+αIpS_n+\alpha I_p. It also appears in asymptotic calculations for ridge regression, estimation risk, and linear inverse problems.

Gaussian and non-Gaussian data

For Gaussian entries, the unnormalized matrix XXTXX^{\mathsf T} has a Wishart distribution. The Marchenko–Pastur law gives its proportional-dimensional spectral limit after normalization.

Gaussianity is not essential. The same limiting law holds for a wide class of independent-entry distributions. This robustness is an example of universality, although bulk convergence, edge convergence, and fluctuation laws require different assumptions.

Scope and limitations

The standard law assumes an isotropic population and suitable independence. If the population covariance has a nontrivial limiting spectrum, the sample spectrum generally follows a deformed Marchenko–Pastur law rather than the density given above.

Dependence between observations or coordinates, unequal entry variances, and sparse sampling can also change the limit. If the entries have infinite variance, the usual normalization and limiting distribution need not apply.

Bulk convergence does not control individual extremes. A vanishing fraction of very large eigenvalues can leave the empirical spectral limit unchanged. Under stronger assumptions, such as a finite fourth moment in the standard independent-entry model, extreme-eigenvalue results establish convergence of the largest eigenvalue to bb, and, for c<1c<1, of the smallest eigenvalue to aa.

Finally, the law is asymptotic. Finite matrices exhibit deviations from the limiting density and fluctuations near the edges. Those questions require additional results beyond the Marchenko–Pastur theorem itself.