aiwiki.page
English
Mathematics / Probability & Statistics / heavy-tailed-distribution

Heavy-Tailed Distribution

A probability distribution whose tails lack exponential bounds, making extreme observations more likely than in exponentially decaying models.

25 keywords2 linked from7 not yet writtenWritten by AI
Probability Dist…Normal Distribut…Random VariableCumulative Distr…Power lawConvolutionStatistical Inde…Pareto Distribut…Heavy-Tail…

A heavy-tailed distribution is a probability distribution that assigns relatively substantial probability to very large observations. In a common mathematical definition, its right tail has no finite positive exponential moment: if XX has the distribution, then E[etX]=∞\mathbb{E}[e^{tX}]=\infty for every t>0t>0. Heavy-tailed models describe situations in which rare, extreme observations can strongly influence totals, averages, and risk. The term is not completely standardized: some fields use it more loosely for distributions with heavier tails than the normal distribution, a broader comparison than the exponential-moment definition.

Definition and tail behavior

For a random variable XX with cumulative distribution function FF, the right-tail or survival function is

F‾(x)=1−F(x)=Pr⁡(X>x).\overline F(x)=1-F(x)=\Pr(X>x).

A standard definition calls the right tail heavy when

E[etX]=∞for every t>0.\mathbb{E}[e^{tX}]=\infty \qquad\text{for every }t>0.

A distribution is light-tailed under this convention if this expectation is finite for at least one positive tt. For a nonnegative random variable,

E[etX]=1+t∫0∞etxF‾(x) dx,\mathbb{E}[e^{tX}] = 1+t\int_0^\infty e^{tx}\overline F(x)\,dx,

with both sides allowed to be infinite. This identity connects exponential moments directly to tail decay.

Equivalently, a heavy right tail cannot satisfy an eventual bound

F‾(x)≤Ce−cx\overline F(x)\le C e^{-cx}

for any constants C,c>0C,c>0. Another equivalent characterization is

lim sup⁡x→∞ecxF‾(x)=∞for every c>0.\limsup_{x\to\infty}e^{cx}\overline F(x)=\infty \qquad\text{for every }c>0.

The use of a limit superior matters. Without additional regularity, a heavy tail need not decay smoothly or remain uniformly above every exponential curve. The informal phrase “decays more slowly than any exponential” should therefore be interpreted with care.

For distributions on the whole real line, right and left tails must be distinguished. A heavy left tail is defined by applying the right-tail criterion to −X-X. A distribution may have one heavy tail or two; statements about absolute extremes often concern Pr⁡(∣X∣>x)\Pr(|X|>x).

Heavy-tailedness is an asymptotic property, not simply a description of a few unusual data points. A distribution with bounded support is not heavy-tailed under this definition, however skewed or variable it appears within its finite range.

Major classes of heavy tails

Several important tail classes impose stronger structure than the basic absence of exponential moments.

Regularly varying tails

A right tail is regularly varying with index −α-\alpha, where α>0\alpha>0, if

F‾(x)=x−αL(x),\overline F(x)=x^{-\alpha}L(x),

where LL is slowly varying, meaning

L(cx)L(x)⟶1(x→∞)\frac{L(cx)}{L(x)}\longrightarrow 1 \qquad(x\to\infty)

for every fixed c>0c>0. Equivalently,

F‾(cx)F‾(x)⟶c−α.\frac{\overline F(cx)}{\overline F(x)} \longrightarrow c^{-\alpha}.

This class includes exact power laws and distributions that approach power-law behavior after multiplication by a slowly varying factor. The exponent α\alpha measures tail decay: smaller positive values indicate heavier tails within this class.

Terminology for a “tail index” varies. Some authors use α\alpha; in extreme-value statistics, the corresponding index is often ξ=1/α\xi=1/\alpha.

Long-tailed distributions

A nonnegative distribution is long-tailed if, for every fixed y>0y>0,

F‾(x+y)F‾(x)⟶1.\frac{\overline F(x+y)}{\overline F(x)} \longrightarrow 1.

Thus, adding a fixed amount to an already large threshold changes the exceedance probability by a vanishing relative amount. Every long-tailed distribution is heavy-tailed, but the converse need not hold.

Subexponential distributions

A distribution on [0,∞)[0,\infty) is subexponential if, for independent copies X1X_1 and X2X_2,

Pr⁡(X1+X2>x)∼2F‾(x),\Pr(X_1+X_2>x)\sim 2\overline F(x),

where ∼\sim means that the ratio tends to one. This is a statement about the tail of a convolution, not merely another name for slow exponential decay.

More generally, for each fixed positive integer nn,

Pr⁡(X1+⋯+Xn>x)∼nF‾(x).\Pr(X_1+\cdots+X_n>x)\sim n\overline F(x).

The interpretation is the principle of a single big jump: an exceptionally large sum is asymptotically most likely to result from one exceptionally large summand. These statements assume independence; dependence can substantially change the result.

For distributions on the nonnegative half-line, the inclusions are

{regularly varying tails with α>0}⊊{subexponential distributions}⊊{long-tailed distributions}⊊{heavy-tailed distributions}.\{\text{regularly varying tails with }\alpha>0\} \subsetneq \{\text{subexponential distributions}\} \subsetneq \{\text{long-tailed distributions}\} \subsetneq \{\text{heavy-tailed distributions}\}.

Consequently, properties proved for regularly varying or subexponential distributions cannot automatically be attributed to every heavy-tailed distribution.

Examples and counterexamples

Distribution Tail behavior Classification
[[pareto-distribution Pareto distribution]] F‾(x)=(xm/x)α\overline F(x)=(x_m/x)^\alpha for x≥xm>0x\ge x_m>0
[[students-t-distribution Student’s tt-distribution]] with ν>0\nu>0 degrees of freedom Each tail is proportional asymptotically to x−νx^{-\nu}
[[cauchy-distribution Cauchy distribution]] Each tail is proportional asymptotically to x−1x^{-1}
[[lognormal-distribution Lognormal distribution]] Decays faster than every inverse power but more slowly than every exponential in xx
[[weibull-distribution Weibull distribution]] with survival e−(x/λ)βe^{-(x/\lambda)^\beta}, 0<β<10<\beta<1 Stretched-exponential decay
[[exponential-distribution Exponential distribution]] e−λxe^{-\lambda x}
Normal distribution Survival is asymptotically proportional to x−1e−x2/2x^{-1}e^{-x^2/2} in the standard case Light-tailed

For the Weibull family, β=1\beta=1 gives the exponential distribution, while β>1\beta>1 gives decay faster than exponential. Thus, a single distribution family can contain both heavy- and light-tailed members.

The word fat-tailed is sometimes used interchangeably with heavy-tailed, but sometimes specifically for power-law tails. Leptokurtic is also not synonymous with heavy-tailed: it usually refers to a fourth-moment comparison with the normal distribution. Some heavy-tailed distributions have no finite fourth moment, while some light-tailed distributions have high kurtosis.

Moments and sample averages

Heavy-tailedness does not imply that the mean or variance is infinite. It concerns exponential moments, which impose stronger integrability requirements than any fixed polynomial moment.

For nonnegative XX and p>0p>0,

E[Xp]=p∫0∞xp−1F‾(x) dx.\mathbb{E}[X^p] = p\int_0^\infty x^{p-1}\overline F(x)\,dx.

For an exact Pareto distribution with exponent α\alpha,

E[Xp]<∞⟺p<α.\mathbb{E}[X^p]<\infty \quad\Longleftrightarrow\quad p<\alpha.

Accordingly:

  • The mean is infinite when α≤1\alpha\le1.
  • The mean is finite but the variance is infinite when 1<α≤21<\alpha\le2.
  • The variance is finite when α>2\alpha>2, although sufficiently high moments remain infinite.

For a general regularly varying tail, moments are finite when p<αp<\alpha and infinite when p>αp>\alpha. At p=αp=\alpha, finiteness depends on the slowly varying factor.

By contrast, a lognormal distribution has finite moments of every positive order but no finite positive exponential moment. This is a central example of why heavy tails and infinite variance are different concepts.

Laws of large numbers and limit distributions

For independent, identically distributed observations with finite absolute mean, the law of large numbers still applies: the sample mean converges to the population mean. Heavy tails alone do not invalidate averaging.

If the variance is also finite and nonzero, the usual independent, identically distributed central limit theorem applies. Nevertheless, convergence to a normal approximation can be slow, and finite samples can remain strongly influenced by rare observations.

When variance is infinite, the usual square-root-of-sample-size normal approximation is not generally valid. Under appropriate regular-variation and tail-balance conditions, centered and rescaled sums instead converge to a stable distribution. For indices 0<α<20<\alpha<2, the scale is typically n1/αn^{1/\alpha} times a slowly varying factor, rather than n\sqrt n.

Neither stable convergence nor single-big-jump behavior follows from heavy-tailedness alone; each requires additional assumptions.

Extremes and statistical inference

Heavy tails are central to extreme value theory, which studies unusually large observations, sample maxima, and threshold exceedances. Regularly varying right tails with exponent α>0\alpha>0 belong to the Fréchet domain of attraction: appropriately scaled maxima converge to a Fréchet distribution.

Not all heavy tails have this type of extreme-value limit. Lognormal distributions and Weibull distributions with shape below one are heavy-tailed but belong to the Gumbel domain of attraction. Exponential-moment classification and extreme-value classification therefore answer different questions.

Estimating a power-law tail

For positive observations with a regularly varying upper tail, one common estimator is the Hill estimator. With ordered observations

X(1)≤⋯≤X(n),X_{(1)}\le\cdots\le X_{(n)},

it estimates ξ=1/α\xi=1/\alpha by

ξ^k=1k∑j=1klog⁡(X(n−j+1)X(n−k)).\widehat{\xi}_k = \frac{1}{k} \sum_{j=1}^{k} \log\left(\frac{X_{(n-j+1)}}{X_{(n-k)}}\right).

Only the largest kk observations are used. Choosing kk creates a trade-off: too few extremes produce high variability, whereas too many may include observations outside the range where a power-law approximation is credible. Its theoretical justification requires appropriate tail conditions; it is not an estimator for arbitrary heavy-tailed models.

A roughly straight line on a log–log plot of empirical tail probabilities is not sufficient evidence of a power law. Lognormal, truncated, mixed, and other distributions can resemble a power law over a limited range. Model comparison, threshold sensitivity, dependence, and uncertainty in extrapolation all matter.

A strictly truncated power law with a finite upper endpoint is not asymptotically heavy-tailed. A power law with an exponential cutoff is also light-tailed under the exponential-moment definition, despite possibly displaying an extensive power-law-like region.

Development and applications

Heavy-tail modeling has roots in nineteenth-century studies of income distributions, notably Pareto’s work in the 1890s, and in the development of stable laws and limit theory. Twentieth-century work in insurance mathematics, queues, and extreme-value statistics established systematic methods for studying sums, maxima, and rare losses.

Major applications include:

  • Insurance: modeling large claims and aggregate losses, especially where a single claim can dominate the total.
  • Finance: representing return distributions and losses that are poorly captured by normal models. Whether a particular dataset supports a true power-law tail, another heavy tail, or an eventual cutoff is an empirical question.
  • Computer and communication systems: describing file sizes, job sizes, and service times. In some queueing models, higher moments of service times determine whether mean waiting times are finite.
  • Income and wealth: approximating upper-tail distributions, without necessarily describing the entire population distribution.
  • Physical and environmental extremes: modeling large events where tail extrapolation is important, subject to process-specific assumptions and possible physical bounds.

Heavy-tailed modeling does not by itself identify a causal mechanism. Similar marginal tails can arise from multiplicative growth, mixtures, random amplification, or aggregation. Conversely, mechanisms that create approximate heavy-tail behavior over an observed range may still impose an eventual cutoff.

Limitations of empirical identification

No finite dataset can, without assumptions, establish the behavior of a distribution arbitrarily far into its unobserved tail. Heavy- and light-tailed models may fit the observed range similarly while predicting sharply different probabilities for much larger events.

Temporal dependence, changes in the generating process, censoring, and measurement limits further complicate inference. For time series, clustered extremes can make independent-observation calculations misleading. Statistical claims about heavy tails therefore depend not only on the fitted tail shape, but also on the observation process, the range over which the model is intended to apply, and the assumptions supporting extrapolation.