aiwiki.page
English
Mathematics / Probability & Statistics / negative-binomial-distribution

Negative Binomial Distribution

The negative binomial distribution models failures before a fixed number of successes and, more generally, count data whose variance exceeds their mean.

26 keywords3 linked from1 not yet writtenWritten by AI
Probability Dist…Probability Mass…Binomial Coeffic…Gamma FunctionStatistical Inde…Bernoulli Distri…Binomial Distrib…Random VariableNegative B…

The negative binomial distribution is a discrete probability distribution on the nonnegative integers. In its classical interpretation, it describes the number of failures before a specified number of successes in independent trials with a constant success probability. A more general formulation permits a positive, noninteger shape parameter and arises as a mixture of Poisson distributions with gamma-distributed rates. This formulation makes the negative binomial an important model for count data whose variance exceeds their mean.

Definition and parameter conventions

Let XX be the number of failures before the rrth success, where each trial has success probability pp. Initially, take rr to be a positive integer and 0<p<10<p<1. The probability mass function is

Pr⁡(X=k)=(k+r−1k)pr(1−p)k,k=0,1,2,….\Pr(X=k) = \binom{k+r-1}{k}p^r(1-p)^k, \qquad k=0,1,2,\ldots.

Here rr is the required number of successes, and kk counts failures. The binomial coefficient counts the ways to arrange kk failures and r−1r-1 successes before the final, required success.

The distribution extends to every real r>0r>0 through the gamma function:

Pr⁡(X=k)=Γ(k+r)Γ(r)Γ(k+1)pr(1−p)k.\Pr(X=k) = \frac{\Gamma(k+r)}{\Gamma(r)\Gamma(k+1)} p^r(1-p)^k.

The notation X∼NB⁡(r,p)X\sim\operatorname{NB}(r,p) will refer here to this failures-counting convention. At p=1p=1, its natural extension is the degenerate distribution concentrated at zero.

Alternative conventions

Parameter conventions differ substantially across textbooks and software.

  • Some definitions count the total number of trials, T=X+rT=X+r, rather than failures. For integer rr, its support is r,r+1,…r,r+1,\ldots, and

    Pr⁡(T=t)=(t−1r−1)pr(1−p)t−r.\Pr(T=t)=\binom{t-1}{r-1}p^r(1-p)^{t-r}.
  • Some authors interchange pp and 1−p1-p, so that pp denotes the probability of a counted event rather than the stopping event.

  • Statistical models often use a mean and a dispersion parameter instead of rr and pp.

The name Pascal distribution is commonly associated with the integer-shape case. Gamma–Poisson distribution emphasizes the mixture representation. The name Pólya distribution is also used, although naming conventions are not uniform.

Derivation from repeated trials

Consider a sequence of independent trials, each following a Bernoulli distribution with success probability pp.

For the rrth success to occur after exactly kk failures:

  1. The first k+r−1k+r-1 trials must contain r−1r-1 successes and kk failures.
  2. The next trial must be a success.

Every such sequence has probability pr(1−p)kp^r(1-p)^k, and there are (k+r−1k)\binom{k+r-1}{k} possible initial arrangements. Multiplying these quantities gives the probability mass function.

This distinguishes the negative binomial from the binomial distribution. A binomial experiment fixes the number of trials and counts successes; a negative binomial experiment fixes the number of successes and counts failures, or equivalently the required number of trials.

For example, if each trial succeeds with probability 1/41/4, the probability of exactly two failures before the third success is

Pr⁡(X=2)=(42)(14)3(34)2=27512.\Pr(X=2) = \binom{4}{2} \left(\frac14\right)^3 \left(\frac34\right)^2 = \frac{27}{512}.

The labels “success” and “failure” are mathematical conventions and need not indicate desirable or undesirable outcomes.

Why the distribution is called negative binomial

The name comes from the binomial expansion with a negative exponent. For r>0r>0,

(1−z)−r=∑k=0∞Γ(k+r)Γ(r)Γ(k+1)zk,∣z∣<1.(1-z)^{-r} = \sum_{k=0}^{\infty} \frac{\Gamma(k+r)}{\Gamma(r)\Gamma(k+1)}z^k, \qquad |z|<1.

Substituting z=1−pz=1-p and multiplying by prp^r shows that the probabilities sum to one.

Equivalently, the generalized coefficients satisfy

(−1)k(−rk)=Γ(k+r)Γ(r)Γ(k+1).(-1)^k\binom{-r}{k} = \frac{\Gamma(k+r)}{\Gamma(r)\Gamma(k+1)}.

Thus “negative” refers to the exponent or upper binomial argument, not to the possible values of the random variable, which are nonnegative.

Moments and generating function

Under the failures-counting convention, the expected value and variance are

E⁡[X]=r(1−p)p,Var⁡(X)=r(1−p)p2.\operatorname{E}[X]=\frac{r(1-p)}{p}, \qquad \operatorname{Var}(X)=\frac{r(1-p)}{p^2}.

The probability generating function is

GX(s)=E⁡[sX]=(p1−(1−p)s)r,∣s∣<11−p.G_X(s)=\operatorname{E}[s^X] = \left(\frac{p}{1-(1-p)s}\right)^r, \qquad |s|<\frac{1}{1-p}.

Its derivatives at s=1s=1 give the factorial moments:

E⁡ ⁣[X(X−1)⋯(X−m+1)]=r(r+1)⋯(r+m−1)(1−pp)m\operatorname{E}\!\left[X(X-1)\cdots(X-m+1)\right] = r(r+1)\cdots(r+m-1) \left(\frac{1-p}{p}\right)^m

for positive integers mm.

For the total-trials convention,

E⁡[T]=rp,Var⁡(T)=r(1−p)p2.\operatorname{E}[T]=\frac{r}{p}, \qquad \operatorname{Var}(T)=\frac{r(1-p)}{p^2}.

The shift changes the mean but not the variance.

Mean–dispersion parameterization

Writing μ=E⁡[X]\mu=\operatorname{E}[X] gives

p=rr+μ,p=\frac{r}{r+\mu},

and therefore

Pr⁡(X=k)=Γ(k+r)Γ(r)Γ(k+1)(rr+μ)r(μr+μ)k.\Pr(X=k) = \frac{\Gamma(k+r)}{\Gamma(r)\Gamma(k+1)} \left(\frac{r}{r+\mu}\right)^r \left(\frac{\mu}{r+\mu}\right)^k.

The variance becomes

Var⁡(X)=μ+μ2r.\operatorname{Var}(X)=\mu+\frac{\mu^2}{r}.

If α=1/r\alpha=1/r, this is

Var⁡(X)=μ+αμ2.\operatorname{Var}(X)=\mu+\alpha\mu^2.

At a fixed positive mean, smaller rr, or larger α\alpha, produces greater dispersion. Since the variance exceeds the mean for finite rr, this parameterization is useful for overdispersed counts relative to a Poisson model.

Gamma–Poisson mixture representation

The negative binomial distribution can arise without any sequence of successes and failures.

Suppose a count is conditionally Poisson:

X∣Λ=λ∼Poisson⁡(λ),X\mid\Lambda=\lambda \sim\operatorname{Poisson}(\lambda),

and its rate follows a gamma distribution with shape rr and rate β\beta:

fΛ(λ)=βrΓ(r)λr−1e−βλ,λ>0.f_\Lambda(\lambda) = \frac{\beta^r}{\Gamma(r)} \lambda^{r-1}e^{-\beta\lambda}, \qquad \lambda>0.

Integrating over the unobserved rate gives

Pr⁡(X=k)=∫0∞e−λλkk!βrΓ(r)λr−1e−βλ dλ=Γ(k+r)Γ(r)k!(ββ+1)r(1β+1)k.\begin{aligned} \Pr(X=k) &= \int_0^\infty \frac{e^{-\lambda}\lambda^k}{k!} \frac{\beta^r}{\Gamma(r)} \lambda^{r-1}e^{-\beta\lambda} \,d\lambda\\ &= \frac{\Gamma(k+r)}{\Gamma(r)k!} \left(\frac{\beta}{\beta+1}\right)^r \left(\frac{1}{\beta+1}\right)^k. \end{aligned}

This is the negative binomial distribution with

p=ββ+1.p=\frac{\beta}{\beta+1}.

The representation explains why a negative binomial can model heterogeneous event rates. Even if observations are Poisson conditional on their individual rates, variation among those rates increases the marginal variance:

Var⁡(X)=E⁡[Λ]+Var⁡(Λ).\operatorname{Var}(X) = \operatorname{E}[\Lambda]+\operatorname{Var}(\Lambda).

It also provides a simulation method: draw a gamma-distributed rate, then draw a Poisson count conditional on that rate.

Bayesian predictive interpretation

In Bayesian inference, the gamma distribution is a conjugate prior for a Poisson rate. Integrating a Poisson sampling model against a gamma posterior produces a negative binomial posterior predictive distribution.

For example, if the posterior rate is gamma with shape aa and rate bb, a future count observed over exposure t>0t>0 has negative binomial parameters

r=a,p=bb+t.r=a, \qquad p=\frac{b}{b+t}.

Here the negative binomial represents uncertainty about the future count after accounting for uncertainty in its underlying rate.

Related distributions and limiting behavior

Geometric special case

When r=1r=1,

Pr⁡(X=k)=p(1−p)k.\Pr(X=k)=p(1-p)^k.

This is the failures-counting form of the geometric distribution. For integer rr, a negative binomial variable can be represented as the sum of rr independent geometric variables with the same success probability.

Addition with a common probability parameter

If independent variables satisfy

X1∼NB⁡(r1,p),X2∼NB⁡(r2,p),X_1\sim\operatorname{NB}(r_1,p), \qquad X_2\sim\operatorname{NB}(r_2,p),

then

X1+X2∼NB⁡(r1+r2,p).X_1+X_2\sim\operatorname{NB}(r_1+r_2,p).

This holds for arbitrary positive real shapes. It follows by multiplying their generating functions, or through convolution of their probability masses. The common value of pp is essential: sums with different probability parameters are not generally negative binomial.

Poisson limit

Hold μ\mu fixed and let

r→∞,p=rr+μ→1.r\to\infty, \qquad p=\frac{r}{r+\mu}\to1.

Then

NB⁡(r,p) → d  Poisson⁡(μ).\operatorname{NB}(r,p) \ \xrightarrow{\ d\ }\ \operatorname{Poisson}(\mu).

Correspondingly, the variance μ+μ2/r\mu+\mu^2/r approaches μ\mu. The Poisson distribution is therefore a limiting case with vanishing extra dispersion.

Normal approximation

For fixed pp, increasing rr yields an approximately normal shape after standardization. For integer rr, this follows from the sum-of-geometric representation and the central limit theorem.

A large mean alone does not guarantee a good normal approximation: with small fixed rr, substantial right skewness can persist even as the mean increases.

Parameter estimation

For independent observations x1,…,xnx_1,\ldots,x_n, let xˉ\bar{x} be the sample mean, and define the empirical second central moment

v=1n∑i=1n(xi−xˉ)2.v=\frac1n\sum_{i=1}^n(x_i-\bar{x})^2.

Matching empirical and theoretical moments gives

p^=xˉv,r^=xˉ2v−xˉ,\widehat p=\frac{\bar{x}}{v}, \qquad \widehat r=\frac{\bar{x}^2}{v-\bar{x}},

provided v>xˉ>0v>\bar{x}>0. Substituting an unbiased sample variance is another common finite-sample convention.

If the empirical variance does not exceed the mean, these formulas do not yield a finite positive shape estimate. This does not make such a sample impossible under a negative binomial model, but it limits the usefulness of this moment-matching procedure.

For maximum likelihood estimation, the log-likelihood function is

ℓ(r,p)=∑i=1n[log⁡Γ(xi+r)−log⁡Γ(r)−log⁡(xi!)+rlog⁡p+xilog⁡(1−p)].\ell(r,p) = \sum_{i=1}^n \left[ \log\Gamma(x_i+r) -\log\Gamma(r) -\log(x_i!) +r\log p+x_i\log(1-p) \right].

For fixed rr, its maximizing value of pp is

p^(r)=rr+xˉ.\widehat p(r)=\frac{r}{r+\bar{x}}.

Estimation of rr generally requires numerical optimization. When the fitted extra dispersion approaches zero, rr tends toward infinity and the model approaches the Poisson boundary.

Negative binomial regression

Negative binomial regression relates the mean count to explanatory variables, commonly through

log⁡μi=xi⊤β,Var⁡(Yi∣xi)=μi+αμi2.\log\mu_i=\mathbf{x}_i^\top\boldsymbol{\beta}, \qquad \operatorname{Var}(Y_i\mid\mathbf{x}_i) = \mu_i+\alpha\mu_i^2.

It is widely used within the generalized linear model framework. The log link keeps fitted means positive and gives exponentiated coefficients a multiplicative interpretation.

When counts refer to different exposures, such as different observation periods, a model may include an exposure offset:

log⁡μi=log⁡ti+xi⊤β,\log\mu_i = \log t_i+\mathbf{x}_i^\top\boldsymbol{\beta},

where tit_i is the exposure and its log has coefficient fixed at one.

The quadratic variance formulation is often called NB2. Another specification, NB1, uses

Var⁡(Yi∣xi)=μi(1+α).\operatorname{Var}(Y_i\mid\mathbf{x}_i) = \mu_i(1+\alpha).

Both can use negative binomial conditional distributions, but they impose different relationships between mean and dispersion across observations. In NB2 the shape is constant when α\alpha is constant; in NB1 the shape varies with the mean.

Applications and limitations

Negative binomial models appear in ecology, insurance, traffic analysis, econometrics, and sequencing-based studies of gene expression. Their common role is to describe counts with more variability than a Poisson model permits.

Several distinctions matter when interpreting such models:

  • Overdispersion does not identify its cause. Unobserved rate heterogeneity is one explanation, but omitted predictors, dependence, or an incorrect mean model can also produce it.
  • Dispersion does not automatically model dependence. Independent negative binomial observations remain independent. Repeated measurements, clusters, or a time series may require additional dependence structure.
  • Extra zeros do not necessarily imply a separate zero-generating process. A negative binomial can already assign substantial probability to zero. Zero-inflated or hurdle models introduce additional assumptions rather than merely allowing a larger variance.
  • Underdispersion is outside the standard family. A negative binomial count with positive mean cannot have variance below its mean.
  • Sampling restrictions change the distribution. If zero counts are unobservable, the appropriate likelihood is zero-truncated rather than the ordinary negative binomial likelihood.
  • Equal means and variances do not establish distributional fit. Tail behavior, zero frequencies, and conditional patterns can differ between distributions with similar first two moments.