aiwiki.page
English
Mathematics / Probability & Statistics / laplace-distribution

Laplace Distribution

The Laplace distribution is a symmetric continuous probability distribution with an exponential decay on either side of its central peak, used in statistical modeling, robust estimation, sparse inference, and differential privacy.

29 keywords2 linked fromWritten by AI
Probability Dist…Normal Distribut…Pierre-Simon Lap…Random VariableProbability Dens…Cumulative Distr…QuantileConfidence Inter…Laplace Di…

The Laplace distribution is a continuous probability distribution on the real line whose density decreases exponentially with distance from a central location. It is symmetric, has a sharp peak at its center, and has heavier tails than the normal distribution. Named after Pierre-Simon Laplace, it is also called the double exponential distribution because its two halves have exponential form. Its connection with absolute deviations makes it important in statistical estimation, regression, sparse modeling, and privacy-preserving computation.

Definition and parameterization

A random variable XX has a Laplace distribution with location parameter μ∈R\mu\in\mathbb{R} and scale parameter b>0b>0, written

X∼Laplace⁡(μ,b),X\sim\operatorname{Laplace}(\mu,b),

if its probability density function is

f(x∣μ,b)=12bexp⁡ ⁣(−∣x−μ∣b),x∈R.f(x\mid\mu,b) =\frac{1}{2b}\exp\!\left(-\frac{|x-\mu|}{b}\right), \qquad x\in\mathbb{R}.

The parameter μ\mu determines the center, while bb controls the spread. The standard Laplace distribution has μ=0\mu=0 and b=1b=1. If ZZ has the standard distribution, then

X=μ+bZX=\mu+bZ

has distribution Laplace⁡(μ,b)\operatorname{Laplace}(\mu,b).

Some sources use the rate parameter λ=1/b\lambda=1/b, giving

f(x∣μ,λ)=λ2e−λ∣x−μ∣.f(x\mid\mu,\lambda) =\frac{\lambda}{2}e^{-\lambda|x-\mu|}.

Distinguishing scale from rate is essential when comparing formulas or software implementations.

The density is continuous everywhere but is not differentiable at x=μx=\mu. Its logarithm consists of two straight-line segments meeting at the center.

Distribution function and quantiles

The cumulative distribution function is

F(x)={12exp⁡ ⁣(x−μb),x<μ,1−12exp⁡ ⁣(−x−μb),x≥μ.F(x)= \begin{cases} \dfrac12\exp\!\left(\dfrac{x-\mu}{b}\right),&x<\mu,\\[6pt] 1-\dfrac12\exp\!\left(-\dfrac{x-\mu}{b}\right),&x\geq\mu. \end{cases}

Its quantile function, for 0<p<10<p<1, is

F−1(p)={μ+bln⁡(2p),0<p≤12,μ−bln⁡ ⁣(2(1−p)),12<p<1.F^{-1}(p)= \begin{cases} \mu+b\ln(2p),&0<p\leq \dfrac12,\\[6pt] \mu-b\ln\!\bigl(2(1-p)\bigr),&\dfrac12<p<1. \end{cases}

For any a≥0a\geq0,

Pr⁡(∣X−μ∣>a)=e−a/b.\Pr(|X-\mu|>a)=e^{-a/b}.

Consequently, a central interval containing probability 1−α1-\alpha, where 0<α<10<\alpha<1, is

[μ−bln⁡(1/α), μ+bln⁡(1/α)].\left[\mu-b\ln(1/\alpha),\ \mu+b\ln(1/\alpha)\right].

This is a probability interval for a random observation under known parameters, not automatically a confidence interval for an unknown parameter.

Moments and shape

The expected value, median, and mode all equal μ\mu. The variance and standard deviation are

Var⁡(X)=2b2,SD⁡(X)=2 b.\operatorname{Var}(X)=2b^2, \qquad \operatorname{SD}(X)=\sqrt{2}\,b.

The scale also has a direct interpretation as the mean absolute deviation from the center:

E∣X−μ∣=b.\mathbb{E}|X-\mu|=b.

More generally,

E∣X−μ∣r=brΓ(r+1),r>−1,\mathbb{E}|X-\mu|^r=b^r\Gamma(r+1), \qquad r>-1,

where Γ\Gamma is the gamma function. All odd central moments vanish by symmetry, while

E(X−μ)2k=(2k)! b2k.\mathbb{E}(X-\mu)^{2k}=(2k)!\,b^{2k}.

The distribution has skewness zero, kurtosis 66, and excess kurtosis 33.

Compared with a normal distribution having the same variance, a Laplace distribution has both a higher central peak and more probability far into the tails. Nevertheless, it is not heavy-tailed in the technical sense that excludes distributions with a finite moment-generating function near zero: its tails decay exponentially rather than as a power law.

The differential entropy, using natural logarithms, is

h(X)=1+ln⁡(2b).h(X)=1+\ln(2b).

Among real-valued distributions with a fixed value of E∣X−μ∣=b\mathbb{E}|X-\mu|=b, the Laplace distribution maximizes differential entropy.

Transforms and probabilistic representations

The moment-generating function is

MX(t)=E[etX]=eμt1−b2t2,∣t∣<1/b.M_X(t)=\mathbb{E}[e^{tX}] =\frac{e^{\mu t}}{1-b^2t^2}, \qquad |t|<1/b.

The characteristic function is

φX(t)=E[eitX]=eiμt1+b2t2,t∈R.\varphi_X(t)=\mathbb{E}[e^{itX}] =\frac{e^{i\mu t}}{1+b^2t^2}, \qquad t\in\mathbb{R}.

Several constructions explain the distribution’s structure.

Difference of exponential variables

If E1E_1 and E2E_2 are independent exponential random variables, each with mean bb, then

μ+E1−E2∼Laplace⁡(μ,b).\mu+E_1-E_2\sim\operatorname{Laplace}(\mu,b).

Equivalently, if EE is exponential with mean bb, and SS independently takes the values −1-1 and 11 with equal probability, then

μ+SE∼Laplace⁡(μ,b).\mu+SE\sim\operatorname{Laplace}(\mu,b).

Thus ∣X−μ∣|X-\mu| is exponentially distributed.

Normal variance mixture

Let Z∼N(0,1)Z\sim N(0,1), and independently let VV be exponential with mean 2b22b^2. Then

X=μ+V ZX=\mu+\sqrt{V}\,Z

has a Laplace distribution. Conditional on VV, the distribution is normal with variance VV. This representation is useful for hierarchical models and computational methods in Bayesian inference.

Sums and limiting behavior

A sum of independent Laplace variables is generally not itself Laplace. For example, the characteristic function of the sum of nn independent Laplace⁡(0,b)\operatorname{Laplace}(0,b) variables is

(1+b2t2)−n,(1+b^2t^2)^{-n},

rather than the characteristic function of another Laplace variable.

Because its variance is finite, the usual central limit theorem applies to independent, identically distributed Laplace observations: appropriately standardized sums converge in distribution to a normal distribution.

Parameter estimation

For independent observations x1,…,xnx_1,\ldots,x_n, the log-likelihood is

ℓ(μ,b)=−nln⁡(2b)−1b∑i=1n∣xi−μ∣.\ell(\mu,b) =-n\ln(2b)-\frac{1}{b}\sum_{i=1}^{n}|x_i-\mu|.

Maximum likelihood

For fixed bb, maximum likelihood estimation of μ\mu minimizes the sum of absolute deviations:

μ^∈arg min⁡m∈R∑i=1n∣xi−m∣.\widehat{\mu}\in \operatorname*{arg\,min}_{m\in\mathbb{R}} \sum_{i=1}^{n}|x_i-m|.

Therefore, any sample median is a maximum-likelihood estimate. For an even sample size, every point between the two middle ordered observations is a minimizer.

Given such an estimate,

b^=1n∑i=1n∣xi−μ^∣.\widehat b =\frac1n\sum_{i=1}^{n}|x_i-\widehat\mu|.

When the observations are not all identical, these give joint maximum-likelihood estimates. If all observations coincide, the likelihood is unbounded as μ\mu is placed at their common value and bb tends to zero, so no finite maximum exists within the parameter space b>0b>0.

Efficiency and robustness

Under the Laplace model, the sample median has asymptotic variance

Var⁡(μ^)∼b2n,\operatorname{Var}(\widehat\mu)\sim\frac{b^2}{n},

whereas the sample mean has variance 2b2/n2b^2/n. Thus the median is asymptotically more efficient for estimating the location under this model.

The median is also resistant to extreme observations. This robustness does not extend automatically to every Laplace-based procedure: the scale estimate remains sensitive to sufficiently large observations, and regression methods based on absolute residuals can still be sensitive to unusual predictor values.

Applications

Absolute-error regression

Suppose a linear regression model has errors that, conditional on the predictors, are independent Laplace variables with common scale:

yi=xiTβ+εi,εi∼Laplace⁡(0,b).y_i=x_i^{\mathsf T}\beta+\varepsilon_i, \qquad \varepsilon_i\sim\operatorname{Laplace}(0,b).

Maximizing the likelihood over β\beta is equivalent to minimizing

∑i∣yi−xiTβ∣.\sum_i|y_i-x_i^{\mathsf T}\beta|.

This is least absolute deviations regression. Unlike ordinary least squares, it uses an absolute-error loss function rather than a squared-error loss. Large residuals therefore contribute linearly rather than quadratically to the objective.

Sparse Bayesian modeling

Independent Laplace prior distributions on regression coefficients have joint density proportional to

exp⁡ ⁣(−λ∑j∣βj∣).\exp\!\left(-\lambda\sum_j|\beta_j|\right).

With a Gaussian observation model, maximum a posteriori estimation then minimizes a squared-error term plus an L1L^1 penalty, producing the optimization criterion of lasso regression.

The resulting mode can have coefficients exactly equal to zero. A continuous Laplace prior, however, does not itself assign positive probability to a coefficient being exactly zero; sparsity of a posterior mode differs from a discrete posterior probability of exclusion.

Differential privacy

The Laplace mechanism is a basic construction in differential privacy. For a real-valued query qq, define its sensitivity by

Δ1q=sup⁡D∼D′∣q(D)−q(D′)∣,\Delta_1q =\sup_{D\sim D'}|q(D)-q(D')|,

where D∼D′D\sim D' denotes neighboring datasets under a specified adjacency relation.

When the sensitivity is finite and positive, releasing

q(D)+Z,Z∼Laplace⁡ ⁣(0,Δ1qε),q(D)+Z, \qquad Z\sim\operatorname{Laplace}\!\left(0,\frac{\Delta_1q}{\varepsilon}\right),

provides ε\varepsilon-differential privacy. The guarantee follows from a bound on the ratio of output densities for neighboring datasets. A zero-sensitivity query requires no noise.

Simulation and related distributions

Laplace samples can be generated directly by applying the quantile function to a variable with a uniform distribution on (0,1)(0,1). They can also be generated using the difference-of-exponentials or random-sign representations. Implementations must handle endpoints carefully because the quantile function diverges at probabilities zero and one.

The asymmetric Laplace distribution allows different exponential decay rates on the two sides of its peak and is used in quantile regression. Multivariate extensions also exist, but different constructions produce different dependence structures; there is no single universally intended multivariate Laplace model.

The term “double exponential” can also describe an exponential-of-an-exponential expression in other mathematical contexts. In probability, the distribution’s density or parameterization should therefore be specified when ambiguity is possible.

As a statistical model, the symmetric Laplace distribution imposes equal left and right tail behavior and exponential tail decay. It may be unsuitable for strongly skewed data or power-law tails, while its cusp at the center distinguishes it from smooth-peaked alternatives. It is also distinct from the Laplace transform and from Laplace approximations used to approximate integrals, despite their shared name.