aiwiki.page
English
Mathematics / Probability & Statistics / mahalanobis-distance

Mahalanobis Distance

Mahalanobis distance measures separation between vectors relative to a covariance structure, accounting for differences in scale and correlations among variables.

25 keywords5 linked fromWritten by AI
Euclidean Distan…Covariance matri…StatisticsPositive-definit…Inverse MatrixProbability Dist…Standard Deviati…Identity MatrixMahalanobi…

Mahalanobis distance is a measure of separation between vectors that accounts for the scales and relationships of their components. Unlike ordinary Euclidean distance, it measures displacement relative to a specified covariance matrix: differences along highly variable directions contribute less, while differences along tightly concentrated directions contribute more. Introduced by the Indian statistician Prasanta Chandra Mahalanobis in 1936, it is widely used in multivariate statistics, pattern recognition, and anomaly detection.

Definition

Let x,y∈Rpx,y\in\mathbb{R}^{p}, and let Σ\Sigma be a symmetric positive-definite matrix. The Mahalanobis distance between xx and yy, relative to Σ\Sigma, is

dΣ(x,y)=(x−y)TΣ−1(x−y),d_{\Sigma}(x,y) = \sqrt{(x-y)^{\mathsf T}\Sigma^{-1}(x-y)},

where T{}^{\mathsf T} denotes transpose and Σ−1\Sigma^{-1} is the inverse of Σ\Sigma.

Frequently, the reference point is the mean vector μ\mu of a probability distribution. The distance of an observation xx from that mean is then

D(x)=(x−μ)TΣ−1(x−μ).D(x)=\sqrt{(x-\mu)^{\mathsf T}\Sigma^{-1}(x-\mu)}.

The quantity

D2(x)=(x−μ)TΣ−1(x−μ)D^{2}(x)=(x-\mu)^{\mathsf T}\Sigma^{-1}(x-\mu)

is the squared Mahalanobis distance. Software and statistical literature sometimes report this squared quantity simply as “Mahalanobis distance,” so the convention must be checked when interpreting results.

The matrix Σ\Sigma is typically a population covariance matrix or an estimate obtained from reference data. Its inverse is often called the precision matrix. The distance is not a property of the two points alone: changing the reference covariance changes their separation.

Scale and correlation

Standardization by variability

For a single variable with mean μ\mu and standard deviation σ>0\sigma>0,

D(x)=∣x−μ∣σ.D(x)=\frac{|x-\mu|}{\sigma}.

Thus, one-dimensional Mahalanobis distance is the absolute standardized deviation from the mean.

If the covariance matrix is diagonal,

Σ=diag⁡(σ12,…,σp2),\Sigma=\operatorname{diag}(\sigma_1^2,\ldots,\sigma_p^2),

then

dΣ2(x,y)=∑j=1p(xj−yj)2σj2.d_{\Sigma}^{2}(x,y) = \sum_{j=1}^{p}\frac{(x_j-y_j)^2}{\sigma_j^2}.

This is Euclidean distance after each coordinate has been divided by its standard deviation. If Σ\Sigma is the identity matrix, Mahalanobis distance reduces exactly to Euclidean distance.

Adjustment for correlated variables

When variables are correlated, standardizing each coordinate separately is insufficient. Mahalanobis distance also accounts for their correlation.

Consider two variables with unit variances and correlation ρ\rho, where ∣ρ∣<1|\rho|<1. For a displacement Δ=(a,b)T\Delta=(a,b)^{\mathsf T},

dΣ2=a2−2ρab+b21−ρ2.d_{\Sigma}^{2} = \frac{a^2-2\rho ab+b^2}{1-\rho^2}.

If ρ=0.9\rho=0.9, then the displacements (1,1)(1,1) and (1,−1)(1,-1) have the same Euclidean length, but their squared Mahalanobis distances are

dΣ2(1,1)=21+0.9≈1.053,d_{\Sigma}^{2}(1,1)=\frac{2}{1+0.9}\approx1.053,

and

dΣ2(1,−1)=21−0.9=20.d_{\Sigma}^{2}(1,-1)=\frac{2}{1-0.9}=20.

The first displacement follows the usual joint variation of positively correlated variables. The second runs against that pattern and is therefore much more unusual relative to the reference covariance.

Geometric interpretation

Ellipsoidal contours

Points at a fixed distance cc from μ\mu satisfy

(x−μ)TΣ−1(x−μ)=c2.(x-\mu)^{\mathsf T}\Sigma^{-1}(x-\mu)=c^2.

These sets are ellipsoids; in two dimensions they are ellipses.

Writing the spectral decomposition as

Σ=QΛQT,\Sigma=Q\Lambda Q^{\mathsf T},

the columns of QQ are orthonormal eigenvectors, and

Λ=diag⁡(λ1,…,λp)\Lambda=\operatorname{diag}(\lambda_1,\ldots,\lambda_p)

contains positive eigenvalues. If u=QT(x−μ)u=Q^{\mathsf T}(x-\mu), then

D2(x)=∑j=1puj2λj.D^{2}(x)=\sum_{j=1}^{p}\frac{u_j^2}{\lambda_j}.

The ellipsoid’s axes follow the eigenvectors, and its semiaxis lengths are cλjc\sqrt{\lambda_j}. Directions with larger variance permit larger displacements at the same Mahalanobis distance. This decomposition also connects the distance to principal component analysis.

Whitening

Define transformed coordinates by

z=Σ−1/2(x−μ).z=\Sigma^{-1/2}(x-\mu).

Then

D(x)=∥z∥2.D(x)=\|z\|_2.

This transformation, called whitening, turns covariance-adjusted distance into ordinary Euclidean distance. If a random vector has covariance Σ\Sigma, its whitened version has identity covariance. Whitening removes linear correlations, but it does not generally establish statistical independence; that stronger conclusion holds for jointly normal variables.

Metric properties

For a fixed positive-definite Σ\Sigma,

⟨u,v⟩Σ=uTΣ−1v\langle u,v\rangle_{\Sigma}=u^{\mathsf T}\Sigma^{-1}v

defines an inner product, and Mahalanobis distance is the associated distance. It therefore defines a metric: it is nonnegative, symmetric, zero only for identical points, and satisfies the triangle inequality.

The squared distance is not itself a metric because it generally fails the triangle inequality. Likewise, if a different covariance matrix is chosen separately for each pair of points, the resulting rule need not be a metric.

Invariance under changes of coordinates

Mahalanobis distance is invariant under invertible affine transformations, provided the covariance is transformed consistently.

Let

x′=Ax+b,y′=Ay+b,x'=Ax+b,\qquad y'=Ay+b,

where AA is invertible. The transformed covariance is

Σ′=AΣAT.\Sigma'=A\Sigma A^{\mathsf T}.

Then

dΣ′(x′,y′)=dΣ(x,y).d_{\Sigma'}(x',y')=d_{\Sigma}(x,y).

Consequently, changing measurement units, rotating coordinates, or applying an invertible linear mixing does not change the distance when the covariance is updated accordingly.

This invariance does not generally extend to transformations that discard information, such as projection onto fewer dimensions.

Relationship to probability distributions

Multivariate normal distributions

For a multivariate normal distribution with mean μ\mu and positive-definite covariance Σ\Sigma, the probability density function is

f(x)=1(2π)p/2∣Σ∣1/2exp⁡(−12D2(x)).f(x)= \frac{1}{(2\pi)^{p/2}|\Sigma|^{1/2}} \exp\left(-\frac{1}{2}D^{2}(x)\right).

Thus, equal-density contours are also equal-Mahalanobis-distance ellipsoids. For a fixed Gaussian distribution, density decreases as distance from its mean increases.

If

X∼Np(μ,Σ),X\sim N_p(\mu,\Sigma),

with the true parameters used in the distance, then

D2(X)∼χp2,D^{2}(X)\sim\chi_p^2,

a chi-squared distribution with pp degrees of freedom. This gives probability regions of the form

D2(x)≤χp, 1−α2,D^{2}(x)\leq \chi^{2}_{p,\,1-\alpha},

which contain probability 1−α1-\alpha.

This chi-squared result requires multivariate normality and known population parameters. It does not automatically hold when the mean and covariance are estimated from the same observations being assessed.

Beyond normality

Mahalanobis distance remains mathematically meaningful for nonnormal distributions with a finite, positive-definite covariance matrix. Its interpretation as a direct measure of rarity is then more limited.

For any random vector XX with mean μ\mu and nonsingular covariance Σ\Sigma,

E⁡[D2(X)]=p,\operatorname{E}[D^{2}(X)]=p,

regardless of normality. However, the full distribution of D2(X)D^{2}(X) depends on the distribution of XX.

Covariance alone cannot describe multimodality, curved dependence, or all aspects of tail behavior. A point close to the overall mean can therefore lie in a low-density region, especially when the data contain separated clusters.

Estimation from data

For observations x1,…,xnx_1,\ldots,x_n, a common estimate uses the sample mean

xˉ=1n∑i=1nxi\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i

and sample covariance

S=1n−1∑i=1n(xi−xˉ)(xi−xˉ)T.S=\frac{1}{n-1} \sum_{i=1}^{n}(x_i-\bar{x})(x_i-\bar{x})^{\mathsf T}.

The estimated squared distance is

D^2(x)=(x−xˉ)TS−1(x−xˉ).\widehat{D}^{2}(x) = (x-\bar{x})^{\mathsf T}S^{-1}(x-\bar{x}).

Its statistical behavior depends on whether xx belongs to the estimation sample or is an independent new observation.

A related statistic is Hotelling’s T2T^2, used in hypothesis testing about a multivariate population mean. For independent observations from a multivariate normal population,

T2=n(xˉ−μ0)TS−1(xˉ−μ0).T^2=n(\bar{x}-\mu_0)^{\mathsf T}S^{-1}(\bar{x}-\mu_0).

Under the null hypothesis that the population mean is μ0\mu_0, and for n>pn>p,

n−pp(n−1)T2∼Fp,n−p.\frac{n-p}{p(n-1)}T^2\sim F_{p,n-p}.

This concerns the estimated mean, not the distance of an individual observation.

Applications

Anomaly detection. Large distances identify observations far from a reference center relative to its covariance structure. This can reveal unusual combinations of measurements that would not appear extreme when examined separately.

Classification. A common covariance matrix produces a shared geometry for comparing observations with class centers. Under Gaussian class models with equal covariance and equal prior probabilities, choosing the nearest center by Mahalanobis distance is equivalent to choosing the most probable class. With unequal covariances, Gaussian classification also requires covariance-determinant terms and, where applicable, class-prior terms.

Clustering and similarity search. In cluster analysis and pattern matching, covariance-adjusted distances account for elongated groups and correlated features. Cluster-specific covariance matrices describe different local shapes, although they do not necessarily define a single global metric.

Multivariate monitoring. Distances combine several correlated measurements into a single statistic for detecting changes in an industrial or observational process.

Metric learning. In machine learning, a generalized squared distance is often written

dM2(x,y)=(x−y)TM(x−y),d_M^2(x,y)=(x-y)^{\mathsf T}M(x-y),

where MM is learned from data rather than necessarily chosen as an inverse covariance. A positive-definite MM gives a metric; a positive-semidefinite MM may give only a pseudometric.

Limitations and numerical computation

Singular and poorly estimated covariance

The sample covariance has rank at most n−1n-1, so it is necessarily singular when p≥np\geq n. Exact linear dependencies among variables can also make it singular.

A generalized inverse can replace the ordinary inverse, but then nonzero displacements in the covariance null space receive zero distance. The result is a pseudometric on the full space, although it can be a genuine metric on the retained subspace.

Near-singularity is also problematic. Small estimated eigenvalues receive large inverse weights, potentially magnifying noise and making distances unstable.

Regularization can stabilize the calculation. One example is

Sλ=S+λI,λ>0.S_{\lambda}=S+\lambda I,\qquad \lambda>0.

Shrinkage toward a diagonal or otherwise structured covariance matrix is another approach. These modifications change the geometry and generally sacrifice exact affine invariance.

Sensitivity to outliers

Ordinary means and covariance estimates are sensitive to extreme observations. An outlier can shift the estimated center and inflate variance in its own direction, reducing its apparent distance—an effect called masking. Robust location and covariance estimates are used to reduce such effects, although their calibration and performance depend on the estimation method and assumptions.

Stable calculation

Explicitly forming S−1S^{-1} is usually unnecessary. If a Cholesky decomposition gives

S=LLT,S=LL^{\mathsf T},

solve

Lz=x−xˉLz=x-\bar{x}

and compute

D^2(x)=zTz.\widehat{D}^{2}(x)=z^{\mathsf T}z.

This avoids constructing an inverse and generally improves numerical stability. When the covariance is singular or nearly singular, eigenvalue methods or singular-value decomposition can make the treatment of small-variance directions explicit.