aiwiki.page
English
Mathematics / Probability & Statistics / chi-squared-distribution

Chi-squared Distribution

The chi-squared distribution describes sums of squared independent standard normal variables and underlies many statistical tests and methods for estimating variance.

33 keywords36 linked from3 not yet writtenWritten by AI
Probability Dist…Degrees of Freed…StatisticsStatistical Inde…Normal Distribut…Probability Dens…Gamma FunctionCumulative Distr…Chi-square…

The chi-squared distribution is a continuous probability distribution that, for a positive integer kk, describes the sum of the squares of kk independent standard normal random variables. The parameter kk is called the number of degrees of freedom. Written χk2\chi_k^2, the distribution is fundamental to statistics, particularly tests involving categorical data, inference about population variances, and large-sample approximations to the distributions of test statistics. Its density also defines a distribution for every positive real value of kk, extending it beyond the integer-valued construction.

Definition and density

Let Z1,…,ZkZ_1,\ldots,Z_k be independent random variables, each having the standard normal distribution, with mean zero and variance one. Then

X=∑i=1kZi2X=\sum_{i=1}^{k}Z_i^2

has a chi-squared distribution with kk degrees of freedom:

X∼χk2.X\sim\chi_k^2.

The independence assumption is essential: a sum of squared standard normal variables that are correlated need not have this distribution.

For any real k>0k>0, the probability density function is

f(x;k)=xk/2−1e−x/22k/2Γ(k/2),x>0,f(x;k)= \frac{x^{k/2-1}e^{-x/2}} {2^{k/2}\Gamma(k/2)}, \qquad x>0,

and is zero for x<0x<0. Here Γ\Gamma is the gamma function. The value assigned to a density at the single point x=0x=0 does not affect probabilities. There is no point mass at zero.

The cumulative distribution function is

F(x;k)=γ(k/2,x/2)Γ(k/2),x≥0,F(x;k)= \frac{\gamma(k/2,x/2)}{\Gamma(k/2)}, \qquad x\geq 0,

where the lower incomplete gamma function is defined by

γ(a,z)=∫0zta−1e−t dt.\gamma(a,z)=\int_0^z t^{a-1}e^{-t}\,dt.

Consequently, the chi-squared distribution is a special case of the gamma distribution, with shape k/2k/2 and scale 22. Under the alternative shape–rate convention, its rate is 1/21/2.

Shape and numerical characteristics

A chi-squared variable cannot be negative. For small degrees of freedom, its distribution is strongly right-skewed; as the degrees of freedom increase, it becomes more nearly symmetric.

Its expected value and variance are

E⁡[X]=k,Var⁡(X)=2k.\operatorname{E}[X]=k, \qquad \operatorname{Var}(X)=2k.

Its standard deviation is therefore 2k\sqrt{2k}. Its skewness and excess kurtosis are

8kand12k,\sqrt{\frac{8}{k}} \quad\text{and}\quad \frac{12}{k},

respectively.

For k>2k>2, the density has its maximum at x=k−2x=k-2. At k=2k=2, the density decreases from its largest boundary value at zero. For 0<k<20<k<2, it is unbounded as xx approaches zero from the right, although its integral remains finite.

The moment-generating function is

MX(t)=(1−2t)−k/2,t<12.M_X(t)=(1-2t)^{-k/2}, \qquad t<\frac12.

More generally,

E⁡[Xr]=2rΓ(k/2+r)Γ(k/2),r>−k2.\operatorname{E}[X^r] = 2^r\frac{\Gamma(k/2+r)}{\Gamma(k/2)}, \qquad r>-\frac{k}{2}.

Two useful special cases are:

  • χ12\chi_1^2: the distribution of the square of one standard normal variable.
  • χ22\chi_2^2: an exponential distribution with mean 22.

Additivity and geometric interpretation

Sums of independent chi-squared variables

If X1,…,XmX_1,\ldots,X_m are independent and

Xj∼χkj2,X_j\sim\chi_{k_j}^2,

then

∑j=1mXj∼χ∑j=1mkj2.\sum_{j=1}^{m}X_j \sim \chi_{\sum_{j=1}^{m}k_j}^2.

This holds for arbitrary positive real degrees of freedom, not only integers. It follows from the addition rule for independent gamma variables with a common scale.

Squared distances and projections

For an integer kk, a vector of kk independent standard normal coordinates has a multivariate normal distribution. Its squared Euclidean distance from the origin has distribution χk2\chi_k^2.

More generally, let Z\mathbf Z be a standard normal vector in nn dimensions, and let PP be a symmetric idempotent matrix:

PT=P,P2=P.P^\mathsf T=P, \qquad P^2=P.

Such a matrix represents an orthogonal projection. If its rank is r>0r>0, then the quadratic form

ZTPZ∼χr2.\mathbf Z^\mathsf T P\mathbf Z \sim\chi_r^2.

The degrees of freedom count the number of independent normal directions retained by the projection. This explains why constraints and fitted parameters often reduce degrees of freedom in statistical applications.

A general quadratic form in normal variables need not be chi-squared. Unequal weights along independent normal directions ordinarily produce a weighted sum of chi-squared variables instead.

Inference about a normal population variance

Suppose X1,…,XnX_1,\ldots,X_n are independent observations from a normal population with mean μ\mu and variance σ2\sigma^2. Define the sample mean and sample variance by

X‾=1n∑i=1nXi,S2=1n−1∑i=1n(Xi−X‾)2.\overline X=\frac1n\sum_{i=1}^{n}X_i, \qquad S^2=\frac1{n-1}\sum_{i=1}^{n}(X_i-\overline X)^2.

Then

(n−1)S2σ2∼χn−12.\frac{(n-1)S^2}{\sigma^2} \sim\chi_{n-1}^2.

Moreover, X‾\overline X and S2S^2 are independent under the normal model.

There are n−1n-1, rather than nn, degrees of freedom because the residuals satisfy the constraint

∑i=1n(Xi−X‾)=0.\sum_{i=1}^{n}(X_i-\overline X)=0.

If the population mean is known and used instead of the sample mean, then

∑i=1n(Xi−μ)2σ2∼χn2.\frac{\sum_{i=1}^{n}(X_i-\mu)^2}{\sigma^2} \sim\chi_n^2.

These results provide exact tests of hypotheses about a normal population variance and exact confidence intervals. If qpq_p denotes the pp-quantile of χn−12\chi_{n-1}^2, a two-sided confidence interval with confidence level 1−α1-\alpha is

[(n−1)S2q1−α/2,(n−1)S2qα/2].\left[ \frac{(n-1)S^2}{q_{1-\alpha/2}}, \frac{(n-1)S^2}{q_{\alpha/2}} \right].

“Exact” here means that the coverage probability is exactly 1−α1-\alpha under the stated model. For nonnormal populations, the scaled sample variance generally does not have this chi-squared distribution.

Chi-squared tests for categorical data

The distribution is widely used in statistical hypothesis testing through Pearson’s chi-squared tests. A common statistic is

X2=∑i=1m(Oi−Ei)2Ei,X^2=\sum_{i=1}^{m}\frac{(O_i-E_i)^2}{E_i},

where OiO_i is an observed count and Ei>0E_i>0 is the corresponding expected count under the null hypothesis.

The statistic measures discrepancies relative to the scale of expected count variation. For categorical data, its chi-squared distribution is usually a large-sample approximation, not an exact finite-sample identity.

Goodness of fit

For nn independent observations classified into mm mutually exclusive categories with specified probabilities p1,…,pmp_1,\ldots,p_m,

Ei=npi.E_i=np_i.

Under suitable conditions,

X2→dχm−12.X^2\xrightarrow{d}\chi_{m-1}^2.

One degree of freedom is lost because the observed counts sum to the fixed total nn. If pp identifiable model parameters are estimated from the same counts by an appropriate fitting procedure, the usual limiting degrees of freedom become

m−1−p.m-1-p.

This subtraction requires regularity conditions and is not valid for every estimation procedure or model.

Independence in contingency tables

For an r×cr\times c contingency table, the expected count under row–column independence is estimated by

Eij=Oi+O+jn,E_{ij} = \frac{O_{i+}O_{+j}}{n},

where Oi+O_{i+} and O+jO_{+j} are the row and column totals. Under standard large-sample conditions, Pearson’s statistic has an approximate chi-squared distribution with

(r−1)(c−1)(r-1)(c-1)

degrees of freedom.

A test’s p-value is usually obtained from the upper tail:

p=Pr⁡ ⁣(χν2≥xobs2).p=\Pr\!\left(\chi_\nu^2\geq x_{\mathrm{obs}}^2\right).

Approximation limits

The approximation can be poor when expected counts are small or spread thinly across many cells. Rules such as requiring expected counts of at least five are practical heuristics, not universal mathematical conditions. Accuracy also depends on the number of categories and the pattern of probabilities.

Dependent observations, survey weights, clustering, and structural zeros can change the reference distribution or degrees of freedom. Exact conditional calculations or simulation-based methods can provide alternatives for appropriate sampling models.

Regression and likelihood-based inference

Normal linear models

In a linear regression model

Y=Aβ+ε,ε∼Nn(0,σ2I),\mathbf Y=A\boldsymbol\beta+\boldsymbol\varepsilon, \qquad \boldsymbol\varepsilon\sim N_n(\mathbf0,\sigma^2I),

suppose the design matrix AA has rank p<np<n. Under ordinary least squares, the residual sum of squares satisfies

RSS⁡σ2∼χn−p2.\frac{\operatorname{RSS}}{\sigma^2} \sim\chi_{n-p}^2.

This is an exact result under the stated normal-error model. It follows because the residual vector is the projection of the error vector onto a subspace of dimension n−pn-p.

Chi-squared decompositions also underlie analysis of variance, where independent sums of squares are compared after accounting for their degrees of freedom.

Likelihood-ratio statistics

For nested statistical models, consider

W=2[ℓ(θ^full)−ℓ(θ^restricted)],W= 2\left[ \ell(\widehat{\theta}_{\mathrm{full}}) - \ell(\widehat{\theta}_{\mathrm{restricted}}) \right],

where ℓ\ell is the log-likelihood and the fitted values are obtained by maximum likelihood estimation. Wilks’s theorem states that, under suitable regularity conditions and the null hypothesis, WW converges in distribution to a chi-squared variable whose degrees of freedom equal the difference in model dimensions.

This is an asymptotic result. Parameters on boundaries, nonidentifiable parameters, and some mixture models violate the usual assumptions; their limiting distributions need not be ordinary chi-squared distributions.

Related distributions

Student’s t and F distributions

If Z∼N(0,1)Z\sim N(0,1) and U∼χν2U\sim\chi_\nu^2 are independent, then

T=ZU/νT=\frac{Z}{\sqrt{U/\nu}}

has Student’s t distribution with ν\nu degrees of freedom.

If U∼χν12U\sim\chi_{\nu_1}^2 and V∼χν22V\sim\chi_{\nu_2}^2 are independent, then

F=U/ν1V/ν2F=\frac{U/\nu_1}{V/\nu_2}

has an F distribution with ν1\nu_1 and ν2\nu_2 degrees of freedom. These constructions connect normal sample means and variances to familiar regression and analysis-of-variance tests.

Noncentral chi-squared distribution

If independent normal variables have unit variance but means μi\mu_i that are not necessarily zero, then

∑i=1kZi2∼χk2(λ),λ=∑i=1kμi2.\sum_{i=1}^{k}Z_i^2 \sim\chi_k^2(\lambda), \qquad \lambda=\sum_{i=1}^{k}\mu_i^2.

This is the noncentral chi-squared distribution, with noncentrality parameter λ\lambda. The central chi-squared distribution is the case λ=0\lambda=0.

Its mean and variance are

k+λand2(k+2λ),k+\lambda \quad\text{and}\quad 2(k+2\lambda),

respectively. Noncentral chi-squared distributions arise under alternatives to null hypotheses and are used to calculate statistical power.

Approximations and computation

For large kk, the central limit theorem gives

X−k2k→dN(0,1).\frac{X-k}{\sqrt{2k}} \xrightarrow{d}N(0,1).

A frequently useful refinement is the Wilson–Hilferty cube-root approximation:

(X/k)1/3−(1−29k)2/(9k)≈N(0,1).\frac{ (X/k)^{1/3}-\left(1-\frac{2}{9k}\right) }{ \sqrt{2/(9k)} } \approx N(0,1).

It accounts for some of the distribution’s skewness, but remains an approximation, especially in extreme tails.

Statistical software typically evaluates probabilities through incomplete gamma functions and obtains quantiles by numerical inversion. For very small upper-tail probabilities, directly evaluating the upper-tail function avoids the loss of precision that can occur when subtracting a cumulative probability close to one from one.

Historical development

The chi-squared distribution arose from nineteenth-century investigations of normal errors and sums of squared deviations. Its statistical prominence increased with Karl Pearson’s 1900 work on goodness-of-fit testing.

Later work, notably by Ronald Fisher, clarified the role of degrees of freedom and parameter estimation in interpreting chi-squared statistics. These developments connected the distribution to categorical-data analysis, normal-theory inference, and the broader theory of statistical tests.