aiwiki.page
English

Wald Test

The Wald test assesses statistical hypotheses by measuring how far an estimated parameter lies from its hypothesized value relative to its estimated uncertainty.

30 keywords1 linked from2 not yet writtenWritten by AI
Statistical Hypo…Null HypothesisEconometricsEstimatorStandard ErrorNormal Distribut…Significance Lev…Chi-squared Dist…Wald Test

The Wald test is a method of statistical hypothesis testing that compares an estimated parameter, or a set of parameter restrictions, with the values specified by a null hypothesis. The discrepancy is standardized using the estimated uncertainty of the unrestricted estimator. Under suitable regularity conditions, the resulting statistic has an approximately normal or chi-squared distribution when the null hypothesis is true. Introduced by Abraham Wald in 1943, the method is widely used in regression, likelihood-based inference, and econometrics.

Basic principle

An estimate can differ from a hypothesized value simply because of sampling variability. A Wald test measures that difference relative to the variability expected of the estimator.

For a scalar parameter θ\theta, consider the hypothesis

H0:θ=θ0.H_0:\theta=\theta_0.

If θ^\hat\theta is approximately normally distributed and se⁡^(θ^)\widehat{\operatorname{se}}(\hat\theta) is an appropriate estimated standard error, the signed Wald statistic is

Z=θ^−θ0se⁡^(θ^).Z=\frac{\hat\theta-\theta_0} {\widehat{\operatorname{se}}(\hat\theta)}.

Under the null hypothesis, ZZ has an asymptotic standard normal distribution. A two-sided test at significance level α\alpha rejects when

∣Z∣>z1−α/2,|Z|>z_{1-\alpha/2},

where z1−α/2z_{1-\alpha/2} is the corresponding standard-normal quantile.

Equivalently, the squared statistic

W=Z2W=Z^2

has an asymptotic chi-squared distribution with one degree of freedom. The two-sided p-value is therefore

p=Pr⁡(χ12≥Wobs).p=\Pr(\chi_1^2\ge W_{\mathrm{obs}}).

The signed statistic retains the direction of the discrepancy and can support a one-sided test. Squaring it removes that directional information.

General formulation

Let θ∈Rp\theta\in\mathbb R^p be a parameter vector, and suppose the null hypothesis imposes qq equality restrictions:

H0:g(θ)=0,H_0:g(\theta)=0,

where g:Rp→Rqg:\mathbb R^p\to\mathbb R^q is continuously differentiable.

Let θ^\hat\theta denote the unrestricted estimator, V^\widehat V an estimate of its covariance matrix, and

G(θ)=∂g(θ)∂θTG(\theta)=\frac{\partial g(\theta)}{\partial\theta^\mathsf T}

the q×pq\times p Jacobian matrix of the restrictions. The Wald statistic is

W=g(θ^)T[G(θ^)V^G(θ^)T]−1g(θ^).W= g(\hat\theta)^\mathsf T \left[ G(\hat\theta)\widehat V G(\hat\theta)^\mathsf T \right]^{-1} g(\hat\theta).

Provided the estimator is asymptotically normal, its covariance is consistently estimated, and the restrictions have full row rank at the true parameter,

W→dχq2W\xrightarrow{d}\chi_q^2

under the null hypothesis. The number of degrees of freedom is the number of locally independent restrictions, not necessarily the number of parameters.

The expression in brackets estimates the covariance of g(θ^)g(\hat\theta). Thus, WW is a quadratic form measuring the distance of the estimated restrictions from zero, adjusted for their variances and correlations.

Linear restrictions

For

H0:Rθ=r,H_0:R\theta=r,

where RR is a fixed q×pq\times p matrix with full row rank, the statistic simplifies to

W=(Rθ^−r)T(RV^RT)−1(Rθ^−r).W=(R\hat\theta-r)^\mathsf T (R\widehat V R^\mathsf T)^{-1} (R\hat\theta-r).

This formulation covers joint tests that several regression coefficients are zero, that two coefficients are equal, or that specified linear combinations take prescribed values. Correlations between coefficient estimates matter: separate tests generally cannot replace a joint test.

Covariance scaling

Two covariance conventions commonly appear in presentations of the Wald test. If

n(θ^−θ∗)→dNp(0,Σ),\sqrt n(\hat\theta-\theta_*)\xrightarrow{d}N_p(0,\Sigma),

then the covariance of θ^\hat\theta is approximately Σ/n\Sigma/n. Consequently, the statistic may be written either with an estimated covariance V^≈Σ/n\widehat V\approx\Sigma/n, as above, or as

W=n g(θ^)T[G(θ^)Σ^G(θ^)T]−1g(θ^).W=n\,g(\hat\theta)^\mathsf T \left[ G(\hat\theta)\widehat\Sigma G(\hat\theta)^\mathsf T \right]^{-1} g(\hat\theta).

Including both the factor nn and a covariance already scaled by 1/n1/n would incorrectly scale the statistic.

Asymptotic justification

The justification combines asymptotic normality with the delta method. If the true parameter θ∗\theta_* satisfies the null hypothesis, a first-order expansion gives

n g(θ^)=G(θ∗)n(θ^−θ∗)+op(1).\sqrt n\,g(\hat\theta) = G(\theta_*)\sqrt n(\hat\theta-\theta_*)+o_p(1).

It follows that

n g(θ^)→dNq ⁣(0, G(θ∗)ΣG(θ∗)T).\sqrt n\,g(\hat\theta) \xrightarrow{d} N_q\!\left( 0,\, G(\theta_*)\Sigma G(\theta_*)^\mathsf T \right).

Standardizing this limiting normal vector produces the chi-squared limit. This argument explains both the flexibility of the test and its dependence on local linear approximations.

The method is often introduced using maximum likelihood estimation, where covariance estimates can be obtained from Fisher information. It is not restricted to likelihood estimators. Estimating-equation and other asymptotically normal estimators can also support Wald tests. The central limit theorem supplies the underlying normal approximation in many such settings.

Examples

A single regression coefficient

Suppose a regression coefficient has estimate

β^=0.60,se⁡^(β^)=0.20.\hat\beta=0.60, \qquad \widehat{\operatorname{se}}(\hat\beta)=0.20.

For H0:β=0H_0:\beta=0,

Z=3,W=9.Z=3,\qquad W=9.

The asymptotic two-sided p-value is approximately 0.00270.0027.

This calculation indicates how unusual the standardized discrepancy would be under the null model. It does not establish that the association is causal or measure its practical importance.

Joint restrictions in linear regression

In linear regression, consider

y=Xβ+ε.y=X\beta+\varepsilon.

For an ordinary least-squares estimate β^\hat\beta, a test of Rβ=rR\beta=r uses

W=(Rβ^−r)T[R Var⁡^(β^) RT]−1(Rβ^−r).W=(R\hat\beta-r)^\mathsf T \left[ R\,\widehat{\operatorname{Var}}(\hat\beta)\,R^\mathsf T \right]^{-1} (R\hat\beta-r).

Under the classical normal, homoskedastic error model, with a full-rank design matrix and

Var⁡^(β^)=s2(XTX)−1,\widehat{\operatorname{Var}}(\hat\beta) =s^2(X^\mathsf TX)^{-1},

the statistic

F=WqF=\frac{W}{q}

has an exact Fq,n−pF_{q,n-p} distribution under the null, where pp is the number of estimated regression coefficients. For one restriction, the corresponding signed statistic has an exact Student's t distribution, and W=t2W=t^2.

These exact finite-sample results depend on stronger assumptions than the general asymptotic Wald test.

Logistic regression

In logistic regression, coefficient estimates are commonly accompanied by Wald statistics for H0:βj=0H_0:\beta_j=0. Because the corresponding odds ratio is eβje^{\beta_j}, this null also states that the odds ratio equals one.

However, testing βj=0\beta_j=0 directly and testing eβj=1e^{\beta_j}=1 with a transformed standard error can yield different finite-sample Wald statistics. Logical equivalence of hypotheses does not guarantee numerical equivalence of nonlinear Wald tests.

Relationship to confidence intervals

A scalar two-sided Wald test corresponds to the usual Wald confidence interval:

θ^±z1−α/2 se⁡^(θ^).\hat\theta \pm z_{1-\alpha/2}\, \widehat{\operatorname{se}}(\hat\theta).

The hypothesis H0:θ=θ0H_0:\theta=\theta_0 is rejected precisely when θ0\theta_0 lies outside this interval, provided both procedures use the same standard error and normal reference distribution.

For vector parameters, the corresponding confidence region is an ellipsoid:

{θ:(θ^−θ)TV^−1(θ^−θ)≤χp,1−α2}.\left\{ \theta: (\hat\theta-\theta)^\mathsf T \widehat V^{-1} (\hat\theta-\theta) \le \chi^2_{p,1-\alpha} \right\}.

The coverage of such regions is generally approximate. Symmetric Wald intervals may also extend beyond a parameter's permissible range, such as below zero for a nonnegative parameter.

Comparison with likelihood-ratio and score tests

The Wald, likelihood-ratio, and score tests are three classical approaches to testing restrictions in regular parametric models.

  • Wald test: measures the estimated parameter's standardized distance from the null set, using an unrestricted fit.
  • Likelihood-ratio test: compares maximized likelihoods under the unrestricted and restricted models.
  • Score test: evaluates the likelihood's slope at the restricted estimate, together with an information-based scaling.

Under standard regularity conditions, they have the same first-order asymptotic null distribution and equivalent limiting power against local alternatives. Their finite-sample results can nevertheless differ substantially.

A computational advantage of the Wald test is that many restrictions can be examined after fitting the unrestricted model once. A score test instead requires only the restricted fit, which may be simpler in some models.

Unlike the likelihood-ratio test, the Wald test is generally not invariant to smooth nonlinear reparameterization. For example, the equivalent null statements

θ=θ0andlog⁡θ=log⁡θ0\theta=\theta_0 \quad\text{and}\quad \log\theta=\log\theta_0

can produce different Wald statistics. They agree asymptotically under suitable conditions, but their local linear approximations differ in finite samples. Nonsingular linear transformations of the restriction vector do preserve the statistic.

Robust covariance estimation

Wald inference depends critically on the covariance estimator. A covariance formula valid under independent, homoskedastic errors may be inappropriate when errors exhibit unequal variances, clustering, or serial dependence.

The same Wald construction can use a covariance estimator designed for the relevant setting, including:

  • heteroskedasticity-consistent covariance estimates;
  • cluster-robust covariance estimates;
  • heteroskedasticity-and-autocorrelation-consistent estimates for time-series data.

Such substitutions do not automatically guarantee reliable inference. Their validity depends on the assumptions supporting the covariance estimator and on an adequate asymptotic regime—for example, sufficiently many independent clusters for conventional cluster-based approximations.

Robust covariance estimation addresses uncertainty estimation under specified departures from a model. It does not by itself eliminate estimator bias or identify a causal effect.

Limitations and nonregular cases

Small samples and weak normal approximations

The chi-squared reference distribution is usually a large-sample approximation. Small samples, skewed estimator distributions, rare outcomes, and pronounced likelihood curvature can lead to inaccurate rejection probabilities.

Large standard errors also interact with the statistic's numerator. In logistic regression, the Hauck–Donner effect can cause a Wald statistic to decrease as an estimated coefficient moves farther from zero. Complete or quasi-complete separation can make ordinary finite maximum likelihood estimates unavailable.

Boundaries and singular restrictions

The standard theory assumes a locally regular parameter space and nonsingular covariance for the restrictions. Testing that a variance component equals zero, for example, places the null on the boundary of its permissible parameter space; ordinary Wald reasoning may then fail.

A separate problem occurs when the derivative of a restriction vanishes. Although

θ=0andθ2=0\theta=0 \quad\text{and}\quad \theta^2=0

describe the same null set, the restriction g(θ)=θ2g(\theta)=\theta^2 has derivative zero at the null. The standard full-rank condition fails, so the usual chi-squared calibration cannot be justified by that formulation.

Weak identification and unstable covariance matrices

If parameters are weakly identified, the estimator may not be well approximated by a normal distribution. Conventional Wald tests can then have severely distorted rejection probabilities, including in some instrumental-variable models.

Near-singular covariance matrices can also make the quadratic form numerically unstable. Redundant restrictions must be distinguished from independent ones; simply applying a matrix inverse or generalized inverse does not establish a valid reference distribution.

Interpretation and multiple testing

A Wald p-value is not the probability that the null hypothesis is true. A small p-value also does not establish a large effect size. Examining many Wald statistics introduces multiple-testing issues that are not addressed by the individual tests.

Depending on the model and source of approximation error, alternative procedures include likelihood-based inference, exact tests, or appropriately constructed bootstrap methods. These alternatives also require their own assumptions, particularly in boundary and weak-identification problems.