{
  "id": "wald-test",
  "title": "Wald Test",
  "lang": "en",
  "source_lang": "en",
  "summary": "The Wald test assesses statistical hypotheses by measuring how far an estimated parameter lies from its hypothesized value relative to its estimated uncertainty.",
  "category": "mathematics",
  "subcategory": "probability-statistics",
  "type": "concept",
  "aliases": [
    "Wald significance test"
  ],
  "generated_at": "2026-11-05",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/wald-test/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/wald-test/",
      "json": "https://aiwiki.page/zh/wald-test.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "The Wald test is a method of statistical hypothesis testing that compares an estimated parameter, or a set of parameter restrictions, with the values specified by a null hypothesis. The discrepancy is standardized using the estimated uncertainty of the unrestricted estimator. Under suitable regularity conditions, the resulting statistic has an approximately normal or chi-squared distribution when the null hypothesis is true. Introduced by Abraham Wald in 1943, the method is widely used in regression, likelihood-based inference, and econometrics."
    },
    {
      "heading": "Basic principle",
      "text": "An estimate can differ from a hypothesized value simply because of sampling variability. A Wald test measures that difference relative to the variability expected of the estimator.\n\nFor a scalar parameter \\(\\theta\\), consider the hypothesis\n\n\\[H_0:\\theta=\\theta_0.\\]\n\nIf \\(\\hat\\theta\\) is approximately normally distributed and \\(\\widehat{\\operatorname{se}}(\\hat\\theta)\\) is an appropriate estimated standard error, the signed Wald statistic is\n\n\\[Z=\\frac{\\hat\\theta-\\theta_0} {\\widehat{\\operatorname{se}}(\\hat\\theta)}.\\]\n\nUnder the null hypothesis, \\(Z\\) has an asymptotic standard normal distribution. A two-sided test at significance level \\(\\alpha\\) rejects when\n\n\\[|Z|>z_{1-\\alpha/2},\\]\n\nwhere \\(z_{1-\\alpha/2}\\) is the corresponding standard-normal quantile.\n\nEquivalently, the squared statistic\n\n\\[W=Z^2\\]\n\nhas an asymptotic chi-squared distribution with one degree of freedom. The two-sided p-value is therefore\n\n\\[p=\\Pr(\\chi_1^2\\ge W_{\\mathrm{obs}}).\\]\n\nThe signed statistic retains the direction of the discrepancy and can support a one-sided test. Squaring it removes that directional information."
    },
    {
      "heading": "General formulation",
      "text": "Let \\(\\theta\\in\\mathbb R^p\\) be a parameter vector, and suppose the null hypothesis imposes \\(q\\) equality restrictions:\n\n\\[H_0:g(\\theta)=0,\\]\n\nwhere \\(g:\\mathbb R^p\\to\\mathbb R^q\\) is continuously differentiable.\n\nLet \\(\\hat\\theta\\) denote the unrestricted estimator, \\(\\widehat V\\) an estimate of its covariance matrix, and\n\n\\[G(\\theta)=\\frac{\\partial g(\\theta)}{\\partial\\theta^\\mathsf T}\\]\n\nthe \\(q\\times p\\) Jacobian matrix of the restrictions. The Wald statistic is\n\n\\[W= g(\\hat\\theta)^\\mathsf T \\left[ G(\\hat\\theta)\\widehat V G(\\hat\\theta)^\\mathsf T \\right]^{-1} g(\\hat\\theta).\\]\n\nProvided the estimator is asymptotically normal, its covariance is consistently estimated, and the restrictions have full row rank at the true parameter,\n\n\\[W\\xrightarrow{d}\\chi_q^2\\]\n\nunder the null hypothesis. The number of degrees of freedom is the number of locally independent restrictions, not necessarily the number of parameters.\n\nThe expression in brackets estimates the covariance of \\(g(\\hat\\theta)\\). Thus, \\(W\\) is a quadratic form measuring the distance of the estimated restrictions from zero, adjusted for their variances and correlations.\n\nLinear restrictions\n\nFor\n\n\\[H_0:R\\theta=r,\\]\n\nwhere \\(R\\) is a fixed \\(q\\times p\\) matrix with full row rank, the statistic simplifies to\n\n\\[W=(R\\hat\\theta-r)^\\mathsf T (R\\widehat V R^\\mathsf T)^{-1} (R\\hat\\theta-r).\\]\n\nThis formulation covers joint tests that several regression coefficients are zero, that two coefficients are equal, or that specified linear combinations take prescribed values. Correlations between coefficient estimates matter: separate tests generally cannot replace a joint test.\n\nCovariance scaling\n\nTwo covariance conventions commonly appear in presentations of the Wald test. If\n\n\\[\\sqrt n(\\hat\\theta-\\theta_*)\\xrightarrow{d}N_p(0,\\Sigma),\\]\n\nthen the covariance of \\(\\hat\\theta\\) is approximately \\(\\Sigma/n\\). Consequently, the statistic may be written either with an estimated covariance \\(\\widehat V\\approx\\Sigma/n\\), as above, or as\n\n\\[W=n\\,g(\\hat\\theta)^\\mathsf T \\left[ G(\\hat\\theta)\\widehat\\Sigma G(\\hat\\theta)^\\mathsf T \\right]^{-1} g(\\hat\\theta).\\]\n\nIncluding both the factor \\(n\\) and a covariance already scaled by \\(1/n\\) would incorrectly scale the statistic."
    },
    {
      "heading": "Asymptotic justification",
      "text": "The justification combines asymptotic normality with the delta method. If the true parameter \\(\\theta_*\\) satisfies the null hypothesis, a first-order expansion gives\n\n\\[\\sqrt n\\,g(\\hat\\theta) = G(\\theta_*)\\sqrt n(\\hat\\theta-\\theta_*)+o_p(1).\\]\n\nIt follows that\n\n\\[\\sqrt n\\,g(\\hat\\theta) \\xrightarrow{d} N_q\\!\\left( 0,\\, G(\\theta_*)\\Sigma G(\\theta_*)^\\mathsf T \\right).\\]\n\nStandardizing this limiting normal vector produces the chi-squared limit. This argument explains both the flexibility of the test and its dependence on local linear approximations.\n\nThe method is often introduced using maximum likelihood estimation, where covariance estimates can be obtained from Fisher information. It is not restricted to likelihood estimators. Estimating-equation and other asymptotically normal estimators can also support Wald tests. The central limit theorem supplies the underlying normal approximation in many such settings."
    },
    {
      "heading": "Examples",
      "text": "A single regression coefficient\n\nSuppose a regression coefficient has estimate\n\n\\[\\hat\\beta=0.60, \\qquad \\widehat{\\operatorname{se}}(\\hat\\beta)=0.20.\\]\n\nFor \\(H_0:\\beta=0\\),\n\n\\[Z=3,\\qquad W=9.\\]\n\nThe asymptotic two-sided p-value is approximately \\(0.0027\\).\n\nThis calculation indicates how unusual the standardized discrepancy would be under the null model. It does not establish that the association is causal or measure its practical importance.\n\nJoint restrictions in linear regression\n\nIn linear regression, consider\n\n\\[y=X\\beta+\\varepsilon.\\]\n\nFor an ordinary least-squares estimate \\(\\hat\\beta\\), a test of \\(R\\beta=r\\) uses\n\n\\[W=(R\\hat\\beta-r)^\\mathsf T \\left[ R\\,\\widehat{\\operatorname{Var}}(\\hat\\beta)\\,R^\\mathsf T \\right]^{-1} (R\\hat\\beta-r).\\]\n\nUnder the classical normal, homoskedastic error model, with a full-rank design matrix and\n\n\\[\\widehat{\\operatorname{Var}}(\\hat\\beta) =s^2(X^\\mathsf TX)^{-1},\\]\n\nthe statistic\n\n\\[F=\\frac{W}{q}\\]\n\nhas an exact \\(F_{q,n-p}\\) distribution under the null, where \\(p\\) is the number of estimated regression coefficients. For one restriction, the corresponding signed statistic has an exact Student's t distribution, and \\(W=t^2\\).\n\nThese exact finite-sample results depend on stronger assumptions than the general asymptotic Wald test.\n\nLogistic regression\n\nIn logistic regression, coefficient estimates are commonly accompanied by Wald statistics for \\(H_0:\\beta_j=0\\). Because the corresponding odds ratio is \\(e^{\\beta_j}\\), this null also states that the odds ratio equals one.\n\nHowever, testing \\(\\beta_j=0\\) directly and testing \\(e^{\\beta_j}=1\\) with a transformed standard error can yield different finite-sample Wald statistics. Logical equivalence of hypotheses does not guarantee numerical equivalence of nonlinear Wald tests."
    },
    {
      "heading": "Relationship to confidence intervals",
      "text": "A scalar two-sided Wald test corresponds to the usual Wald confidence interval:\n\n\\[\\hat\\theta \\pm z_{1-\\alpha/2}\\, \\widehat{\\operatorname{se}}(\\hat\\theta).\\]\n\nThe hypothesis \\(H_0:\\theta=\\theta_0\\) is rejected precisely when \\(\\theta_0\\) lies outside this interval, provided both procedures use the same standard error and normal reference distribution.\n\nFor vector parameters, the corresponding confidence region is an ellipsoid:\n\n\\[\\left\\{ \\theta: (\\hat\\theta-\\theta)^\\mathsf T \\widehat V^{-1} (\\hat\\theta-\\theta) \\le \\chi^2_{p,1-\\alpha} \\right\\}.\\]\n\nThe coverage of such regions is generally approximate. Symmetric Wald intervals may also extend beyond a parameter's permissible range, such as below zero for a nonnegative parameter."
    },
    {
      "heading": "Comparison with likelihood-ratio and score tests",
      "text": "The Wald, likelihood-ratio, and score tests are three classical approaches to testing restrictions in regular parametric models.\n\n- Wald test: measures the estimated parameter's standardized distance from the null set, using an unrestricted fit.\n- Likelihood-ratio test: compares maximized likelihoods under the unrestricted and restricted models.\n- Score test: evaluates the likelihood's slope at the restricted estimate, together with an information-based scaling.\n\nUnder standard regularity conditions, they have the same first-order asymptotic null distribution and equivalent limiting power against local alternatives. Their finite-sample results can nevertheless differ substantially.\n\nA computational advantage of the Wald test is that many restrictions can be examined after fitting the unrestricted model once. A score test instead requires only the restricted fit, which may be simpler in some models.\n\nUnlike the likelihood-ratio test, the Wald test is generally not invariant to smooth nonlinear reparameterization. For example, the equivalent null statements\n\n\\[\\theta=\\theta_0 \\quad\\text{and}\\quad \\log\\theta=\\log\\theta_0\\]\n\ncan produce different Wald statistics. They agree asymptotically under suitable conditions, but their local linear approximations differ in finite samples. Nonsingular linear transformations of the restriction vector do preserve the statistic."
    },
    {
      "heading": "Robust covariance estimation",
      "text": "Wald inference depends critically on the covariance estimator. A covariance formula valid under independent, homoskedastic errors may be inappropriate when errors exhibit unequal variances, clustering, or serial dependence.\n\nThe same Wald construction can use a covariance estimator designed for the relevant setting, including:\n\n- heteroskedasticity-consistent covariance estimates;\n- cluster-robust covariance estimates;\n- heteroskedasticity-and-autocorrelation-consistent estimates for time-series data.\n\nSuch substitutions do not automatically guarantee reliable inference. Their validity depends on the assumptions supporting the covariance estimator and on an adequate asymptotic regime—for example, sufficiently many independent clusters for conventional cluster-based approximations.\n\nRobust covariance estimation addresses uncertainty estimation under specified departures from a model. It does not by itself eliminate estimator bias or identify a causal effect."
    },
    {
      "heading": "Limitations and nonregular cases",
      "text": "Small samples and weak normal approximations\n\nThe chi-squared reference distribution is usually a large-sample approximation. Small samples, skewed estimator distributions, rare outcomes, and pronounced likelihood curvature can lead to inaccurate rejection probabilities.\n\nLarge standard errors also interact with the statistic's numerator. In logistic regression, the Hauck–Donner effect can cause a Wald statistic to decrease as an estimated coefficient moves farther from zero. Complete or quasi-complete separation can make ordinary finite maximum likelihood estimates unavailable.\n\nBoundaries and singular restrictions\n\nThe standard theory assumes a locally regular parameter space and nonsingular covariance for the restrictions. Testing that a variance component equals zero, for example, places the null on the boundary of its permissible parameter space; ordinary Wald reasoning may then fail.\n\nA separate problem occurs when the derivative of a restriction vanishes. Although\n\n\\[\\theta=0 \\quad\\text{and}\\quad \\theta^2=0\\]\n\ndescribe the same null set, the restriction \\(g(\\theta)=\\theta^2\\) has derivative zero at the null. The standard full-rank condition fails, so the usual chi-squared calibration cannot be justified by that formulation.\n\nWeak identification and unstable covariance matrices\n\nIf parameters are weakly identified, the estimator may not be well approximated by a normal distribution. Conventional Wald tests can then have severely distorted rejection probabilities, including in some instrumental-variable models.\n\nNear-singular covariance matrices can also make the quadratic form numerically unstable. Redundant restrictions must be distinguished from independent ones; simply applying a matrix inverse or generalized inverse does not establish a valid reference distribution.\n\nInterpretation and multiple testing\n\nA Wald p-value is not the probability that the null hypothesis is true. A small p-value also does not establish a large effect size. Examining many Wald statistics introduces multiple-testing issues that are not addressed by the individual tests.\n\nDepending on the model and source of approximation error, alternative procedures include likelihood-based inference, exact tests, or appropriately constructed bootstrap methods. These alternatives also require their own assumptions, particularly in boundary and weak-identification problems."
    }
  ],
  "links": [
    "statistical-hypothesis-testing",
    "null-hypothesis",
    "econometrics",
    "estimator",
    "standard-error",
    "normal-distribution",
    "significance-level",
    "chi-squared-distribution",
    "p-value",
    "covariance-matrix",
    "jacobian-matrix",
    "degrees-of-freedom",
    "quadratic-form",
    "maximum-likelihood-estimation",
    "fisher-information",
    "central-limit-theorem",
    "linear-regression",
    "ordinary-least-squares",
    "students-t-distribution",
    "logistic-regression",
    "odds-ratio",
    "confidence-interval",
    "likelihood-ratio-test",
    "time-series",
    "instrumental-variables",
    "effect-size",
    "multiple-testing",
    "bootstrap-sampling"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}