{
  "id": "frisch-waugh-lovell-theorem",
  "title": "Frisch–Waugh–Lovell theorem",
  "lang": "en",
  "source_lang": "en",
  "summary": "The Frisch–Waugh–Lovell theorem shows that regression coefficients can be obtained by first removing the linear effects of other regressors from both the outcome and the regressors of interest.",
  "category": "mathematics",
  "subcategory": "probability-statistics",
  "type": "concept",
  "aliases": [
    "FWL theorem",
    "partialling-out theorem",
    "regression decomposition theorem"
  ],
  "generated_at": "2026-11-06",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/frisch-waugh-lovell-theorem/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/frisch-waugh-lovell-theorem/",
      "json": "https://aiwiki.page/zh/frisch-waugh-lovell-theorem.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "The Frisch–Waugh–Lovell theorem is a result in linear regression stating that the coefficients on a selected group of regressors can be obtained by first removing the linear effects of the remaining regressors from both the outcome and the selected regressors, then regressing the resulting residuals on one another. This procedure, called partialling out or residualization, gives exactly the same selected coefficients and final residuals as the full ordinary least squares regression. The theorem is an algebraic identity: it does not require normally distributed errors, a correctly specified statistical model, or a causal interpretation."
    },
    {
      "heading": "Statement",
      "text": "Consider the regression\n\n\\[y=X\\beta+Z\\gamma+\\varepsilon,\\]\n\nwhere:\n\n- \\(y\\) is an \\(n\\times1\\) outcome vector;\n- \\(X\\) is an \\(n\\times k\\) matrix of regressors whose coefficients are of interest;\n- \\(Z\\) is an \\(n\\times m\\) matrix of control regressors;\n- \\(\\beta\\) and \\(\\gamma\\) are coefficient vectors.\n\nAssume the combined design matrix \\([X\\;Z]\\) has full column rank. An intercept, if required, is included as a column in one of these blocks, usually \\(Z\\).\n\nDefine the projection matrix onto the column space of \\(Z\\) and its residual-maker matrix by\n\n\\[P_Z=Z(Z^\\top Z)^{-1}Z^\\top, \\qquad M_Z=I_n-P_Z,\\]\n\nwhere \\(I_n\\) is the identity matrix and \\({}^\\top\\) denotes transpose.\n\nThe theorem states that the full-regression estimate of \\(\\beta\\) is\n\n\\[\\boxed{ \\hat\\beta=(X^\\top M_ZX)^{-1}X^\\top M_Zy. }\\]\n\nEquivalently, define\n\n\\[\\widetilde y=M_Zy, \\qquad \\widetilde X=M_ZX.\\]\n\nThen\n\n\\[\\hat\\beta =(\\widetilde X^\\top\\widetilde X)^{-1} \\widetilde X^\\top\\widetilde y.\\]\n\nThus, the coefficient vector can be computed in three steps:\n\n- Regress \\(y\\) on \\(Z\\) and retain the residuals \\(\\widetilde y\\).\n- Regress each column of \\(X\\) on \\(Z\\) and retain the residual columns \\(\\widetilde X\\).\n- Regress \\(\\widetilde y\\) on \\(\\widetilde X\\), without adding an intercept.\n\nIf \\(Z\\) contains an intercept, all residualized variables already have sample mean zero. The no-intercept formulation remains the general statement, including when \\(Z\\) contains no intercept.\n\nThe full-regression residual vector also satisfies\n\n\\[\\hat\\varepsilon =y-X\\hat\\beta-Z\\hat\\gamma =\\widetilde y-\\widetilde X\\hat\\beta.\\]\n\nConsequently, the full and residualized regressions have identical residual sums of squares."
    },
    {
      "heading": "Proof by minimizing over the controls",
      "text": "Ordinary least squares minimizes the squared Euclidean norm\n\n\\[\\min_{\\beta,\\gamma} \\|y-X\\beta-Z\\gamma\\|^2.\\]\n\nFor any fixed \\(\\beta\\), the minimizing value of \\(\\gamma\\) is\n\n\\[\\hat\\gamma(\\beta) =(Z^\\top Z)^{-1}Z^\\top(y-X\\beta).\\]\n\nSubstituting this expression into the objective gives\n\n\\[\\min_\\beta \\|M_Z(y-X\\beta)\\|^2 = \\min_\\beta \\|\\widetilde y-\\widetilde X\\beta\\|^2.\\]\n\nThis is precisely the residualized regression. Because \\(M_Z\\) is symmetric and idempotent,\n\n\\[M_Z^\\top M_Z=M_Z,\\]\n\nits normal equations are\n\n\\[X^\\top M_ZX\\hat\\beta=X^\\top M_Zy,\\]\n\nwhich yields the stated formula.\n\nOnce \\(\\hat\\beta\\) has been obtained, the coefficients on the controls can be recovered as\n\n\\[\\hat\\gamma=(Z^\\top Z)^{-1}Z^\\top(y-X\\hat\\beta).\\]\n\nThey are not generally equal to the coefficients from regressing \\(y\\) on \\(Z\\) alone."
    },
    {
      "heading": "Geometric interpretation",
      "text": "In linear algebra, least squares is an orthogonal projection of the outcome vector onto the space spanned by the regressors.\n\nThe columns of \\(Z\\) span a subspace representing the variation explained by the controls. Applying \\(M_Z\\) removes the component of a vector lying in that subspace and leaves its component in the orthogonal complement.\n\nThe residualized regressors satisfy\n\n\\[Z^\\top\\widetilde X=0.\\]\n\nMoreover, the combined regressor space has the orthogonal decomposition\n\n\\[\\operatorname{col}([Z\\;X]) = \\operatorname{col}(Z) \\oplus \\operatorname{col}(M_ZX).\\]\n\nThe full fitted outcome therefore decomposes as\n\n\\[\\hat y=P_Zy+\\widetilde X\\hat\\beta.\\]\n\nThe first term is the projection onto the control space; the second is the additional fitted component contributed by \\(X\\) after its overlap with that space has been removed.\n\nThis explains the phrase “holding other regressors constant.” The coefficient on \\(X\\) is determined by variation in \\(X\\) that cannot be represented as a linear combination of the controls."
    },
    {
      "heading": "One-regressor case and an example",
      "text": "For a single regressor \\(x\\),\n\n\\[\\hat\\beta = \\frac{\\widetilde x^\\top\\widetilde y} {\\widetilde x^\\top\\widetilde x}.\\]\n\nIf the controls include an intercept, this is the sample covariance between the residualized variables divided by the sample variance of the residualized regressor, using matching normalization conventions.\n\nAs a simple example, consider\n\n\\[y_i=\\alpha+\\beta x_i+\\varepsilon_i.\\]\n\nTaking \\(Z\\) to be the intercept column makes residualization equivalent to subtracting the sample mean:\n\n\\[\\widetilde y_i=y_i-\\bar y, \\qquad \\widetilde x_i=x_i-\\bar x.\\]\n\nThe theorem then gives the familiar slope formula\n\n\\[\\hat\\beta= \\frac{\\sum_{i=1}^n(x_i-\\bar x)(y_i-\\bar y)} {\\sum_{i=1}^n(x_i-\\bar x)^2}.\\]\n\nWith additional controls, exactly the same principle applies, but deviations from the mean are replaced by residuals from regressions on those controls.\n\nIt is generally incorrect to residualize only the outcome and then regress it on the original \\(X\\). Residualizing only \\(X\\) and regressing the original outcome on \\(\\widetilde X\\) does give the same coefficient, because \\(\\widetilde X^\\top Z=0\\), but its residuals differ from those of the full regression."
    },
    {
      "heading": "Statistical inference",
      "text": "The theorem itself is deterministic. Distributional and moment assumptions enter when interpreting the coefficients as estimators or constructing standard errors and tests.\n\nUnder the classical conditional error assumptions\n\n\\[\\operatorname{E}[\\varepsilon\\mid X,Z]=0, \\qquad \\operatorname{Var}(\\varepsilon\\mid X,Z)=\\sigma^2I_n,\\]\n\nthe conditional covariance matrix of \\(\\hat\\beta\\) is\n\n\\[\\operatorname{Var}(\\hat\\beta\\mid X,Z) =\\sigma^2(X^\\top M_ZX)^{-1}.\\]\n\nWhen \\(n>k+m\\), the usual unbiased estimate of the error variance is\n\n\\[\\hat\\sigma^2 =\\frac{\\hat\\varepsilon^\\top\\hat\\varepsilon}{n-k-m}.\\]\n\nThe denominator is the residual degrees of freedom of the full regression. A software routine applied only to the \\(k\\) residualized regressors may instead use \\(n-k\\), producing different conventional standard errors despite identical coefficients and residuals.\n\nAn unadjusted heteroskedasticity-robust covariance estimator, commonly called HC0, can be computed as\n\n\\[\\widehat{\\operatorname{Var}}_{\\mathrm{HC0}}(\\hat\\beta) = (\\widetilde X^\\top\\widetilde X)^{-1} \\left( \\sum_{i=1}^n \\widetilde x_i\\widetilde x_i^\\top\\hat\\varepsilon_i^2 \\right) (\\widetilde X^\\top\\widetilde X)^{-1},\\]\n\nwhere \\(\\widetilde x_i^\\top\\) is row \\(i\\) of \\(\\widetilde X\\). This equals the corresponding coefficient block from the full regression.\n\nHowever, finite-sample corrections must be matched explicitly. Corrections based on parameter counts require the full-model count, while leverage-based corrections require the appropriate full-model leverages. Coefficient equivalence alone does not guarantee that default software output will reproduce every inferential quantity."
    },
    {
      "heading": "Applications",
      "text": "Interpreting adjusted associations\n\nThe theorem makes the meaning of a multiple-regression coefficient explicit: it describes an association after removing the linear components explained by the other regressors.\n\nThis is important in econometrics and other applied fields, but it does not establish causal identification. Residualizing observed controls does not by itself eliminate confounding, reverse causation, measurement error, or other forms of endogeneity.\n\nFixed effects and demeaning\n\nWhen \\(Z\\) consists of group indicators, projecting onto its column space gives group means. Residualization therefore becomes subtraction of group means:\n\n\\[\\widetilde y_{ig}=y_{ig}-\\bar y_g, \\qquad \\widetilde x_{ig}=x_{ig}-\\bar x_g.\\]\n\nThis establishes the equivalence between a regression containing a complete set of group effects and the corresponding within-group regression, provided the indicator coding spans the same control space.\n\nWith several sets of group effects, a single sequence of mean subtractions is not generally enough. Iterative residualization methods can be used to obtain the projection onto the complement of the combined control space.\n\nComputation and diagnostics\n\nPartialling out can avoid explicitly estimating large numbers of nuisance coefficients. It also underlies added-variable plots, which display residualized outcomes against residualized regressors to visualize an adjusted linear relationship.\n\nFor numerical computation, explicit construction of an \\(n\\times n\\) residual-maker matrix is usually unnecessary. QR decomposition or singular value decomposition can residualize variables directly. These approaches can offer better numerical stability than forming and inverting cross-product matrices."
    },
    {
      "heading": "Extensions and limitations",
      "text": "Weighted least squares\n\nAn analogous identity holds for weighted least squares. Let \\(W\\) be a known symmetric positive-definite weight matrix and define\n\n\\[M_{Z,W} = I_n-Z(Z^\\top WZ)^{-1}Z^\\top W.\\]\n\nThen\n\n\\[\\hat\\beta_W = (X^\\top WM_{Z,W}X)^{-1} X^\\top WM_{Z,W}y.\\]\n\nThe residualization and final regression must use the same weights. Here orthogonality is defined by the weighted inner product \\(a^\\top Wb\\), rather than the ordinary Euclidean inner product.\n\nRank and identification\n\nThe standard formula requires that \\(X\\) retain full column rank after residualization. If a column of \\(X\\) lies entirely in the span of \\(Z\\), its residualized version is zero and its coefficient cannot be separately identified.\n\nWhen the controls contain redundant columns, projection onto their column space remains well defined, even though \\((Z^\\top Z)^{-1}\\) does not exist. A generalized inverse or a nonredundant representation can define the projection, but unique estimation of the target coefficients still requires independent residualized regressor directions.\n\nNonlinear and penalized models\n\nThe ordinary FWL identity does not generally carry over to logistic regression or other nonlinear regression models. Their objectives cannot usually be reduced through ordinary linear residualization.\n\nLikewise, penalized estimation requires separate analysis. If only \\(\\beta\\) is penalized and \\(\\gamma\\) remains unpenalized in a squared-error objective, the controls can still be eliminated by the same projection. Penalizing the controls as well generally changes that result.\n\nThe theorem also concerns linear projection, not arbitrary conditioning. Removing a linear projection on \\(Z\\) need not be the same as subtracting the conditional expectation given \\(Z\\)."
    },
    {
      "heading": "Historical development",
      "text": "The theorem is named after Ragnar Frisch, Frederick V. Waugh, and Michael C. Lovell. Frisch and Waugh presented a regression decomposition in their 1933 paper “Partial Time Regressions as Compared with Individual Trends,” addressing the role of time trends in regression. Lovell developed related results in his 1963 paper “Seasonal Adjustment of Economic Time Series and Multiple Regression Analysis.”\n\nThe result is now commonly presented as a general theorem about partitioned least-squares regression, rather than as a technique restricted to trends or seasonal adjustment."
    }
  ],
  "links": [
    "linear-regression",
    "ordinary-least-squares",
    "matrix",
    "matrix-rank",
    "identity-matrix",
    "matrix-transpose",
    "norm-mathematics",
    "idempotence",
    "linear-algebra",
    "orthogonal-projection",
    "linear-subspace",
    "orthogonal-complement",
    "covariance",
    "variance",
    "sample-mean",
    "estimator",
    "standard-error",
    "covariance-matrix",
    "degrees-of-freedom",
    "econometrics",
    "causal-inference",
    "confounding",
    "endogeneity",
    "qr-decomposition",
    "singular-value-decomposition",
    "numerical-stability",
    "inner-product",
    "logistic-regression",
    "regularization",
    "conditional-expectation"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}