The Frisch–Waugh–Lovell theorem is a result in linear regression stating that the coefficients on a selected group of regressors can be obtained by first removing the linear effects of the remaining regressors from both the outcome and the selected regressors, then regressing the resulting residuals on one another. This procedure, called partialling out or residualization, gives exactly the same selected coefficients and final residuals as the full ordinary least squares regression. The theorem is an algebraic identity: it does not require normally distributed errors, a correctly specified statistical model, or a causal interpretation.
Statement
Consider the regression
where:
- is an outcome vector;
- is an matrix of regressors whose coefficients are of interest;
- is an matrix of control regressors;
- and are coefficient vectors.
Assume the combined design matrix has full column rank. An intercept, if required, is included as a column in one of these blocks, usually .
Define the projection matrix onto the column space of and its residual-maker matrix by
where is the identity matrix and denotes transpose.
The theorem states that the full-regression estimate of is
Equivalently, define
Then
Thus, the coefficient vector can be computed in three steps:
- Regress on and retain the residuals .
- Regress each column of on and retain the residual columns .
- Regress on , without adding an intercept.
If contains an intercept, all residualized variables already have sample mean zero. The no-intercept formulation remains the general statement, including when contains no intercept.
The full-regression residual vector also satisfies
Consequently, the full and residualized regressions have identical residual sums of squares.
Proof by minimizing over the controls
Ordinary least squares minimizes the squared Euclidean norm
For any fixed , the minimizing value of is
Substituting this expression into the objective gives
This is precisely the residualized regression. Because is symmetric and idempotent,
its normal equations are
which yields the stated formula.
Once has been obtained, the coefficients on the controls can be recovered as
They are not generally equal to the coefficients from regressing on alone.
Geometric interpretation
In linear algebra, least squares is an orthogonal projection of the outcome vector onto the space spanned by the regressors.
The columns of span a subspace representing the variation explained by the controls. Applying removes the component of a vector lying in that subspace and leaves its component in the orthogonal complement.
The residualized regressors satisfy
Moreover, the combined regressor space has the orthogonal decomposition
The full fitted outcome therefore decomposes as
The first term is the projection onto the control space; the second is the additional fitted component contributed by after its overlap with that space has been removed.
This explains the phrase “holding other regressors constant.” The coefficient on is determined by variation in that cannot be represented as a linear combination of the controls.
One-regressor case and an example
For a single regressor ,
If the controls include an intercept, this is the sample covariance between the residualized variables divided by the sample variance of the residualized regressor, using matching normalization conventions.
As a simple example, consider
Taking to be the intercept column makes residualization equivalent to subtracting the sample mean:
The theorem then gives the familiar slope formula
With additional controls, exactly the same principle applies, but deviations from the mean are replaced by residuals from regressions on those controls.
It is generally incorrect to residualize only the outcome and then regress it on the original . Residualizing only and regressing the original outcome on does give the same coefficient, because , but its residuals differ from those of the full regression.
Statistical inference
The theorem itself is deterministic. Distributional and moment assumptions enter when interpreting the coefficients as estimators or constructing standard errors and tests.
Under the classical conditional error assumptions
the conditional covariance matrix of is
When , the usual unbiased estimate of the error variance is
The denominator is the residual degrees of freedom of the full regression. A software routine applied only to the residualized regressors may instead use , producing different conventional standard errors despite identical coefficients and residuals.
An unadjusted heteroskedasticity-robust covariance estimator, commonly called HC0, can be computed as
where is row of . This equals the corresponding coefficient block from the full regression.
However, finite-sample corrections must be matched explicitly. Corrections based on parameter counts require the full-model count, while leverage-based corrections require the appropriate full-model leverages. Coefficient equivalence alone does not guarantee that default software output will reproduce every inferential quantity.
Applications
Interpreting adjusted associations
The theorem makes the meaning of a multiple-regression coefficient explicit: it describes an association after removing the linear components explained by the other regressors.
This is important in econometrics and other applied fields, but it does not establish causal identification. Residualizing observed controls does not by itself eliminate confounding, reverse causation, measurement error, or other forms of endogeneity.
Fixed effects and demeaning
When consists of group indicators, projecting onto its column space gives group means. Residualization therefore becomes subtraction of group means:
This establishes the equivalence between a regression containing a complete set of group effects and the corresponding within-group regression, provided the indicator coding spans the same control space.
With several sets of group effects, a single sequence of mean subtractions is not generally enough. Iterative residualization methods can be used to obtain the projection onto the complement of the combined control space.
Computation and diagnostics
Partialling out can avoid explicitly estimating large numbers of nuisance coefficients. It also underlies added-variable plots, which display residualized outcomes against residualized regressors to visualize an adjusted linear relationship.
For numerical computation, explicit construction of an residual-maker matrix is usually unnecessary. QR decomposition or singular value decomposition can residualize variables directly. These approaches can offer better numerical stability than forming and inverting cross-product matrices.
Extensions and limitations
Weighted least squares
An analogous identity holds for weighted least squares. Let be a known symmetric positive-definite weight matrix and define
Then
The residualization and final regression must use the same weights. Here orthogonality is defined by the weighted inner product , rather than the ordinary Euclidean inner product.
Rank and identification
The standard formula requires that retain full column rank after residualization. If a column of lies entirely in the span of , its residualized version is zero and its coefficient cannot be separately identified.
When the controls contain redundant columns, projection onto their column space remains well defined, even though does not exist. A generalized inverse or a nonredundant representation can define the projection, but unique estimation of the target coefficients still requires independent residualized regressor directions.
Nonlinear and penalized models
The ordinary FWL identity does not generally carry over to logistic regression or other nonlinear regression models. Their objectives cannot usually be reduced through ordinary linear residualization.
Likewise, penalized estimation requires separate analysis. If only is penalized and remains unpenalized in a squared-error objective, the controls can still be eliminated by the same projection. Penalizing the controls as well generally changes that result.
The theorem also concerns linear projection, not arbitrary conditioning. Removing a linear projection on need not be the same as subtracting the conditional expectation given .
Historical development
The theorem is named after Ragnar Frisch, Frederick V. Waugh, and Michael C. Lovell. Frisch and Waugh presented a regression decomposition in their 1933 paper “Partial Time Regressions as Compared with Individual Trends,” addressing the role of time trends in regression. Lovell developed related results in his 1963 paper “Seasonal Adjustment of Economic Time Series and Multiple Regression Analysis.”
The result is now commonly presented as a general theorem about partitioned least-squares regression, rather than as a technique restricted to trends or seasonal adjustment.