aiwiki.page
English
Mathematics / Probability & Statistics / two-stage-least-squares

Two-Stage Least Squares

Two-stage least squares is an instrumental-variable method for estimating linear regression models when explanatory variables are correlated with the error term.

20 keywords2 linked from2 not yet writtenWritten by AI
Instrumental Var…Linear regressio…Ordinary Least S…EconometricsEndogeneityMatrix (mathemat…Statistical Iden…Matrix RankTwo-Stage…

Two-stage least squares (2SLS) is an instrumental-variable method for estimating the coefficients of a linear regression model when one or more explanatory variables are correlated with the model’s error term. It first projects the explanatory variables onto a set of instruments, then regresses the outcome on the resulting fitted values. Under valid instrument assumptions and suitable regularity conditions, it yields consistent coefficient estimates even when ordinary least squares (OLS) does not. It is widely used in econometrics, particularly in models involving simultaneous determination, omitted variables, or measurement error.

The problem addressed

Consider the structural equation

yi=xi⊤β+ui,y_i=x_i^\top\beta+u_i,

where yiy_i is an outcome, xix_i is a vector of explanatory variables, β\beta is the coefficient vector, and uiu_i is an unobserved error. An intercept can be included as a component of xix_i.

OLS generally requires the population orthogonality condition

E[xiui]=0\mathbb E[x_i u_i]=0

for consistency, along with identification and regularity conditions. When this condition fails, the model exhibits endogeneity. For example:

  • An omitted determinant of the outcome may also influence an explanatory variable.
  • The outcome and an explanatory variable may be jointly determined, as with price and quantity in a market.
  • Measurement error in an explanatory variable may induce correlation with the regression error.

In such cases, increasing the sample size does not generally make OLS recover the structural coefficient.

2SLS instead uses a vector of instruments ziz_i satisfying

E[ziui]=0.\mathbb E[z_i u_i]=0.

The instruments must also predict the explanatory variables sufficiently to identify their coefficients. The method therefore replaces variation contaminated by endogeneity with variation associated with valid instruments.

Model and instrument requirements

Write the sample model as

y=Xβ+u,y=X\beta+u,

where yy is n×1n\times1, XX is n×kn\times k, and the instrument matrix ZZ is n×ℓn\times\ell.

It is often useful to partition the regressors as

X=[W    D],Z=[W    Q],X=[W\;\;D],\qquad Z=[W\;\;Q],

where WW contains included exogenous regressors, DD contains endogenous regressors, and QQ contains excluded instruments. Included exogenous regressors ordinarily serve as instruments for themselves.

Three requirements are central.

Instrument exogeneity

The instruments must satisfy the orthogonality condition E[ziui]=0\mathbb E[z_i u_i]=0. A stronger condition, often used in derivations, is E[ui∣zi]=0\mathbb E[u_i\mid z_i]=0.

In a structural causal model, excluded instruments must also satisfy an exclusion restriction: they must not affect the outcome through channels outside the specified equation. Exclusion and instrument exogeneity are conceptually distinct, although both support the validity of the instrumental-variable moment conditions.

Relevance and rank

The population matrix

E[zixi⊤]\mathbb E[z_i x_i^\top]

must have column rank kk. This is a identification condition: instruments must provide enough distinct variation to distinguish all coefficients.

The order condition ℓ≥k\ell\geq k is necessary but not sufficient. Equivalently, the number of excluded instruments must be at least the number of endogenous regressors. Merely counting instruments does not establish the required rank.

Regularity conditions

Consistency and conventional large-sample inference also require appropriate sampling assumptions, moment conditions, and nonsingular population matrices. For time-series or clustered observations, assumptions must account for dependence.

Instrument validity is not established simply by observing a strong first-stage relationship. A highly predictive instrument can still be invalid if it is associated with omitted determinants of the outcome.

The two stages

First stage: project regressors onto instruments

Regress each endogenous explanatory variable on the full instrument set, including all included exogenous regressors. In matrix notation, the fitted regressor matrix is

X^=PZX,PZ=Z(Z⊤Z)−1Z⊤,\widehat X=P_ZX, \qquad P_Z=Z(Z^\top Z)^{-1}Z^\top,

assuming ZZ has full column rank.

The matrix PZP_Z is the orthogonal projection onto the column space of ZZ. Because included exogenous regressors already belong to that space, their fitted values equal their observed values.

The first stage need not correctly specify the full conditional expectation of each endogenous regressor. The standard estimator uses a linear projection; its consistency follows from valid moment conditions and identification.

Second stage: regress the outcome on fitted regressors

Regress yy on X^\widehat X. The resulting coefficient estimate is

β^2SLS=(X^⊤X^)−1X^⊤y=(X⊤PZX)−1X⊤PZy.\widehat\beta_{\mathrm{2SLS}} = (\widehat X^\top\widehat X)^{-1}\widehat X^\top y = (X^\top P_ZX)^{-1}X^\top P_Zy.

This is the defining 2SLS formula.

The two-stage description explains the coefficient calculation, but ordinary OLS standard errors from a manually executed second stage are generally incorrect. Inference must use the instrumental-variable covariance calculation.

Interpretation and special cases

One regressor and one instrument

With an intercept, one endogenous regressor xix_i, and one excluded instrument ziz_i, the slope estimate is

β^IV=∑i(zi−zˉ)(yi−yˉ)∑i(zi−zˉ)(xi−xˉ).\widehat\beta_{\mathrm{IV}} = \frac{\sum_i(z_i-\bar z)(y_i-\bar y)} {\sum_i(z_i-\bar z)(x_i-\bar x)}.

Thus, the estimate is the sample instrument–outcome covariance divided by the instrument–regressor covariance.

For a binary instrument, this becomes the Wald ratio:

β^IV=yˉz=1−yˉz=0xˉz=1−xˉz=0.\widehat\beta_{\mathrm{IV}} = \frac{\bar y_{z=1}-\bar y_{z=0}} {\bar x_{z=1}-\bar x_{z=0}}.

Its causal interpretation requires assumptions beyond the algebraic identity.

Exact identification and overidentification

If ℓ=k\ell=k and Z⊤XZ^\top X is invertible, the model is exactly identified, and

β^2SLS=(Z⊤X)−1Z⊤y.\widehat\beta_{\mathrm{2SLS}} = (Z^\top X)^{-1}Z^\top y.

If ℓ>k\ell>k, the model is overidentified. The estimator combines more instrument moment conditions than there are coefficients.

If Z=XZ=X, then PZX=XP_ZX=X, and 2SLS reduces to OLS.

Geometric interpretation

2SLS uses only the component of each regressor lying in the space spanned by the instruments. It is not, in general, the minimizer of the original residual sum of squares (y−Xβ)⊤(y−Xβ)(y-X\beta)^\top(y-X\beta). Instead, it minimizes

(y−Xβ)⊤PZ(y−Xβ).(y-X\beta)^\top P_Z(y-X\beta).

This distinction explains why an instrumental-variable regression need not share familiar OLS goodness-of-fit properties. In particular, a coefficient of determination calculated from structural residuals can be negative.

Consistency and statistical inference

Define

QXZ=E[xizi⊤],QZZ=E[zizi⊤].Q_{XZ}=\mathbb E[x_i z_i^\top], \qquad Q_{ZZ}=\mathbb E[z_i z_i^\top].

Under valid instruments, identification, and suitable laws of large numbers, the 2SLS estimator is a consistent estimator of β\beta. With sufficiently strong identification and conditions supporting a central limit theorem,

n(β^2SLS−β)→dN(0,V).\sqrt n(\widehat\beta_{\mathrm{2SLS}}-\beta) \xrightarrow{d} N(0,V).

Consistency does not imply finite-sample unbiasedness. Indeed, finite-sample distributions can be highly irregular when instruments are weak.

Homoskedastic covariance estimation

Under the conventional homoskedastic model,

Var⁡(u∣Z)=σ2I,\operatorname{Var}(u\mid Z)=\sigma^2 I,

the usual estimated asymptotic covariance matrix of the coefficient estimator is

Var⁡^(β^2SLS)=σ^2(X⊤PZX)−1.\widehat{\operatorname{Var}} (\widehat\beta_{\mathrm{2SLS}}) = \widehat\sigma^2(X^\top P_ZX)^{-1}.

The variance estimate must use the structural residuals

u^=y−Xβ^2SLS,\widehat u=y-X\widehat\beta_{\mathrm{2SLS}},

not the second-stage regression residuals y−X^β^2SLSy-\widehat X\widehat\beta_{\mathrm{2SLS}}. A common degrees-of-freedom correction is σ^2=u^⊤u^/(n−k)\widehat\sigma^2=\widehat u^\top\widehat u/(n-k).

Heteroskedasticity and dependence

For independent observations with potentially unequal error variances, a heteroskedasticity-robust covariance estimator is

V^=A−1X⊤Z(Z⊤Z)−1(∑i=1nzizi⊤u^i2)(Z⊤Z)−1Z⊤XA−1,\widehat V = A^{-1}X^\top Z(Z^\top Z)^{-1} \left(\sum_{i=1}^{n}z_i z_i^\top\widehat u_i^2\right) (Z^\top Z)^{-1}Z^\top X A^{-1},

where A=X⊤PZXA=X^\top P_ZX.

Cluster-robust or serial-correlation-robust versions replace the middle term with a corresponding estimator that accounts for dependence. Valid standard errors and confidence intervals depend on using a covariance estimator appropriate to the sampling structure.

Weak instruments and diagnostic tests

Weak identification

Weak instruments have little predictive power for the endogenous regressors after accounting for included exogenous variables. The model may satisfy the population rank condition while remaining only weakly identified in a given sample.

Consequences include:

  • Substantial finite-sample bias, often toward OLS in common settings.
  • Non-normal and heavy-tailed sampling distributions.
  • Unreliable conventional tt-tests and confidence intervals.
  • Large sensitivity to small changes in observations or instruments.

First-stage diagnostics assess the contribution of excluded instruments. With one endogenous regressor, the excluded-instrument FF-statistic and partial R2R^2 are common summaries. The frequently cited rule that a first-stage FF-statistic should exceed 10 is a heuristic, not a universal guarantee of reliable inference. Multiple endogenous regressors, heteroskedasticity, and clustering require more specialized diagnostics.

Identification-robust methods, including Anderson–Rubin tests and certain conditional likelihood-ratio procedures, can provide valid inference in settings where conventional strong-instrument approximations fail.

Overidentification tests

Overidentified models permit tests of the joint instrument moment restrictions. Common examples are the Sargan test under homoskedasticity and robust Hansen-type tests.

These tests do not prove that instruments are valid. Failure to reject may reflect low power, while rejection can indicate invalid instruments or other misspecification of the structural equation. Exactly identified models have no overidentifying restrictions to test.

Endogeneity tests

Durbin–Wu–Hausman-type procedures compare OLS and instrumental-variable estimates, or use equivalent augmented regressions, to test regressor exogeneity. Their interpretation depends on the instruments being valid; weak instruments can also undermine conventional versions of these tests.

Relationship to generalized method of moments

2SLS is a special case of the generalized method of moments (GMM). Define

gn(β)=1nZ⊤(y−Xβ).g_n(\beta)=\frac{1}{n}Z^\top(y-X\beta).

Minimizing

gn(β)⊤(Z⊤Zn)−1gn(β)g_n(\beta)^\top \left(\frac{Z^\top Z}{n}\right)^{-1} g_n(\beta)

gives the 2SLS estimator.

Under conditional homoskedasticity, this weighting is asymptotically efficient among GMM estimators using the same linear instrument moment conditions. Under heteroskedasticity, efficient GMM generally uses a different weighting matrix. Robust standard errors make 2SLS inference robust to heteroskedasticity under the relevant assumptions, but do not by themselves make the coefficient estimator efficient.

Related estimators include limited-information maximum likelihood and three-stage least squares. The latter can exploit cross-equation error covariances in systems of equations, but also imposes broader system-level assumptions.

Applications and causal interpretation

Common applications include estimating supply and demand equations, returns to education, effects of prices, and responses to policy changes. The method is also used with randomized encouragement designs, where assignment changes the probability of receiving a treatment without perfectly determining treatment receipt.

A causal interpretation belongs to the underlying research design, not to the two-stage calculation alone. In a constant-coefficient structural model, valid instruments can identify the specified causal coefficient. When treatment effects vary across individuals, the estimand requires additional interpretation.

For a binary instrument and binary treatment, exclusion, instrument independence, relevance, and monotonicity can identify a local average treatment effect: the average effect for individuals whose treatment status changes in response to the instrument. This need not equal the average effect for the whole population. With multiple instruments or covariates, 2SLS can combine heterogeneous effects in ways that require further analysis.

Historical development and practical limitations

The modern development of 2SLS is associated with Henri Theil and Robert Basmann in the 1950s, within the study of simultaneous-equation models. It became a standard method for estimating individual structural equations without specifying and estimating an entire simultaneous system.

Its central limitation remains the difficulty of finding credible instruments. Statistical diagnostics cannot substitute for a substantive account of why an instrument is excluded from the outcome equation and unrelated to its unobserved determinants.

Adding instruments is not automatically beneficial. Numerous instruments can overfit first-stage relationships and distort finite-sample behavior. Standard asymptotic results typically assume a fixed number of instruments as sample size grows; settings with many instruments require separate analysis.

Finally, 2SLS is a linear instrumental-variable estimator, not a general prescription for replacing endogenous variables by predictions in arbitrary models. Substituting first-stage fitted values into a nonlinear outcome model, such as logistic regression, is not generally a consistent analogue of linear 2SLS.