Double machine learning (DML) is a framework in statistics and econometrics for estimating a low-dimensional parameter while using machine learning to estimate complex auxiliary relationships. Its two central ingredients are Neyman-orthogonal estimating equations, which reduce sensitivity to errors in those auxiliary estimates, and cross-fitting, which separates their training from their evaluation. Under suitable assumptions, this combination permits approximately normal estimation and valid uncertainty quantification even when the auxiliary relationships are learned more slowly than conventional parametric rates. DML is widely used in causal inference, but it also applies to noncausal statistical parameters.
Motivation and scope
Many estimation problems involve a target parameter, such as a treatment effect or a regression coefficient, together with unknown functions that are necessary for estimation but are not themselves the main object of interest. These auxiliary objects are called nuisance functions. Examples include conditional outcome means, treatment-assignment probabilities, and conditional expectations of instruments.
Flexible prediction methods can estimate these functions without imposing a short, fixed parametric formula. However, simply inserting their predictions into a conventional estimating equation can produce substantial bias. Two difficulties are especially important:
- Regularization bias: methods using regularization, variable selection, or other complexity controls deliberately trade some fitting accuracy for predictive stability. The resulting nuisance-estimation error can propagate into the target estimate at first order.
- Overfitting dependence: if an observation helps train a nuisance model and is then evaluated using that model, its fitted value can be statistically dependent on its own noise.
Good predictive performance alone therefore does not guarantee valid inference for the target parameter. DML addresses these problems through the design of the estimating equation and the arrangement of the data, rather than through any particular prediction algorithm.
The word “double” is not a requirement to use exactly two machine-learning models. It refers to the framework’s combination of orthogonalization and sample splitting or cross-fitting; particular applications may require several nuisance functions.
General formulation
Let be observations, initially assumed independent and identically distributed. Let be the target parameter and collect the nuisance functions. Suppose the target is characterized by a population moment equation:
Here is an estimating function, often called a score in the DML literature. It need not be the derivative of a log-likelihood.
A direct plug-in method estimates , substitutes the estimate into this equation, and solves its sample counterpart. DML instead uses a score with a particular insensitivity property and evaluates it using out-of-fold nuisance predictions.
Neyman orthogonality
A score is Neyman-orthogonal at the truth if its population expectation has zero first-order derivative with respect to admissible perturbations of the nuisance functions:
for every admissible direction .
This is a statement about local insensitivity of the expected estimating equation, not ordinary geometric perpendicularity of observed data vectors. If the nuisance estimate is close to the truth, orthogonality makes its leading contribution to target-estimation bias typically second order rather than first order.
Orthogonality does not make arbitrary nuisance estimates acceptable. Their errors still must become sufficiently small, and other regularity conditions must hold.
Cross-fitting
Divide the observations into disjoint folds . For each fold :
- Estimate the nuisance functions using observations outside , obtaining .
- Evaluate the score for observations inside using those fitted functions.
A pooled DML estimator solves
Every observation contributes to the target-estimation stage, but its own nuisance prediction is generated without using that observation for training.
Cross-fitting resembles cross-validation, but their purposes differ. Cross-validation usually assesses predictive performance or selects tuning parameters; cross-fitting constructs out-of-sample predictions for an estimating equation. Prediction-model tuning can be conducted within each training sample without using the held-out observations.
Example: the partially linear regression model
A standard example is
where is an outcome, is a treatment or explanatory variable, and is a vector of controls. The unknown function may be nonlinear and high-dimensional, while is a scalar.
Define the conditional expectations
Under the model,
An orthogonal score is
After estimating and by cross-fitting, define
where identifies the fold containing observation . The pooled estimator is
This is a regression through the origin of residualized outcomes on residualized treatments. It extends the partialling-out logic of linear regression to flexibly estimated conditional means.
The denominator must contain sufficient residual treatment variation. If is almost determined by , estimation becomes unstable.
A causal interpretation requires more than this computational construction. The controls and model must support the relevant causal assumptions. With heterogeneous treatment effects, the partially linear coefficient need not equal the population average treatment effect; under suitable conditions, it instead represents a variance-weighted average of conditional effects.
Example: the average treatment effect
For a binary treatment , let and denote potential outcomes. The average treatment effect is
A standard identification argument assumes:
- Consistency: the observed outcome equals the potential outcome corresponding to the treatment received.
- Conditional exchangeability: is independent of conditional on .
- Overlap: treatment probabilities are bounded away from zero and one.
Conditional exchangeability rules out unmeasured confounding after conditioning on . DML does not establish this assumption from observational data.
Define the outcome regressions and propensity score as
An orthogonal score is
The estimator averages the expression preceding , using cross-fitted nuisance predictions. This combines outcome regression with inverse-probability weighting.
This score also has double robustness: at the population level, with appropriate integrability and positivity conditions, it identifies the average treatment effect if either the propensity function is correct or both outcome-regression functions are correct.
Double robustness and orthogonality are related but distinct. Double robustness concerns correctness under alternative nuisance-model specifications. Orthogonality concerns local sensitivity to nuisance perturbations. Not every orthogonal score has a global double-robustness property, and double robustness alone does not ensure valid conventional inference when one nuisance model is misspecified.
Convergence rates and statistical inference
DML can permit root- inference for the target parameter even when individual nuisance estimates converge more slowly than .
For the average-treatment-effect score, a representative remainder condition is
together with nuisance consistency, overlap, and suitable moment conditions. Here the norm measures root-mean-square error under the covariate distribution.
A familiar sufficient case is that both nuisance-error factors are . This is not a universal requirement: one nuisance estimate can converge more slowly if the other converges sufficiently quickly. Exact requirements depend on the score. For example, some partially linear score formulations also introduce squared nuisance-error terms.
Under the relevant conditions, an asymptotic linear representation takes the form
where is the estimator’s influence function. For a scalar target,
with
A central limit theorem then yields an asymptotic normal distribution. An estimated influence-function variance provides the standard error, from which confidence intervals and hypothesis tests can be constructed.
Orthogonality alone does not guarantee statistical efficiency. Attaining a semiparametric efficiency bound additionally requires an appropriate efficient score and the assumptions supporting its use.
Choice of prediction methods and implementation
DML is largely agnostic about the nuisance-learning algorithm. Possible choices include lasso, ridge regression, random forests, boosting, and neural networks, as well as combinations of prediction methods.
This flexibility is conditional rather than unrestricted: a learner must estimate the relevant nuisance functions accurately enough for the target problem. Low predictive error on one task does not establish all the rate, moment, or overlap conditions needed for inference.
Important implementation details include:
- Training-sample isolation: learned preprocessing, feature selection, and tuning must respect the fold structure. Full-sample preprocessing that uses outcome information can undermine the intended separation.
- Fold aggregation: pooled score solving and averaging fold-specific estimates are related DML constructions, but they need not produce identical finite-sample results.
- Repeated splitting: repeated cross-fitting can assess sensitivity to the random partition. Formal aggregation procedures can also account for between-split variability.
- Variance estimation: uncertainty calculations must correspond to the score, estimator, and sampling design, rather than automatically reuse a naive regression formula.
Applications and extensions
DML is used to estimate treatment effects and policy-relevant parameters when many observed covariates require adjustment. Applications include observational studies of programs, labor-market interventions, and other economic relationships.
The framework also extends beyond unconfounded treatment assignment. Orthogonal scores can be constructed for instrumental variables, treatment effects on treated populations, certain difference-in-differences parameters, and missing-data problems.
DML is not inherently a method for estimating an entire heterogeneous treatment-effect function. Its standard theory concerns a fixed-dimensional target. Estimating individualized effects or a growing collection of subgroup effects requires additional methods and assumptions.
Limitations
Identification remains separate from estimation
DML addresses errors arising from nuisance-function estimation; it does not repair a failure of statistical identification. Omitted confounders, invalid instruments, inappropriate conditioning variables, or misspecified causal structures can invalidate interpretation even when predictions are excellent.
Weak overlap and weak identifying variation
Extreme propensity scores produce large inverse weights and unstable estimates. Clipping estimated probabilities may improve numerical behavior but introduces additional approximation issues; trimming observations can change the target population. In partially linear and instrumental-variable settings, weak residual variation or weak instruments present analogous difficulties.
Finite-sample performance
DML’s principal guarantees are asymptotic. Small samples, rare treatments, heavy-tailed outcomes, and poor nuisance estimates can lead to biased estimates or inaccurate confidence intervals. Cross-fitting reduces the training sample available to each fitted model and does not itself ensure predictive quality.
Dependent observations
Ordinary random folds and independent-observation variance formulas are not automatically valid for clustered, longitudinal, spatial, or time-series data. Dependence-aware sample splitting and variance estimation require separate justification.
Model misspecification
Local robustness is not universal protection against misspecification. If nuisance estimates converge to unsuitable limits, the target estimate may remain biased. Double-robust consistency in a particular model does not automatically preserve the standard DML asymptotic distribution.
Historical development and related methods
DML brings together older ideas from orthogonal estimating equations, semiparametric statistics, influence-function theory, sample splitting, and residualization. Earlier work on partially linear models already demonstrated how a finite-dimensional parameter could be estimated while leaving other relationships nonparametric.
The modern framework was systematized by Victor Chernozhukov and coauthors in the 2018 paper Double/debiased machine learning for treatment and structural parameters. Its distinctive contribution was to combine orthogonal scores with cross-fitting in a general framework compatible with modern, high-complexity nuisance estimators.
DML is closely related to augmented inverse-probability-weighted estimation and targeted maximum likelihood estimation. These approaches can use related influence functions and share double-robustness or efficiency properties. They differ in how they construct the final estimator: common DML implementations solve orthogonal estimating equations, whereas targeted estimation updates an initial model toward the target parameter before evaluating a substitution estimator.