Partial correlation is a measure in statistics of the association between two variables after the linear effects of a specified set of other variables have been removed from both. It is ordinarily defined as the Pearson correlation between the residuals from two linear regressions on the control variables. Its value lies between and , provided both residuals have positive variance. Partial correlation describes an adjusted linear association; it does not, by itself, establish a causal relationship or independence conditional on the controls.
Definition
Let and be random variables, and let be a vector of control variables. Assume that all variables have finite second moments. Define the residuals
where and are the population best linear predictors of and from , including an intercept. The population partial correlation is
Here, covariance measures joint variation, and variance measures the variation remaining in each variable after adjustment.
The phrase “controlling for ” means removing the components captured by these linear predictors. It does not generally mean comparing observations with exactly the same values of , nor removing every possible nonlinear relationship with .
A partial correlation controlling for one variable is called first-order; one controlling for variables is called th-order. Ordinary, unadjusted correlation is sometimes called zero-order correlation.
Calculation
One control variable
For three variables , the partial correlation is
The corresponding sample formula replaces the population correlations with sample correlations .
For example, suppose
Then
Thus, the residual linear association is weaker than the original correlation, but remains positive. Adjustment can also strengthen a correlation or reverse its sign; it is not inherently a procedure for reducing associations.
If either variable is perfectly linearly predicted by the controls, its residual variance is zero and the partial correlation is undefined.
Residual calculation in a sample
With observations:
- Regress on the controls, including an intercept.
- Regress on the same controls, also including an intercept.
- Calculate the Pearson correlation between the two residual vectors.
These regressions are ordinarily fitted by ordinary least squares. The same observations must be used in both regressions and in the residual correlation.
Geometrically, regression removes the orthogonal projections of the two data vectors onto the space spanned by the controls and the intercept. Partial correlation is the cosine of the angle between the remaining residual vectors. This interpretation explains both its symmetry in and and its restriction to the interval .
Covariance and inverse-matrix formulas
Let , and partition the covariance matrix of as
If is invertible, the covariance matrix of the residuals of after linear adjustment for is
Consequently,
For a full set of variables with a positive-definite covariance matrix , let , its inverse, also called the precision matrix. Then
This formula controls for every other variable included in the matrix. It need not equal a partial correlation that controls for only a selected subset.
Relationship to regression
Partial correlation is closely related to multiple linear regression. In the model
the fitted coefficient of can be obtained by regressing the residualized on the residualized . Accordingly,
where and are the residual standard deviations.
The regression coefficient and partial correlation therefore have the same sign, when both are defined. Their scales and interpretations differ:
- The regression coefficient measures change in , in its original units, per unit change in , conditional on the model.
- Partial correlation measures a unit-free association between residuals.
- Partial correlation is not generally equal to the standardized coefficient from a multiple regression.
Its square also has a regression interpretation. Let be the coefficient of determination for a regression of on , and let be that for a regression of on and . With an intercept and the same observations,
Thus, squared partial correlation is the fraction of the reduced model’s unexplained sum of squares accounted for by adding , not the fraction of the original total variation in .
Statistical inference
For testing
the usual test statistic is
Under independent observations from a nonsingular multivariate normal distribution, it has a Student’s distribution with degrees of freedom under the null hypothesis. The same test arises for the coefficient of under the classical normal-error linear regression model.
An approximate confidence interval can be constructed using Fisher’s transformation,
with approximate standard error
An interval formed on this transformed scale is mapped back using the hyperbolic tangent. This is a large-sample approximation, not a universal distribution-free interval.
Dependence between observations, nonconstant error variance, or model misspecification can invalidate the usual test and interval. Testing many partial correlations also raises multiple-testing issues.
Conditional correlation and independence
Partial correlation must be distinguished from correlation within a conditional distribution. The conditional correlation
may vary with . Ordinary partial correlation instead produces a single number from linear residualization.
For jointly multivariate normal variables, the conditional covariance matrix does not depend on the realized value of , and partial correlation equals conditional correlation. In that setting,
Outside this setting, zero partial correlation generally does not imply conditional independence. A nonlinear dependence can remain even when residual covariance is zero.
Conversely, conditional independence does not generally imply zero ordinary partial correlation. For example, let have a standard normal distribution and define
where the two mean-zero noise variables have finite positive variances and are mutually independent and independent of . Conditional on , and are independent. Yet their linear regressions on do not remove the shared quadratic component, so their partial correlation controlling linearly for is positive.
Replacing linear predictors with conditional expectations defines a different residual-based measure. Even then, zero residual correlation is not a general test of conditional independence.
Related measures and applications
Semipartial correlation
A semipartial correlation, also called a part correlation, removes the controls from only one of the two variables. For example,
residualizes but leaves unchanged.
When is added to a regression of on , the square of this semipartial correlation equals the increase in . Partial correlation instead expresses that increase relative to the variation left unexplained by the reduced model. Semipartial correlation is not generally symmetric in the two variables.
Graphical models
In a Gaussian graphical model, a zero off-diagonal entry of the precision matrix corresponds to conditional independence between two variables given all remaining variables. Partial correlations therefore provide standardized measures of the associations represented by edges in a Gaussian conditional-independence graph.
In high-dimensional data, the sample covariance matrix may be singular or poorly estimated. Methods using regularization can estimate a structured precision matrix, but the resulting partial correlations depend on the estimation method and tuning choices.
Rank-based variants
A common rank-based variant replaces observations by ranks and then calculates a partial correlation using the ranked variables. This can reduce sensitivity to measurement scale and some extreme observations. It still adjusts linearly on the chosen rank scale; it is not a general measure of conditional independence.
Interpretation and limitations
Partial correlation is useful for describing adjusted associations in observational studies, comparing predictors, and investigating multivariate dependence. Its interpretation depends on which controls are included and how their effects are modeled.
Important limitations include:
- Adjustment is not automatically causal. Removing associations with measured controls does not by itself eliminate confounding or identify a causal effect. The assumptions required for causal inference go beyond the correlation calculation.
- Control selection can introduce bias. Adjusting for a common consequence of two variables can create an association through selection bias. Adjusting for an intermediate variable can remove part of the pathway being studied.
- The default adjustment is linear and additive. Nonlinear terms and interactions are not removed unless represented in the control model.
- Measurement error matters. Imperfectly measured controls may leave substantial residual confounding.
- Instability can arise from redundant controls. Highly correlated controls or very small residual variances can make estimates sensitive to sampling variation and numerical error.
- Outliers and missingness can alter results. Residual correlation inherits the sensitivity of Pearson correlation and least-squares fitting, while different sets of complete observations can produce noncomparable estimates.
A partial correlation is therefore specified not just by its two focal variables, but also by its control set, the form of adjustment, and the population or sample in which it is calculated.