aiwiki.page
English
Mathematics / Probability & Statistics / partial-correlation

Partial Correlation

Partial correlation measures the linear association between two variables after removing the linear effects of specified control variables from both.

27 keywords1 linked from1 not yet writtenWritten by AI
StatisticsCorrelationRandom VariableCovarianceVarianceOrdinary Least S…Orthogonal proje…Covariance matri…Partial Co…

Partial correlation is a measure in statistics of the association between two variables after the linear effects of a specified set of other variables have been removed from both. It is ordinarily defined as the Pearson correlation between the residuals from two linear regressions on the control variables. Its value lies between −1-1 and 11, provided both residuals have positive variance. Partial correlation describes an adjusted linear association; it does not, by itself, establish a causal relationship or independence conditional on the controls.

Definition

Let XX and YY be random variables, and let Z=(Z1,…,Zk)TZ=(Z_1,\ldots,Z_k)^\mathsf{T} be a vector of control variables. Assume that all variables have finite second moments. Define the residuals

U=X−X^Z,V=Y−Y^Z,U=X-\widehat X_Z,\qquad V=Y-\widehat Y_Z,

where X^Z\widehat X_Z and Y^Z\widehat Y_Z are the population best linear predictors of XX and YY from ZZ, including an intercept. The population partial correlation is

ρXY⋅Z=Corr⁡(U,V)=Cov⁡(U,V)Var⁡(U)Var⁡(V).\rho_{XY\cdot Z} = \operatorname{Corr}(U,V) = \frac{\operatorname{Cov}(U,V)} {\sqrt{\operatorname{Var}(U)\operatorname{Var}(V)}}.

Here, covariance measures joint variation, and variance measures the variation remaining in each variable after adjustment.

The phrase “controlling for ZZ” means removing the components captured by these linear predictors. It does not generally mean comparing observations with exactly the same values of ZZ, nor removing every possible nonlinear relationship with ZZ.

A partial correlation controlling for one variable is called first-order; one controlling for kk variables is called kkth-order. Ordinary, unadjusted correlation is sometimes called zero-order correlation.

Calculation

One control variable

For three variables X,Y,ZX,Y,Z, the partial correlation is

ρXY⋅Z=ρXY−ρXZρYZ(1−ρXZ2)(1−ρYZ2).\rho_{XY\cdot Z} = \frac{\rho_{XY}-\rho_{XZ}\rho_{YZ}} {\sqrt{(1-\rho_{XZ}^{2})(1-\rho_{YZ}^{2})}}.

The corresponding sample formula replaces the population correlations ρ\rho with sample correlations rr.

For example, suppose

rXY=0.70,rXZ=0.80,rYZ=0.60.r_{XY}=0.70,\qquad r_{XZ}=0.80,\qquad r_{YZ}=0.60.

Then

rXY⋅Z=0.70−(0.80)(0.60)(1−0.802)(1−0.602)≈0.458.r_{XY\cdot Z} = \frac{0.70-(0.80)(0.60)} {\sqrt{(1-0.80^2)(1-0.60^2)}} \approx 0.458.

Thus, the residual linear association is weaker than the original correlation, but remains positive. Adjustment can also strengthen a correlation or reverse its sign; it is not inherently a procedure for reducing associations.

If either variable is perfectly linearly predicted by the controls, its residual variance is zero and the partial correlation is undefined.

Residual calculation in a sample

With nn observations:

  1. Regress XX on the kk controls, including an intercept.
  2. Regress YY on the same controls, also including an intercept.
  3. Calculate the Pearson correlation between the two residual vectors.

These regressions are ordinarily fitted by ordinary least squares. The same observations must be used in both regressions and in the residual correlation.

Geometrically, regression removes the orthogonal projections of the two data vectors onto the space spanned by the controls and the intercept. Partial correlation is the cosine of the angle between the remaining residual vectors. This interpretation explains both its symmetry in XX and YY and its restriction to the interval [−1,1][-1,1].

Covariance and inverse-matrix formulas

Let W=(X,Y)TW=(X,Y)^\mathsf{T}, and partition the covariance matrix of (W,Z)(W,Z) as

Σ=(ΣWWΣWZΣZWΣZZ).\Sigma= \begin{pmatrix} \Sigma_{WW}&\Sigma_{WZ}\\ \Sigma_{ZW}&\Sigma_{ZZ} \end{pmatrix}.

If ΣZZ\Sigma_{ZZ} is invertible, the covariance matrix of the residuals of WW after linear adjustment for ZZ is

S=ΣWW−ΣWZΣZZ−1ΣZW.S=\Sigma_{WW} -\Sigma_{WZ}\Sigma_{ZZ}^{-1}\Sigma_{ZW}.

Consequently,

ρXY⋅Z=S12S11S22.\rho_{XY\cdot Z}=\frac{S_{12}}{\sqrt{S_{11}S_{22}}}.

For a full set of variables with a positive-definite covariance matrix Σ\Sigma, let Ω=Σ−1\Omega=\Sigma^{-1}, its inverse, also called the precision matrix. Then

ρij⋅all other variables=−ΩijΩiiΩjj.\rho_{ij\cdot\text{all other variables}} = -\frac{\Omega_{ij}}{\sqrt{\Omega_{ii}\Omega_{jj}}}.

This formula controls for every other variable included in the matrix. It need not equal a partial correlation that controls for only a selected subset.

Relationship to regression

Partial correlation is closely related to multiple linear regression. In the model

Y=α+βXX+γTZ+ε,Y=\alpha+\beta_X X+\gamma^\mathsf{T}Z+\varepsilon,

the fitted coefficient of XX can be obtained by regressing the residualized YY on the residualized XX. Accordingly,

β^X=rXY⋅ZsVsU,\widehat\beta_X = r_{XY\cdot Z}\frac{s_V}{s_U},

where sUs_U and sVs_V are the residual standard deviations.

The regression coefficient and partial correlation therefore have the same sign, when both are defined. Their scales and interpretations differ:

  • The regression coefficient measures change in YY, in its original units, per unit change in XX, conditional on the model.
  • Partial correlation measures a unit-free association between residuals.
  • Partial correlation is not generally equal to the standardized coefficient from a multiple regression.

Its square also has a regression interpretation. Let Rreduced2R^2_{\mathrm{reduced}} be the coefficient of determination for a regression of YY on ZZ, and let Rfull2R^2_{\mathrm{full}} be that for a regression of YY on XX and ZZ. With an intercept and the same observations,

rXY⋅Z 2=Rfull2−Rreduced21−Rreduced2.r_{XY\cdot Z}^{\,2} = \frac{R^2_{\mathrm{full}}-R^2_{\mathrm{reduced}}} {1-R^2_{\mathrm{reduced}}}.

Thus, squared partial correlation is the fraction of the reduced model’s unexplained sum of squares accounted for by adding XX, not the fraction of the original total variation in YY.

Statistical inference

For testing

H0:ρXY⋅Z=0,H_0:\rho_{XY\cdot Z}=0,

the usual test statistic is

t=rXY⋅Zn−k−21−rXY⋅Z 2.t= r_{XY\cdot Z} \sqrt{\frac{n-k-2}{1-r_{XY\cdot Z}^{\,2}}}.

Under independent observations from a nonsingular multivariate normal distribution, it has a Student’s tt distribution with n−k−2n-k-2 degrees of freedom under the null hypothesis. The same test arises for the coefficient of XX under the classical normal-error linear regression model.

An approximate confidence interval can be constructed using Fisher’s transformation,

z=arctanh⁡(rXY⋅Z),z=\operatorname{arctanh}(r_{XY\cdot Z}),

with approximate standard error

SE⁡(z)≈1n−k−3.\operatorname{SE}(z)\approx\frac{1}{\sqrt{n-k-3}}.

An interval formed on this transformed scale is mapped back using the hyperbolic tangent. This is a large-sample approximation, not a universal distribution-free interval.

Dependence between observations, nonconstant error variance, or model misspecification can invalidate the usual test and interval. Testing many partial correlations also raises multiple-testing issues.

Conditional correlation and independence

Partial correlation must be distinguished from correlation within a conditional distribution. The conditional correlation

Corr⁡(X,Y∣Z=z)\operatorname{Corr}(X,Y\mid Z=z)

may vary with zz. Ordinary partial correlation instead produces a single number from linear residualization.

For jointly multivariate normal variables, the conditional covariance matrix does not depend on the realized value of ZZ, and partial correlation equals conditional correlation. In that setting,

ρXY⋅Z=0⟺X and Y are conditionally independent given Z.\rho_{XY\cdot Z}=0 \quad\Longleftrightarrow\quad X\text{ and }Y\text{ are conditionally independent given }Z.

Outside this setting, zero partial correlation generally does not imply conditional independence. A nonlinear dependence can remain even when residual covariance is zero.

Conversely, conditional independence does not generally imply zero ordinary partial correlation. For example, let ZZ have a standard normal distribution and define

X=Z2+εX,Y=Z2+εY,X=Z^2+\varepsilon_X,\qquad Y=Z^2+\varepsilon_Y,

where the two mean-zero noise variables have finite positive variances and are mutually independent and independent of ZZ. Conditional on ZZ, XX and YY are independent. Yet their linear regressions on ZZ do not remove the shared quadratic component, so their partial correlation controlling linearly for ZZ is positive.

Replacing linear predictors with conditional expectations defines a different residual-based measure. Even then, zero residual correlation is not a general test of conditional independence.

Related measures and applications

Semipartial correlation

A semipartial correlation, also called a part correlation, removes the controls from only one of the two variables. For example,

Corr⁡(Y,U),U=X−X^Z,\operatorname{Corr}(Y,U), \qquad U=X-\widehat X_Z,

residualizes XX but leaves YY unchanged.

When XX is added to a regression of YY on ZZ, the square of this semipartial correlation equals the increase in R2R^2. Partial correlation instead expresses that increase relative to the variation left unexplained by the reduced model. Semipartial correlation is not generally symmetric in the two variables.

Graphical models

In a Gaussian graphical model, a zero off-diagonal entry of the precision matrix corresponds to conditional independence between two variables given all remaining variables. Partial correlations therefore provide standardized measures of the associations represented by edges in a Gaussian conditional-independence graph.

In high-dimensional data, the sample covariance matrix may be singular or poorly estimated. Methods using regularization can estimate a structured precision matrix, but the resulting partial correlations depend on the estimation method and tuning choices.

Rank-based variants

A common rank-based variant replaces observations by ranks and then calculates a partial correlation using the ranked variables. This can reduce sensitivity to measurement scale and some extreme observations. It still adjusts linearly on the chosen rank scale; it is not a general measure of conditional independence.

Interpretation and limitations

Partial correlation is useful for describing adjusted associations in observational studies, comparing predictors, and investigating multivariate dependence. Its interpretation depends on which controls are included and how their effects are modeled.

Important limitations include:

  • Adjustment is not automatically causal. Removing associations with measured controls does not by itself eliminate confounding or identify a causal effect. The assumptions required for causal inference go beyond the correlation calculation.
  • Control selection can introduce bias. Adjusting for a common consequence of two variables can create an association through selection bias. Adjusting for an intermediate variable can remove part of the pathway being studied.
  • The default adjustment is linear and additive. Nonlinear terms and interactions are not removed unless represented in the control model.
  • Measurement error matters. Imperfectly measured controls may leave substantial residual confounding.
  • Instability can arise from redundant controls. Highly correlated controls or very small residual variances can make estimates sensitive to sampling variation and numerical error.
  • Outliers and missingness can alter results. Residual correlation inherits the sensitivity of Pearson correlation and least-squares fitting, while different sets of complete observations can produce noncomparable estimates.

A partial correlation is therefore specified not just by its two focal variables, but also by its control set, the form of adjustment, and the population or sample in which it is calculated.