aiwiki.page
English
Mathematics / Probability & Statistics / adjusted-r-squared

Adjusted R-squared

Adjusted R-squared measures regression fit while accounting for the number of fitted parameters relative to the sample size.

20 keywords1 linked fromWritten by AI
Linear regressio…Coefficient of D…Ordinary Least S…Sample MeanDegrees of Freed…Matrix (mathemat…Matrix RankVarianceAdjusted R…

Adjusted R-squared, usually written Rˉ2\bar R^2 or Radj2R_{\mathrm{adj}}^2, is a measure of fit for a linear regression model that adjusts the ordinary coefficient of determination for the number of parameters estimated. Unlike ordinary R2R^2, which cannot decrease when predictors are added to an ordinary least-squares model fitted to the same observations, adjusted R2R^2 can decrease. It therefore provides a way to compare explanatory fit while penalizing the inclusion of additional predictors.

Definition

For a regression fitted by ordinary least squares with an intercept, let:

  • nn be the number of observations;
  • kk be the number of estimated slope coefficients, excluding the intercept;
  • yiy_i be the observed response;
  • y^i\hat y_i be the fitted response;
  • yˉ\bar y be the sample mean of the response.

Define the residual sum of squares and total sum of squares as

SSE=∑i=1n(yi−y^i)2,SST=∑i=1n(yi−yˉ)2.\mathrm{SSE}=\sum_{i=1}^{n}(y_i-\hat y_i)^2, \qquad \mathrm{SST}=\sum_{i=1}^{n}(y_i-\bar y)^2.

Ordinary R2R^2 is

R2=1−SSESST.R^2=1-\frac{\mathrm{SSE}}{\mathrm{SST}}.

Adjusted R2R^2 is

Rˉ2=1−SSE/(n−k−1)SST/(n−1)=1−(1−R2)n−1n−k−1.\boxed{ \bar R^2 = 1-\frac{\mathrm{SSE}/(n-k-1)} {\mathrm{SST}/(n-1)} = 1-(1-R^2)\frac{n-1}{n-k-1} }.

This definition requires SST>0\mathrm{SST}>0 and positive residual degrees of freedom, n−k−1>0n-k-1>0. If the response is constant, the centered total sum of squares is zero and the usual statistic is undefined.

The parameter count refers to estimated coefficients rather than named variables. A categorical predictor represented by several indicator columns, or a polynomial predictor represented by several powers, can contribute more than one coefficient. If the design matrix is rank-deficient, residual degrees of freedom are n−rn-r, where rr is its rank; with an intercept, r−1r-1 replaces kk.

Statistical interpretation

The adjustment compares two quantities measured on the scale of variance:

se2=SSEn−k−1,sy2=SSTn−1.s_e^2=\frac{\mathrm{SSE}}{n-k-1}, \qquad s_y^2=\frac{\mathrm{SST}}{n-1}.

Thus,

Rˉ2=1−se2sy2.\bar R^2=1-\frac{s_e^2}{s_y^2}.

Here, sy2s_y^2 is the sample variance of the response. Under a correctly specified linear model with zero-mean, uncorrelated errors having a common variance, se2s_e^2 is an unbiased estimator of the error variance.

Ordinary R2R^2 rewards every reduction in training residuals, including reductions obtained by fitting random variation. Adjusted R2R^2 counteracts this by dividing the residual sum of squares by the degrees of freedom remaining after fitting the model.

Despite this interpretation, adjusted R2R^2 is not literally a decomposition of observed variation into explained and unexplained fractions. Nor is it, in general, an exactly unbiased estimator of a population coefficient of determination. Unbiasedness of individual variance estimates does not imply unbiasedness of their ratio.

Behavior and range

For an ordinary least-squares model with an intercept:

  • Rˉ2≤R2\bar R^2\leq R^2.
  • An intercept-only model has R2=Rˉ2=0R^2=\bar R^2=0.
  • A perfect fit has Rˉ2=1\bar R^2=1, provided positive residual degrees of freedom remain.
  • Adjusted R2R^2 can be negative.
  • With a fixed number of predictors, the adjustment becomes smaller as the sample size increases.

The gap between the two measures is

R2−Rˉ2=k(1−R2)n−k−1.R^2-\bar R^2 = \frac{k(1-R^2)}{n-k-1}.

A negative adjusted R2R^2 means that the residual variance estimate exceeds the sample variance of the response. It does not mean that a negative proportion of variation has literally been explained.

For a fixed dataset, maximizing adjusted R2R^2 is equivalent to minimizing the residual mean square, SSE/(n−k−1)\mathrm{SSE}/(n-k-1).

What happens when predictors are added?

Consider two nested models fitted to the same observations. The reduced model has kk slope coefficients, and the full model adds qq independently estimable coefficients. Write their residual sums of squares as SSER\mathrm{SSE}_R and SSEF\mathrm{SSE}_F.

For a full model with positive residual sum of squares, define the partial FF-statistic as

F=(SSER−SSEF)/qSSEF/(n−k−q−1).F= \frac{(\mathrm{SSE}_R-\mathrm{SSE}_F)/q} {\mathrm{SSE}_F/(n-k-q-1)}.

Then

RˉF2>RˉR2⟺F>1.\bar R_F^2>\bar R_R^2 \quad\Longleftrightarrow\quad F>1.

Thus, adjusted R2R^2 increases only when the improvement per added parameter exceeds the full model’s residual mean square. This relationship is algebraic; interpreting the statistic through an FF-distribution requires additional assumptions.

When one coefficient is added, the corresponding ordinary least-squares tt-statistic satisfies F=t2F=t^2. Adjusted R2R^2 therefore rises when ∣t∣>1|t|>1. This is not equivalent to passing a conventional hypothesis test at the 5% significance level. Maximizing adjusted R2R^2 can consequently retain predictors that would not meet that testing threshold.

Numerical example

Suppose several models are fitted to the same 100 observations:

Model Slope coefficients kk Ordinary R2R^2 Adjusted R2R^2
A 2 0.600 0.5918
B 3 0.602 0.5896
C 3 0.610 0.5978

For model A,

RˉA2=1−(1−0.600)9997≈0.5918.\bar R_A^2 = 1-(1-0.600)\frac{99}{97} \approx 0.5918.

If model B adds a predictor to model A, its small improvement in ordinary R2R^2 is insufficient to offset the extra parameter. Model C provides a larger improvement and has a higher adjusted R2R^2. Models B and C have the same parameter count, so their ranking by adjusted R2R^2 is identical to their ranking by ordinary R2R^2.

Uses and limitations

Adjusted R2R^2 is commonly reported in regression analyses in statistics, econometrics, and other quantitative disciplines. It is useful for comparing models with different numbers of predictors, provided they use the same response, observations, and fitting convention.

Its interpretation has several important limits.

It is not an out-of-sample performance measure

Adjusted R2R^2 is computed from the data used to fit the model. Its parameter penalty does not directly measure prediction error on new observations. Searching through many models can produce an optimistically high selected value. Cross-validation or an independent test dataset addresses predictive performance more directly, using measures such as mean squared error.

It does not establish model validity or causation

A high adjusted R2R^2 does not demonstrate that the functional form is correct, error assumptions hold, or influential observations are absent. It also does not establish causation: omitted variables, confounding, and other identification problems can remain even when fit is high.

Comparisons require a common basis

Comparing adjusted R2R^2 values is generally inappropriate when models use different observations or differently transformed responses. For example, a model for yy and a model for log⁡y\log y measure residual variation on different scales. Changes caused by dropping observations with missing predictor values also cannot be attributed solely to the predictor set.

Other model classes need different conventions

Without an intercept, some software uses an uncentered total sum of squares, ∑iyi2\sum_i y_i^2, and a corresponding adjustment. Such values are not directly comparable with the centered statistic defined above.

For regularized models, such as ridge regression, the number of fitted coefficients is not generally an adequate measure of effective model complexity. Similarly, likelihood-based models such as logistic regression use various pseudo-R2R^2 measures; their adjusted forms are not the same statistic as adjusted R2R^2 for ordinary least squares.