Adjusted R-squared, usually written or , is a measure of fit for a linear regression model that adjusts the ordinary coefficient of determination for the number of parameters estimated. Unlike ordinary , which cannot decrease when predictors are added to an ordinary least-squares model fitted to the same observations, adjusted can decrease. It therefore provides a way to compare explanatory fit while penalizing the inclusion of additional predictors.
Definition
For a regression fitted by ordinary least squares with an intercept, let:
- be the number of observations;
- be the number of estimated slope coefficients, excluding the intercept;
- be the observed response;
- be the fitted response;
- be the sample mean of the response.
Define the residual sum of squares and total sum of squares as
Ordinary is
Adjusted is
This definition requires and positive residual degrees of freedom, . If the response is constant, the centered total sum of squares is zero and the usual statistic is undefined.
The parameter count refers to estimated coefficients rather than named variables. A categorical predictor represented by several indicator columns, or a polynomial predictor represented by several powers, can contribute more than one coefficient. If the design matrix is rank-deficient, residual degrees of freedom are , where is its rank; with an intercept, replaces .
Statistical interpretation
The adjustment compares two quantities measured on the scale of variance:
Thus,
Here, is the sample variance of the response. Under a correctly specified linear model with zero-mean, uncorrelated errors having a common variance, is an unbiased estimator of the error variance.
Ordinary rewards every reduction in training residuals, including reductions obtained by fitting random variation. Adjusted counteracts this by dividing the residual sum of squares by the degrees of freedom remaining after fitting the model.
Despite this interpretation, adjusted is not literally a decomposition of observed variation into explained and unexplained fractions. Nor is it, in general, an exactly unbiased estimator of a population coefficient of determination. Unbiasedness of individual variance estimates does not imply unbiasedness of their ratio.
Behavior and range
For an ordinary least-squares model with an intercept:
- .
- An intercept-only model has .
- A perfect fit has , provided positive residual degrees of freedom remain.
- Adjusted can be negative.
- With a fixed number of predictors, the adjustment becomes smaller as the sample size increases.
The gap between the two measures is
A negative adjusted means that the residual variance estimate exceeds the sample variance of the response. It does not mean that a negative proportion of variation has literally been explained.
For a fixed dataset, maximizing adjusted is equivalent to minimizing the residual mean square, .
What happens when predictors are added?
Consider two nested models fitted to the same observations. The reduced model has slope coefficients, and the full model adds independently estimable coefficients. Write their residual sums of squares as and .
For a full model with positive residual sum of squares, define the partial -statistic as
Then
Thus, adjusted increases only when the improvement per added parameter exceeds the full model’s residual mean square. This relationship is algebraic; interpreting the statistic through an -distribution requires additional assumptions.
When one coefficient is added, the corresponding ordinary least-squares -statistic satisfies . Adjusted therefore rises when . This is not equivalent to passing a conventional hypothesis test at the 5% significance level. Maximizing adjusted can consequently retain predictors that would not meet that testing threshold.
Numerical example
Suppose several models are fitted to the same 100 observations:
| Model | Slope coefficients | Ordinary | Adjusted |
|---|---|---|---|
| A | 2 | 0.600 | 0.5918 |
| B | 3 | 0.602 | 0.5896 |
| C | 3 | 0.610 | 0.5978 |
For model A,
If model B adds a predictor to model A, its small improvement in ordinary is insufficient to offset the extra parameter. Model C provides a larger improvement and has a higher adjusted . Models B and C have the same parameter count, so their ranking by adjusted is identical to their ranking by ordinary .
Uses and limitations
Adjusted is commonly reported in regression analyses in statistics, econometrics, and other quantitative disciplines. It is useful for comparing models with different numbers of predictors, provided they use the same response, observations, and fitting convention.
Its interpretation has several important limits.
It is not an out-of-sample performance measure
Adjusted is computed from the data used to fit the model. Its parameter penalty does not directly measure prediction error on new observations. Searching through many models can produce an optimistically high selected value. Cross-validation or an independent test dataset addresses predictive performance more directly, using measures such as mean squared error.
It does not establish model validity or causation
A high adjusted does not demonstrate that the functional form is correct, error assumptions hold, or influential observations are absent. It also does not establish causation: omitted variables, confounding, and other identification problems can remain even when fit is high.
Comparisons require a common basis
Comparing adjusted values is generally inappropriate when models use different observations or differently transformed responses. For example, a model for and a model for measure residual variation on different scales. Changes caused by dropping observations with missing predictor values also cannot be attributed solely to the predictor set.
Other model classes need different conventions
Without an intercept, some software uses an uncentered total sum of squares, , and a corresponding adjustment. Such values are not directly comparable with the centered statistic defined above.
For regularized models, such as ridge regression, the number of fitted coefficients is not generally an adequate measure of effective model complexity. Similarly, likelihood-based models such as logistic regression use various pseudo- measures; their adjusted forms are not the same statistic as adjusted for ordinary least squares.