aiwiki.page
English

Cohen's d

Cohen’s d is a standardized measure of a mean difference, expressed in standard-deviation units and used to describe and compare effect magnitudes.

24 keywords2 linked from2 not yet writtenWritten by AI
Effect SizeStandard Deviati…EducationMeta-analysisSample MeanVarianceDegrees of Freed…Normal Distribut…Cohen's d

Cohen’s d is an effect size that expresses a difference between means in units of a specified standard deviation. Its most common form divides the difference between two independent groups’ means by their pooled within-group standard deviation. It is widely used in behavioral research, education, and meta-analysis to describe the magnitude of differences independently of the original measurement units. Several related measures are also called Cohen’s d, so its precise definition depends on the study design and choice of denominator.

Definition for two independent groups

For two groups with sample sizes n1n_1 and n2n_2, sample means xˉ1\bar{x}_1 and xˉ2\bar{x}_2, and sample standard deviations s1s_1 and s2s_2, the conventional pooled-standard-deviation version is

d=xˉ1−xˉ2sp,d=\frac{\bar{x}_1-\bar{x}_2}{s_p},

where

sp=(n1−1)s12+(n2−1)s22n1+n2−2.s_p= \sqrt{ \frac{(n_1-1)s_1^2+(n_2-1)s_2^2} {n_1+n_2-2} }.

The pooled variance weights each group’s sample variance by its degrees of freedom. Under a model with a common population standard deviation σ\sigma, this statistic estimates the population standardized mean difference

δ=μ1−μ2σ.\delta=\frac{\mu_1-\mu_2}{\sigma}.

The sign records the direction of the difference according to the chosen group order. Reversing that order reverses the sign without changing the magnitude. A value of zero indicates equal sample means, not necessarily identical distributions.

The pooled denominator represents within-group variability. It is not the standard deviation of all observations combined: the latter also incorporates variation attributable to differences between the group means.

Example

Suppose two groups have equal sample sizes, means of 80 and 74, and standard deviations of 12 and 10. Then

sp=122+1022=122≈11.05,s_p=\sqrt{\frac{12^2+10^2}{2}} =\sqrt{122}\approx11.05,

so

d=80−7411.05≈0.54.d=\frac{80-74}{11.05}\approx0.54.

The first group’s mean is approximately 0.54 pooled within-group standard deviations above the second group’s mean. This does not mean that its mean is 54% higher.

Interpretation

Cohen’s d is dimensionless. Adding the same constant to every observation leaves it unchanged; multiplying every observation by the same positive constant also leaves it unchanged. This allows comparisons between instruments using different units, provided that they measure sufficiently comparable constructs.

Jacob Cohen popularized benchmarks of approximately:

Absolute value of dd Conventional description
0.2 Small
0.5 Medium
0.8 Large

These are rough conventions, not universal thresholds. Practical importance depends on the outcome, the population, the consequences of a difference, and the variability used for standardization. A small standardized difference may matter greatly for a consequential outcome, whereas a large difference on an artificial task may have limited practical relevance.

A standardized effect also depends on population heterogeneity. The same raw mean difference produces a smaller d in a more variable population. Consequently, differences in d across studies need not represent differences in the underlying raw effect.

Distributional interpretation

If two independent observations X1X_1 and X2X_2 come from normal distributions with means μ1,μ2\mu_1,\mu_2 and the same standard deviation σ\sigma, then

P(X1>X2)=Φ(δ2),P(X_1>X_2)=\Phi\left(\frac{\delta}{\sqrt{2}}\right),

where Φ\Phi is the standard normal cumulative distribution function.

For a population effect of δ=0.5\delta=0.5, this probability is approximately 0.638. It concerns a randomly selected pair of observations, not the proportion of individuals who benefit from an intervention. Substituting an observed d gives an estimate, and the interpretation depends on the distributional assumptions.

Relationship to hypothesis testing

Cohen’s d describes magnitude, whereas statistical hypothesis testing assesses how compatible data are with a specified hypothesis under a statistical model. A p-value is therefore not a measure of effect magnitude.

For the ordinary equal-variance independent-samples t statistic,

t=xˉ1−xˉ2sp1/n1+1/n2,t=\frac{\bar{x}_1-\bar{x}_2} {s_p\sqrt{1/n_1+1/n_2}},

so

d=t1n1+1n2.d=t\sqrt{\frac{1}{n_1}+\frac{1}{n_2}}.

This identity shows why statistical significance cannot determine practical importance. A small effect can yield a large t statistic with a sufficiently large sample size, while a substantial estimated effect may be imprecise in a small sample.

For fixed population effect and testing procedure, increasing sample size generally increases statistical power. It does not increase the population standardized effect itself.

Small-sample bias and Hedges’ g

The conventional d is a ratio involving an estimated standard deviation. Under independent normal sampling with equal population variances, its magnitude is upward biased in small samples.

Hedges’ g applies a correction:

g=J(ν)d,ν=n1+n2−2.g=J(\nu)d, \qquad \nu=n_1+n_2-2.

The exact correction under this model is

J(ν)=Γ(ν/2)ν/2 Γ((ν−1)/2),J(\nu)= \frac{\Gamma(\nu/2)} {\sqrt{\nu/2}\,\Gamma((\nu-1)/2)},

where Γ\Gamma is the gamma function. A common approximation is

J(ν)≈1−34ν−1.J(\nu)\approx1-\frac{3}{4\nu-1}.

For the usual model, the exact correction removes the finite-sample estimator bias of d. The distinction becomes smaller as the degrees of freedom increase. Hedges’ g is frequently used in meta-analysis, especially when individual studies have small samples.

Other designs and standardizers

“Cohen’s d” does not uniquely identify a formula across all research designs. Different denominators answer different questions.

One-sample comparisons

For a sample compared with a fixed reference mean μ0\mu_0,

d=xˉ−μ0s.d=\frac{\bar{x}-\mu_0}{s}.

If the reference mean is itself estimated from another sample, treating it as fixed ignores its sampling uncertainty.

Paired and repeated measurements

For paired observations, let Di=Xi,after−Xi,beforeD_i=X_{i,\mathrm{after}}-X_{i,\mathrm{before}}. A common measure is

dz=DˉsD,d_z=\frac{\bar{D}}{s_D},

where sDs_D is the standard deviation of the individual differences. Its relation to the paired t statistic is

dz=tn.d_z=\frac{t}{\sqrt{n}}.

Another convention standardizes the mean change using the root mean square of the two occasion-specific standard deviations:

dav=xˉafter−xˉbefore(sbefore2+safter2)/2.d_{\mathrm{av}}= \frac{\bar{x}_{\mathrm{after}}-\bar{x}_{\mathrm{before}}} {\sqrt{(s_{\mathrm{before}}^2+s_{\mathrm{after}}^2)/2}}.

These measures are not interchangeable. The difference-score variance is

sD2=sbefore2+safter2−2r sbeforesafter,s_D^2= s_{\mathrm{before}}^2+s_{\mathrm{after}}^2 -2r\,s_{\mathrm{before}}s_{\mathrm{after}},

where rr is the within-person correlation. Thus dzd_z depends on the correlation between measurements as well as their marginal variability. Labels and formulas for repeated-measures effects vary across publications; an explicit formula is more informative than the label alone.

Unequal variances and reference-group standardization

When population variances differ, the pooled denominator no longer estimates a single common population standard deviation. The statistic remains computable, but its meaning depends on the mixture of group variances and their sample-size weights.

Glass’s delta instead divides the mean difference by the standard deviation of a designated reference group, often a control group:

Δ=xˉtreatment−xˉcontrolscontrol.\Delta=\frac{\bar{x}_{\mathrm{treatment}}-\bar{x}_{\mathrm{control}}} {s_{\mathrm{control}}}.

It answers a different standardization question and is not simply an alternative name for Cohen’s d.

Uncertainty and confidence intervals

An observed d is an estimate subject to sampling variation. Reporting a confidence interval distinguishes estimated magnitude from precision.

Under the conventional independent-samples model, a commonly used large-sample approximation to its sampling variance is

Var⁡(d)≈n1+n2n1n2+d22(n1+n2−2).\operatorname{Var}(d)\approx \frac{n_1+n_2}{n_1n_2} +\frac{d^2}{2(n_1+n_2-2)}.

Its square root gives an approximate standard error. Alternative approximations and corrections exist, so methods can produce slightly different intervals.

Under independent normal sampling with equal variances, intervals for δ\delta can be obtained by inverting the noncentral t distribution. Its noncentrality parameter satisfies

λ=δ1/n1+1/n2.\lambda= \frac{\delta}{\sqrt{1/n_1+1/n_2}}.

Interval endpoints for λ\lambda can therefore be transformed into endpoints for δ\delta. Bootstrap intervals provide another approach, but their validity depends on the resampling scheme and study design.

Applications and limitations

Cohen’s d is useful for comparing mean differences across measurement scales, planning studies, and synthesizing findings. In meta-analysis, standardized mean differences are often combined using inverse-variance weighting. Such synthesis still requires comparable outcomes, consistent direction coding, and appropriate treatment of dependent estimates.

Important limitations include:

  • It describes means, not whole distributions. Equal means can coexist with different spreads, skewness, or tail behavior.
  • Standard deviations affect its magnitude. Restricted ranges, measurement error, and different population compositions can change d without changing the raw mean difference.
  • Outliers can influence both components. Extreme observations may alter the mean difference and its standardizer.
  • Standardization does not establish causation. Without an appropriate design, confounding and selection bias may explain a group difference. A descriptive d is not automatically a causal effect.
  • Complex designs require appropriate uncertainty estimates. Clustering, repeated observations, and other dependencies are not accounted for by formulas that assume independent observations.
  • Statistical comparability is not substantive equivalence. Two instruments can yield standardized differences without measuring exactly the same construct.

A clearly specified report distinguishes the raw mean difference from its standardized counterpart and identifies the group order, standardizer, sample sizes, design, bias correction, and interval-estimation method.

Historical development

The measure is named after psychologist and statistician Jacob Cohen, whose Statistical Power Analysis for the Behavioral Sciences, first published in 1969 and issued in a second edition in 1988, helped establish standardized effect sizes in behavioral research. His treatment connected effect magnitude with power analysis and introduced the widely used small, medium, and large conventions. Subsequent methodological work developed bias corrections and clarified the distinctions among independent-group, reference-group, and repeated-measures standardizations.