aiwiki.page
English
Mathematics / Probability & Statistics / random-effects-model

Random-effects Model

A statistical model that represents variation among groups, subjects, or studies through probability distributions assigned to group-specific effects.

50 keywords4 linked from7 not yet writtenWritten by AI
Random VariableProbability Dist…Mixed-effects mo…Fixed-effects mo…VarianceRandom samplingAnalysis of Vari…Experimental Des…Random-eff…

A random-effects model is a statistical model in which some effects associated with groups, subjects, experimental units, or other sources of variation are represented as random variables. Rather than estimating every group effect as an unrelated parameter, the model describes those effects through a shared probability distribution, typically characterized by a mean and one or more variance parameters. This allows information to be combined across groups while retaining differences among them.

Random-effects models are central to the analysis of clustered and repeated observations: pupils within schools, repeated measurements within individuals, production batches within factories, and results from separate studies. They explain why observations from the same group may resemble one another and provide a framework for estimating population-average relationships, variation between groups, and predictions for particular groups. A model containing both fixed and random effects is generally called a mixed-effects model; models with explicitly organized levels are also known as multilevel models.

The meaning of “random effects” varies somewhat across statistical traditions. In experimental design it often concerns variation among sampled factor levels; in longitudinal analysis it represents subject-specific trajectories; in econometrics it commonly denotes an assumption about unobserved unit characteristics and explanatory variables; and in meta-analysis it describes variation in underlying effects across studies. These uses share a probabilistic treatment of heterogeneity, but their assumptions and inferential targets are not interchangeable.

Meaning, interpretation, and historical development

The distinction between fixed and random effects concerns how variation is represented and what population the analysis addresses. In a fixed-effects model, the effects associated with observed groups are treated as separate unknown constants. In a random-effects formulation, group effects are modeled as realizations from a distribution. The unknown quantities then include parameters of that distribution, such as its variance, as well as any common regression coefficients.

For example, an experiment comparing three specifically chosen manufacturing processes may treat process as a fixed factor because those three processes exhaust the comparisons of interest. If the experiment also uses batches sampled from continuing production, batch-to-batch differences may be represented by random effects. The distinction does not mean that fixed effects are physically immutable or that random effects change unpredictably every time a measurement is taken. A batch effect can remain constant for all observations from that batch while being random in the model describing variation across batches.

Actual random sampling of group levels gives a direct justification for population generalization, but it is not a universal prerequisite for using a random-effects model. A model may instead posit that observed groups are sufficiently exchangeable after accounting for measured predictors. Exchangeability means, informally, that their joint distribution is unchanged by relabeling the groups. Whether this approximation is defensible depends on the sampling process, substantive setting, and included covariates. Modeling a convenient set of schools as exchangeable does not by itself make them representative of all schools.

The historical development is closely connected to analysis of variance and experimental design. In the early twentieth century, Ronald Fisher developed methods for partitioning observed variation into components associated with different sources. Balanced agricultural and biological experiments made it possible to estimate variance components through expected mean squares. These methods were especially convenient when groups had equal sizes and the design yielded orthogonal comparisons.

Animal breeding subsequently became an important setting for mixed-model theory. C. R. Henderson developed computational methods for simultaneously estimating fixed effects and predicting random genetic effects, publishing his influential variance-component work in 1953. Restricted maximum likelihood was developed by H. D. Patterson and R. Thompson in 1971, providing a likelihood-based approach that accounts for the estimation of fixed effects. Later work connected these methods with longitudinal models, hierarchical Bayesian analysis, and large-scale numerical computation. The longitudinal random-effects formulation of Nan Laird and James Ware, published in 1982, became particularly influential for irregularly timed repeated measurements.

Linear formulation and induced dependence

The standard linear mixed model is written

y=Xβ+Zb+ε.\mathbf y=X\boldsymbol\beta+Z\mathbf b+\boldsymbol\varepsilon.

Here y\mathbf y contains nn observations, XX is the design matrix for the fixed coefficients β\boldsymbol\beta, and ZZ links observations to a vector of random effects b\mathbf b. A conventional Gaussian specification assumes

b∼N(0,G),ε∼N(0,R),\mathbf b\sim N(\mathbf 0,G),\qquad \boldsymbol\varepsilon\sim N(\mathbf 0,R),

with independence between b\mathbf b and ε\boldsymbol\varepsilon. The matrices GG and RR describe random-effect and residual covariance. Integrating out the random effects gives

y∼N(Xβ,V),V=ZGZT+R.\mathbf y\sim N(X\boldsymbol\beta,V), \qquad V=ZGZ^{\mathsf T}+R.

The superscript T\mathsf T denotes transposition. This marginal representation shows that random effects provide a structured model for the covariance matrix of the observations.

A simple random-intercept model for observation jj in group ii is

yij=β0+xijTβ+bi+εij,y_{ij}=\beta_0+\mathbf x_{ij}^{\mathsf T}\boldsymbol\beta +b_i+\varepsilon_{ij},

where bi∼N(0,τ2)b_i\sim N(0,\tau^2) and εij∼N(0,σ2)\varepsilon_{ij}\sim N(0,\sigma^2). If residuals are independent, each observation has variance τ2+σ2\tau^2+\sigma^2, while two distinct observations in the same group have covariance τ2\tau^2. Observations in different independent groups have zero covariance. Their within-group correlation is therefore

ρ=τ2τ2+σ2.\rho=\frac{\tau^2}{\tau^2+\sigma^2}.

This quantity is the intraclass correlation coefficient. It measures the fraction of unexplained marginal variance attributable to differences between groups under this particular model.

For illustration, if the between-group variance is 99 and the within-group residual variance is 1616, the intraclass correlation is 9/25=0.369/25=0.36. Collecting additional observations from an existing group then adds less independent information than collecting comparable observations from new groups. In a balanced, intercept-only setting with mm observations per group, the familiar design-effect expression 1+(m−1)ρ1+(m-1)\rho describes the inflation in the variance of a mean relative to an independent-observation calculation. It is not a universal correction for every regression coefficient or sampling design.

The conditional and marginal descriptions answer different questions. Conditional on b\mathbf b, the model describes outcomes for particular groups. Marginally, it describes the distribution obtained across groups. In the linear model with zero-mean random effects, both descriptions share the mean XβX\boldsymbol\beta, although their uncertainty differs. Normality makes likelihood calculations and conditional predictions especially tractable; the covariance identity itself requires only appropriate second moments and the stated independence assumptions.

Random intercepts, random slopes, and grouping structures

A random intercept allows groups to differ in their baseline level but assumes that the effect of each predictor is common across groups. A random-slope model also permits a predictor’s association with the outcome to vary. For repeated measurements over time, a common specification is

yij=β0+β1tij+b0i+b1itij+εij,y_{ij}=\beta_0+\beta_1t_{ij} +b_{0i}+b_{1i}t_{ij}+\varepsilon_{ij},

with

(b0ib1i)∼N[(00),(τ02τ01τ01τ12)].\begin{pmatrix}b_{0i}\\b_{1i}\end{pmatrix} \sim N\left[ \begin{pmatrix}0\\0\end{pmatrix}, \begin{pmatrix} \tau_0^2&\tau_{01}\\ \tau_{01}&\tau_1^2 \end{pmatrix} \right].

The covariance τ01\tau_{01} describes whether groups with higher values at the time origin tend to have larger or smaller slopes. A nonzero random-slope variance means that trajectories differ even after accounting for the common slope.

For two measurements at times ss and tt on the same individual, the random-effect contribution to covariance is

τ02+(s+t)τ01+stτ12.\tau_0^2+(s+t)\tau_{01}+st\tau_1^2.

Thus random slopes produce dependence that changes with measurement times, rather than the constant covariance of a random-intercept model. Centering time changes the meaning of the intercept and transforms the intercept–slope covariance. It does not change the fitted family of trajectories when the full covariance structure is transformed consistently. Restricting intercepts and slopes to be independent, however, makes the choice of time origin consequential.

Grouping factors may be nested or crossed. Pupils within classes within schools form a nested structure when every pupil belongs to one class and every class to one school. A model can assign separate random effects to schools and classes, with class identifiers interpreted within schools. Crossed structures occur when observations simultaneously belong to groups that do not nest: a response may depend on both the person producing it and the test item answered. Random person and item effects then induce dependence along two different dimensions.

Some observations have multiple memberships within the same grouping system. A pupil taught by several teachers may receive a weighted combination of teacher effects rather than a single teacher intercept. Genetic models provide another extension: random effects for related individuals can have covariance proportional to a known or estimated relationship matrix. Independence among group effects is therefore a common special case, not a defining requirement of every random-effects model.

Random effects also need not explain all within-group dependence. Repeated measurements may retain serial correlation after individual intercepts and slopes have been included. The residual matrix RR can represent this additional structure, for example through an autoregressive model or time-dependent measurement variance. Distinguishing a slowly varying individual trajectory from residual serial dependence requires sufficiently informative observation times and replication. Adding many covariance parameters without such information can produce structures that the data cannot separately identify.

Estimation of coefficients and variance components

If VV were known, the fixed coefficients could be estimated by generalized least squares:

β^=(XTV−1X)−1XTV−1y,\widehat{\boldsymbol\beta} =(X^{\mathsf T}V^{-1}X)^{-1} X^{\mathsf T}V^{-1}\mathbf y,

assuming the required rank conditions hold. Unlike ordinary least squares, this calculation weights observations according to both their variances and their dependence. In practice, VV contains unknown variance components, so coefficient estimation and covariance estimation are linked. Estimated covariance parameters are substituted into the generalized least-squares expression, usually within an iterative fitting procedure.

In a balanced one-way random-intercept model with mm observations per group, classical analysis of variance gives

E(MSwithin)=σ2,E(MSbetween)=σ2+mτ2.E(MS_{\text{within}})=\sigma^2,\qquad E(MS_{\text{between}})=\sigma^2+m\tau^2.

This yields the moment estimates

σ^2=MSwithin,τ^2=MSbetween−MSwithinm.\widehat{\sigma}^2=MS_{\text{within}},\qquad \widehat{\tau}^2= \frac{MS_{\text{between}}-MS_{\text{within}}}{m}.

The second expression can be negative even though a variance cannot be. Such results reflect sampling variation in an unconstrained moment estimator. Unbalanced designs are more complicated because simple mean-square decompositions no longer isolate the components so neatly.

Maximum likelihood estimation instead optimizes the marginal Gaussian likelihood. Apart from constants, its log likelihood is

ℓ(β,θ)=−12[log⁡∣V∣+(y−Xβ)TV−1(y−Xβ)+nlog⁡(2π)],\ell(\boldsymbol\beta,\boldsymbol\theta) =-\frac12\left[ \log|V| +(\mathbf y-X\boldsymbol\beta)^{\mathsf T} V^{-1}(\mathbf y-X\boldsymbol\beta) +n\log(2\pi) \right],

where θ\boldsymbol\theta denotes covariance parameters. Variances are constrained to be nonnegative, and covariance matrices must be positive semidefinite. Parameterizations based on triangular factors are commonly used to enforce these constraints. Maximum likelihood estimates of variance components can be downward biased in small samples because fitting the mean structure uses information that a direct likelihood maximization does not explicitly deduct.

Restricted maximum likelihood, usually abbreviated REML, bases estimation on linear combinations of the observations that eliminate the fixed effects. For a full-rank XX, an equivalent objective includes

ℓR(θ)=−12[log⁡∣V∣+log⁡∣XTV−1X∣+yTPy+(n−p)log⁡(2π)],\ell_R(\boldsymbol\theta) =-\frac12\left[ \log|V| +\log|X^{\mathsf T}V^{-1}X| +\mathbf y^{\mathsf T}P\mathbf y +(n-p)\log(2\pi) \right],

up to a parameter-independent constant, where

P=V−1−V−1X(XTV−1X)−1XTV−1P=V^{-1}-V^{-1}X (X^{\mathsf T}V^{-1}X)^{-1}X^{\mathsf T}V^{-1}

and pp is the number of fixed-effect coefficients. REML often reduces finite-sample bias, but it does not guarantee unbiased estimates in every design.

Numerical fitting may use Newton-type methods, scoring algorithms, or the expectation–maximization algorithm. Sparse factorizations are especially important when there are many groups but relatively few observations linked to each random effect. REML likelihoods generally cannot be compared directly across models with different fixed-effect spaces; ordinary maximum likelihood is commonly used for such comparisons. A variance estimate of zero is a legitimate boundary solution, whereas failure of the numerical optimizer is a separate computational issue.

Prediction, shrinkage, and Bayesian interpretation

Random-effects models distinguish estimation of population parameters from prediction of group-specific deviations. If the covariance parameters are known, the best linear unbiased predictor of the random-effect vector is

b^=GZTV−1(y−Xβ^).\widehat{\mathbf b} =GZ^{\mathsf T}V^{-1} (\mathbf y-X\widehat{\boldsymbol\beta}).

“Best” refers to minimum prediction-error variance within the relevant class of linear unbiased predictors under the assumed covariance model. It does not assert optimality under arbitrary misspecification. When covariance parameters are estimated, the resulting predictor is usually called an empirical BLUP.

The prediction mechanism is especially transparent in an intercept-only model with independent residuals and nin_i observations in group ii. If the overall mean μ\mu and variance components are treated as known,

E(bi∣yi)=λi(yˉi−μ),λi=τ2τ2+σ2/ni.E(b_i\mid\mathbf y_i) =\lambda_i(\bar y_i-\mu), \qquad \lambda_i= \frac{\tau^2}{\tau^2+\sigma^2/n_i}.

The predicted group mean is

μ^i=μ+λi(yˉi−μ).\widehat{\mu}_i =\mu+\lambda_i(\bar y_i-\mu).

This is a weighted combination of the observed group mean and the overall mean. Groups with little information receive more shrinkage toward the common mean; well-measured groups receive less. Greater between-group variation also reduces shrinkage because large group differences become more plausible under the model.

This behavior is often called partial pooling. Complete pooling would assign every group the same mean, while no pooling would estimate each group independently. Partial pooling can lower average mean squared error by accepting some bias for individual group effects in exchange for reduced sampling variability. It does not imply that every particular group prediction improves. Unusual groups may be pulled substantially toward a common distribution that describes them poorly, making the adequacy of that distribution substantively important.

Under Gaussian assumptions, the prediction formula also has a Bayesian interpretation. With β\boldsymbol\beta and covariance parameters known,

b∣y∼N(GZTV−1(y−Xβ), G−GZTV−1ZG).\mathbf b\mid\mathbf y \sim N\left( GZ^{\mathsf T}V^{-1}(\mathbf y-X\boldsymbol\beta), \ G-GZ^{\mathsf T}V^{-1}ZG \right).

The conditional mean is both the predictor and the mean of the posterior distribution. Estimating distributional parameters from the data and then treating them as fixed is an empirical Bayes procedure. A full Bayesian analysis instead assigns distributions to regression coefficients and covariance parameters and integrates over their uncertainty.

The same idea connects random effects to regularization. For known positive-definite GG and RR, Gaussian fitting can be expressed through minimizing

(y−Xβ−Zb)TR−1(y−Xβ−Zb)+bTG−1b.(\mathbf y-X\boldsymbol\beta-Z\mathbf b)^{\mathsf T} R^{-1}(\mathbf y-X\boldsymbol\beta-Z\mathbf b) +\mathbf b^{\mathsf T}G^{-1}\mathbf b.

The second term penalizes large random effects, resembling ridge regression. Unlike a fixed externally chosen penalty, its strength is often estimated through variance components. Predictions for entirely new groups additionally require drawing or integrating over a new group effect, rather than assuming that the new group’s deviation is already known.

Inference, identifiability, and model assessment

Inference for fixed coefficients is relatively straightforward when covariance parameters are known: the coefficient estimator has covariance (XTV−1X)−1(X^{\mathsf T}V^{-1}X)^{-1}. With estimated covariance components, plugging estimates into this expression neglects some uncertainty. Large-sample normal approximations can be adequate when the design provides many independent groups, but many measurements within only a few groups do not necessarily justify them. For cluster-level predictors, the number and arrangement of clusters are especially important.

Small-sample methods include Satterthwaite approximations to denominator degrees of freedom and Kenward–Roger adjustments to coefficient covariance and reference distributions. These methods address aspects of uncertainty introduced by covariance estimation, but their performance still depends on design and model assumptions. A confidence interval for a population coefficient is distinct from an interval for an existing group’s latent effect and from a prediction interval for a future observation. The last must include the relevant residual variation and, for a new group, new between-group variation.

Testing variance components raises a boundary problem. Under the null hypothesis τ2=0\tau^2=0, the parameter lies on the edge of its permitted range. Consequently, the ordinary likelihood-ratio reference distribution need not be a standard chi-squared distribution. In some simple one-component settings the asymptotic distribution is an equal mixture of a point mass at zero and a chi-squared distribution with one degree of freedom. More complicated random-slope and covariance tests can have different reference behavior. Parametric bootstrap sampling can approximate a test’s null distribution by repeatedly simulating and refitting the specified model.

Identifiability concerns whether distinct parameter values imply distinguishable observable distributions. With one observation per group, a simple random-intercept variance and an independent residual variance cannot be separated: only their sum is observed. Random slopes require sufficient variation in the associated predictor within relevant groups. An estimated random-effect covariance matrix may be singular, with one or more zero eigenvalues; this indicates that the fitted variation occupies fewer dimensions than the proposed structure. Singularity is not synonymous with numerical nonconvergence, although the two can occur together.

Model assessment considers both the mean and covariance specifications. Residual patterns may reveal nonlinear mean relationships, unequal residual variances, or serial dependence. Distributions of predicted random effects require caution because shrinkage makes them differ from directly observed independent draws. Simulation-based checks can compare observed summaries with replicated data under the fitted model. Cross-validation must also match the prediction target: withholding observations within existing groups evaluates a different task from withholding entire groups. Finally, informative cluster size or outcome-related dropout can invalidate a standard likelihood analysis when the observation process is treated as ignorable despite depending on unmodeled latent effects.

Generalized and nonlinear random-effects models

For binary outcomes, counts, and other non-Gaussian responses, random effects can be incorporated into a generalized linear model. A generalized linear mixed model specifies a conditional response distribution together with

g(μij)=xijTβ+zijTbi,g(\mu_{ij}) =\mathbf x_{ij}^{\mathsf T}\boldsymbol\beta +\mathbf z_{ij}^{\mathsf T}\mathbf b_i,

where gg is a link function and μij=E(yij∣bi)\mu_{ij}=E(y_{ij}\mid\mathbf b_i). A binary model may use a Bernoulli distribution and the logit link; a count model may use a Poisson distribution and a logarithmic link. Conditional independence is often assumed after accounting for the random effects, although additional dependence structures are possible.

Unlike the Gaussian linear case, the marginal mean generally differs from the inverse link applied to the fixed linear predictor:

E ⁣[g−1(Xβ+Zb)]≠g−1(Xβ).E\!\left[g^{-1}(X\boldsymbol\beta+Z\mathbf b)\right] \ne g^{-1}(X\boldsymbol\beta).

For a logistic random-intercept model, a fixed coefficient describes a conditional log odds ratio comparing observations at the same random-effect value. It is not generally equal to the log odds ratio obtained after averaging over groups. This distinction can occur without confounding because odds ratios are non-collapsible. Consequently, conditional and population-average models may legitimately produce different coefficient magnitudes.

Some marginal quantities have simple forms. In a Poisson log-link model with an independent Gaussian random intercept bi∼N(0,τ2)b_i\sim N(0,\tau^2),

E(yij∣xij)=exp⁡(xijTβ+τ2/2).E(y_{ij}\mid\mathbf x_{ij}) =\exp(\mathbf x_{ij}^{\mathsf T}\boldsymbol\beta+\tau^2/2).

The random intercept both alters the marginal mean and induces overdispersion relative to an ordinary Poisson model. It also creates positive dependence among observations sharing the intercept. Overdispersion can arise through other mechanisms, however, so a random intercept and an alternative conditional response distribution do not necessarily represent the same scientific explanation.

The marginal likelihood requires integration over the random effects. Logistic-normal models usually lack a closed-form integral. Fitting methods include Laplace approximation, adaptive Gaussian quadrature, numerical integration, and simulation-based approaches. Bayesian implementations often use Markov chain Monte Carlo. Approximation accuracy depends on cluster sizes, response sparsity, random-effect dimension, and the magnitude of heterogeneity. Binary data with few observations per group can provide limited information about complex covariance structures.

Nonlinear mixed models extend the approach by allowing a nonlinear response function,

yij=f(tij,β,bi)+εij.y_{ij}=f(t_{ij},\boldsymbol\beta,\mathbf b_i) +\varepsilon_{ij}.

Examples include growth curves and concentration–time curves in which individual-specific parameters describe rates, capacities, or decay constants. Transformations can enforce constraints, such as positive rate parameters. Random effects on different nonlinear parameters may imply markedly different observable trajectories, and their distributions must be interpreted on the scale where they are defined. These models often require careful numerical optimization because mean parameters and variance parameters can interact strongly.

Random effects in panel-data econometrics

In econometrics, random effects commonly arise in panel data, where the same units are observed repeatedly. A standard model is

yit=xitTβ+αi+εit.y_{it}=\mathbf x_{it}^{\mathsf T}\boldsymbol\beta +\alpha_i+\varepsilon_{it}.

The unit-specific effect αi\alpha_i represents persistent unobserved characteristics. The conventional random-effects formulation assumes that this effect is uncorrelated with the regressors across observation periods; stronger formulations impose an appropriate zero conditional mean. This assumption is central to consistent estimation, not merely a convenient distributional detail.

With an independent homoskedastic residual and a balanced panel of TT observations per unit, generalized least squares can be implemented through quasi-demeaning:

yit−θyˉi=(xit−θxˉi)Tβ+(1−θ)αi+εit−θεˉi,y_{it}-\theta\bar y_i =(\mathbf x_{it}-\theta\bar{\mathbf x}_i)^{\mathsf T} \boldsymbol\beta +(1-\theta)\alpha_i +\varepsilon_{it}-\theta\bar\varepsilon_i,

where

θ=1−σε2σε2+Tσα2.\theta= 1-\sqrt{\frac{\sigma_\varepsilon^2} {\sigma_\varepsilon^2+T\sigma_\alpha^2}}.

When σα2=0\sigma_\alpha^2=0, the transformation becomes pooled regression. As the persistent unit variation becomes large relative to residual variation, θ\theta approaches one and the transformation approaches within-unit demeaning. Unbalanced panels have unit-specific versions involving TiT_i.

A fixed-effects panel estimator eliminates αi\alpha_i through within-unit deviations or equivalent unit indicators. Under suitable exogeneity conditions, this permits correlation between persistent unit characteristics and the regressors. The trade-off is that coefficients of predictors constant within each unit cannot be separately estimated from unrestricted unit effects. A random-effects model can estimate such coefficients, but only through assumptions that make between-unit comparisons informative about the intended relationship.

The Hausman specification test, introduced by Jerry Hausman in 1978, compares estimators whose difference should be small under the random-effects restrictions. A systematic discrepancy between fixed-effects and random-effects estimates can indicate that the latter restrictions fail. The test is not a universal decision rule: its standard form depends on covariance assumptions, and robust implementations require care. A nonsignificant result does not establish independence between unobserved effects and regressors.

Correlated random-effects formulations provide another approach. Yair Mundlak’s 1978 formulation represents part of the unit effect through unit-level regressor means:

αi=xˉiTπ+ai.\alpha_i=\bar{\mathbf x}_i^{\mathsf T}\boldsymbol\pi+a_i.

Under the corresponding conditional-mean assumptions, including these means separates within-unit and between-unit associations while retaining a random residual effect. This makes explicit a relationship that a conventional random-effects model excludes. Nevertheless, neither random effects nor fixed effects automatically solve all forms of endogeneity. Time-varying omitted variables, reverse causation, measurement error, or dynamic feedback can require additional assumptions and methods. A covariance model for repeated observations is not by itself a causal identification strategy.

Random-effects meta-analysis and between-study heterogeneity

In meta-analysis, a random-effects model describes variation in underlying effects across studies. A basic two-stage formulation is

θ^i∣θi∼N(θi,si2),θi∼N(μ,τ2),\widehat{\theta}_i\mid\theta_i \sim N(\theta_i,s_i^2), \qquad \theta_i\sim N(\mu,\tau^2),

where θ^i\widehat{\theta}_i is a study’s estimated effect size, si2s_i^2 is its sampling variance, and τ2\tau^2 is the between-study variance. Marginally,

θ^i∼N(μ,si2+τ2).\widehat{\theta}_i\sim N(\mu,s_i^2+\tau^2).

The model separates uncertainty within studies from genuine variation among their underlying effects. Its interpretation depends on whether the included studies can reasonably be regarded as representing a meaningful distribution of study settings.

For a specified τ2\tau^2, the estimated mean effect is an inverse-variance weighted average,

μ^=∑iwiθ^i∑iwi,wi=1si2+τ2.\widehat{\mu} =\frac{\sum_iw_i\widehat{\theta}_i}{\sum_iw_i}, \qquad w_i=\frac{1}{s_i^2+\tau^2}.

When τ2=0\tau^2=0, the weights reduce to those of a common-effect analysis. As heterogeneity increases, weights become more similar, so small studies receive greater relative weight than under common-effect weighting. This change does not correct study bias. If smaller studies differ systematically because of selective publication or other mechanisms, changing weights may alter the pooled estimate without resolving the underlying problem.

Rebecca DerSimonian and Nan Laird’s 1986 method popularized a moment-based estimate of between-study variance. REML and other estimators are also widely used. The uncertainty in τ^2\widehat{\tau}^2 can be substantial when few studies are available. Conventional plug-in intervals for μ\mu can then underrepresent uncertainty, motivating methods such as Hartung–Knapp-type inference and Bayesian analyses with explicit distributions for heterogeneity. Their behavior depends on study count, variance patterns, and the particular implementation.

A confidence interval for μ\mu concerns the mean of the modeled distribution of underlying effects. A prediction interval instead concerns the true effect in a new comparable study setting. A schematic large-sample expression is

μ^±qτ^2+Var⁡^(μ^),\widehat{\mu}\pm q\sqrt{\widehat{\tau}^2+ \widehat{\operatorname{Var}}(\widehat{\mu})},

where the reference quantile qq depends on the inferential method. This interval does not include sampling error in a future study’s observed estimate; predicting that estimate requires its sampling variance as well. Prediction intervals can be much wider than confidence intervals for the mean.

Meta-regression introduces study-level predictors to explain some heterogeneity, leaving a residual between-study variance. Associations at the study level need not equal individual-level associations. Furthermore, multiple outcomes or comparisons from the same study violate a simple independent-study formulation unless their dependence is represented. Multivariate and multilevel meta-analysis can model such structures, but neither a heterogeneity parameter nor a richer hierarchy removes selection bias, errors in reported sampling variances, or systematic differences in study design.