{
  "id": "mixed-effects-model",
  "title": "Mixed-effects model",
  "lang": "en",
  "source_lang": "en",
  "summary": "A statistical model combining fixed effects and random effects to describe population relationships, group variation, and dependence among observations.",
  "category": "mathematics",
  "subcategory": "probability-statistics",
  "type": "concept",
  "aliases": [
    "Mixed model",
    "Multilevel model",
    "Hierarchical linear model",
    "Random-effects regression model"
  ],
  "generated_at": "2026-11-05",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/mixed-effects-model/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/mixed-effects-model/",
      "json": "https://aiwiki.page/zh/mixed-effects-model.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "A mixed-effects model is a statistical model that combines fixed effects, represented by coefficients shared across a population or specified groups, with random effects, represented by unobserved quantities assigned a probability distribution. The distinction allows a model to describe both systematic relationships and variation among subjects, institutions, locations, experimental units, or other groupings. A familiar example is a study of repeated measurements in which each person has an individual baseline while the population shares an average relationship between an outcome and time.\n\nMixed-effects models are important when observations are not independent, when information is distributed unevenly across groups, or when variation among groups is itself of scientific interest. They extend linear regression and connect with analysis of variance, longitudinal analysis, Bayesian inference, and statistical prediction. Their principal forms include linear mixed models for approximately continuous outcomes, generalized linear mixed models for binary or count outcomes, and nonlinear mixed models for relationships such as biological growth and drug concentration curves.\n\nThe terms multilevel model and hierarchical model emphasize the organization of observations into levels, whereas mixed-effects model emphasizes the combination of fixed and random components. Their usage overlaps substantially, although mixed-effects models also accommodate crossed structures that are not strictly hierarchical. The central methodological questions concern how group variation is represented, how model parameters are estimated, how information is shared across groups, and which conclusions remain valid under the model’s assumptions."
    },
    {
      "heading": "Linear mixed-model structure",
      "text": "The standard linear mixed model is\n\n\\[\\mathbf y=X\\boldsymbol\\beta+Z\\mathbf b+\\boldsymbol\\varepsilon,\\]\n\nwhere \\(\\mathbf y\\) contains \\(n\\) observations, \\(X\\) is the fixed-effects design matrix, \\(\\boldsymbol\\beta\\) contains fixed coefficients, \\(Z\\) is the random-effects design matrix, and \\(\\mathbf b\\) contains random effects. A common Gaussian specification is\n\n\\[\\mathbf b\\sim N(\\mathbf 0,G),\\qquad \\boldsymbol\\varepsilon\\sim N(\\mathbf 0,R),\\]\n\nwith \\(\\mathbf b\\) and \\(\\boldsymbol\\varepsilon\\) independent. Here \\(G\\) and \\(R\\) are covariance matrices governed by a usually much smaller vector of unknown parameters. Often \\(R=\\sigma^2I\\), although unequal residual variances and serial dependence can also be represented.\n\nConditional on the random effects, the mean is \\(X\\boldsymbol\\beta+Z\\mathbf b\\). After averaging over their distribution, the model becomes\n\n\\[\\mathbf y\\sim N(X\\boldsymbol\\beta,V), \\qquad V=ZGZ^\\top+R.\\]\n\nThis distinction between conditional and marginal descriptions is fundamental. Random effects enter the conditional mean, but they also induce dependence in the marginal distribution. Two observations can therefore be conditionally independent given their shared random effect while remaining correlated when that effect is unobserved.\n\nA random-intercept model for observation \\(j\\) in group \\(i\\) is\n\n\\[y_{ij}=\\beta_0+\\beta_1x_{ij}+b_{0i}+\\varepsilon_{ij},\\]\n\nwith \\(b_{0i}\\sim N(0,\\tau^2)\\) and independent residuals of variance \\(\\sigma^2\\). Each group has its own intercept \\(\\beta_0+b_{0i}\\). Two distinct observations from the same group have covariance \\(\\tau^2\\), whereas observations from different groups are independent under this specification. The resulting intraclass correlation coefficient is\n\n\\[\\rho=\\frac{\\tau^2}{\\tau^2+\\sigma^2}.\\]\n\nIt measures the fraction of marginal outcome variation attributable to group-level intercept differences in this particular model.\n\nA random-slope model additionally allows the relationship with a predictor to vary:\n\n\\[y_{ij}=\\beta_0+\\beta_1t_{ij} +b_{0i}+b_{1i}t_{ij}+\\varepsilon_{ij}.\\]\n\nIf the intercept and slope variances are \\(\\tau_{00}\\) and \\(\\tau_{11}\\), with covariance \\(\\tau_{01}\\), then\n\n\\[\\operatorname{Cov}(y_{ij},y_{ik}) =\\tau_{00}+\\tau_{01}(t_{ij}+t_{ik}) +\\tau_{11}t_{ij}t_{ik}\\]\n\nfor distinct observations with independent residuals. Unlike the random-intercept case, dependence changes with the measurement times. Centering time changes the interpretation and covariance of the intercept and slope without changing the underlying fitted trajectories when the model is transformed consistently."
    },
    {
      "heading": "Fixed effects, random effects, and grouping structures",
      "text": "A fixed effect is ordinarily a coefficient estimated directly without assigning it a population distribution within the sampling model. A random effect is a latent group-specific quantity modeled as a draw from a distribution. Both can be unknown; the distinction is not that fixed effects are known or that random effects arise only from randomized experiments. Nor is a variable intrinsically fixed or random. The modeling choice depends on the intended comparison, the population being represented, and the assumptions that can reasonably be made about differences among groups.\n\nFor example, a model comparing three particular teaching methods may represent method contrasts with fixed coefficients while representing variation among schools with random intercepts. School indicators could instead be included as fixed coefficients, producing a different model with different assumptions and inferential targets. Treating schools as random expresses a distributional model for school differences and enables prediction for additional schools drawn from that modeled population. It does not establish that the observed schools were selected through representative random sampling.\n\nGrouping structures may be nested, crossed, or partially crossed. Pupils nested within classrooms nested within schools form a hierarchy. In a language experiment, however, each participant may respond to multiple words and each word may be presented to multiple participants. Participant and word effects are then crossed. A model can contain both:\n\n\\[y_{ij}=\\mathbf x_{ij}^{\\top}\\boldsymbol\\beta +u_i+v_j+\\varepsilon_{ij}.\\]\n\nObservations share dependence when they involve either the same participant or the same word. Multiple-membership models extend this idea to observations associated with several groups simultaneously, such as pupils taught by several teachers, potentially using specified membership weights.\n\nIn econometrics, “fixed effects” often refers more specifically to unit-specific intercepts used to control for stable unobserved differences. A conventional random-intercept model assumes that these latent differences have mean zero conditional on the included predictors. If high-intercept units systematically have higher predictor values for unmodeled reasons, this assumption can fail, producing endogeneity. Adding a random intercept does not automatically remove such bias.\n\nA useful decomposition separates a changing predictor into its within-group deviation and group mean:\n\n\\[y_{ij}=\\beta_0+ \\beta_W(x_{ij}-\\bar x_i)+ \\beta_B\\bar x_i+b_i+\\varepsilon_{ij}.\\]\n\nThe within-group coefficient \\(\\beta_W\\) compares observations within the same group; \\(\\beta_B\\) compares groups with different predictor means. These relationships need not coincide. Correlated-random-effects formulations associated with Yair Mundlak model the dependence of group heterogeneity on observed group means. They clarify assumptions about unobserved heterogeneity but do not, by themselves, establish causal identification."
    },
    {
      "heading": "Likelihood estimation and variance components",
      "text": "For a Gaussian linear mixed model, integrating out the random effects gives an explicit likelihood function. Apart from constants, the log likelihood is\n\n\\[\\ell(\\boldsymbol\\beta,\\boldsymbol\\theta) =-\\frac12\\left[ \\log|V(\\boldsymbol\\theta)| +(\\mathbf y-X\\boldsymbol\\beta)^\\top V(\\boldsymbol\\theta)^{-1} (\\mathbf y-X\\boldsymbol\\beta) \\right],\\]\n\nwhere \\(\\boldsymbol\\theta\\) parameterizes the covariance structure. Maximum likelihood estimation chooses fixed coefficients and covariance parameters jointly. For a given \\(V\\), the fixed-effect estimate is the generalized least-squares solution\n\n\\[\\widehat{\\boldsymbol\\beta} =(X^\\top V^{-1}X)^{-1}X^\\top V^{-1}\\mathbf y.\\]\n\nThe covariance parameters generally require numerical optimization rather than a closed-form solution.\n\nAn important alternative is restricted maximum likelihood, usually abbreviated REML. It estimates covariance parameters using combinations of the observations that eliminate the fixed-effect mean. With \\(p\\) linearly independent columns in \\(X\\), these combinations occupy an \\(n-p\\)-dimensional space. Up to terms not depending on the covariance parameters, the restricted log likelihood contains\n\n\\[-\\frac12\\left[ \\log|V|+\\log|X^\\top V^{-1}X| +\\mathbf y^\\top P\\mathbf y \\right],\\]\n\nwhere\n\n\\[P=V^{-1}-V^{-1}X(X^\\top V^{-1}X)^{-1}X^\\top V^{-1}.\\]\n\nThe additional determinant term reflects the estimation of the fixed effects.\n\nIn the elementary model of independent observations with an unknown common mean, maximum likelihood estimates the residual variance using the divisor \\(n\\), whereas REML gives the familiar divisor \\(n-1\\). This illustrates why REML often reduces downward bias in variance estimation. It is not a guarantee that every covariance estimate is unbiased in every unbalanced or constrained model. Restricted likelihoods based on different fixed-effect design spaces are also not directly comparable in ordinary likelihood-ratio tests. Maximum likelihood is generally used when comparing models that differ in their fixed-effect terms.\n\nVariance components must satisfy structural constraints. Variances cannot be negative, and a random-intercept–slope covariance matrix must be positive semidefinite. Implementations commonly use a Cholesky factorization or another constrained parameterization to enforce these requirements. A fitted variance of zero is a legitimate boundary estimate; it is not necessarily an optimization error. Likewise, a covariance matrix of reduced rank can indicate that the data support fewer independent dimensions of random variation than the specified model allows.\n\nComputational approaches include Newton-type methods, profile likelihood, and the expectation–maximization algorithm. Their behavior depends strongly on predictor scaling, the number of covariance parameters, and the amount of replication. Increasing the number of observations within a small number of groups does not provide the same information about between-group variation as observing many independent groups."
    },
    {
      "heading": "Partial pooling and prediction of random effects",
      "text": "One of the distinctive consequences of a mixed-effects model is partial pooling: group estimates lie between completely separate estimates and a single common estimate. In a random-intercept model with known variance components, suppose group \\(i\\) has \\(n_i\\) observations and residual mean\n\n\\[\\bar r_i=\\frac{1}{n_i}\\sum_j (y_{ij}-\\mathbf x_{ij}^{\\top}\\boldsymbol\\beta).\\]\n\nThe conditional mean of its random intercept is\n\n\\[E(b_i\\mid\\mathbf y) =\\lambda_i\\bar r_i,\\qquad \\lambda_i=\\frac{\\tau^2}{\\tau^2+\\sigma^2/n_i}.\\]\n\nThe multiplier \\(\\lambda_i\\) lies between zero and one. A large group with precise measurements receives little shrinkage, while a small or noisy group is pulled more strongly toward the population mean.\n\nThis is a model-based allocation of information, not simply a mechanical preference for small effects. If estimated between-group variation is large relative to measurement noise, even moderately sized groups remain distinct. If between-group variation is close to zero, group estimates are pooled strongly. With random slopes, shrinkage acts jointly on intercepts and slopes according to their covariance structure. The resulting estimate need not be a coordinate-by-coordinate movement toward zero because information about one random coefficient can affect another.\n\nFor the Gaussian linear mixed model with known covariance parameters,\n\n\\[\\widehat{\\mathbf b} =GZ^\\top V^{-1} (\\mathbf y-X\\widehat{\\boldsymbol\\beta})\\]\n\nis the best linear unbiased predictor, or BLUP, of the random effects under the usual estimability conditions. “Best” refers to minimum prediction error variance within the class of linear unbiased predictors, not to universal optimality under arbitrary misspecification. Once \\(G\\) and \\(R\\) are estimated, the result is commonly called an empirical BLUP. Under Gaussian assumptions, and with the model parameters known, the corresponding expression is also the conditional expectation of the random effects.\n\nShrinkage connects mixed models with regularization. For known covariance matrices, estimating coefficients and predicting random effects can be expressed through minimization of\n\n\\[(\\mathbf y-X\\boldsymbol\\beta-Z\\mathbf b)^\\top R^{-1}(\\mathbf y-X\\boldsymbol\\beta-Z\\mathbf b) +\\mathbf b^\\top G^{-1}\\mathbf b.\\]\n\nThe second term penalizes random effects in directions determined by \\(G\\). For independent equal-variance coefficients, this resembles ridge regression, but the penalty strength is tied to estimated variance components rather than necessarily being selected as an external tuning parameter.\n\nPrediction must distinguish existing and new groups. For an observed group, its data inform a group-specific random effect. For a new group with no observations, the predicted random effect is zero under a centered model, but the uncertainty still includes its between-group variance. A prediction interval for a new outcome additionally includes residual variability. Intervals that treat estimated variance components as known can understate uncertainty, especially when the number of groups is small."
    },
    {
      "heading": "Inference for fixed effects and variance components",
      "text": "Inference in mixed models is complicated by estimated covariance parameters and by the distinction between observation counts and independent replication. With known \\(V\\), the covariance of the generalized least-squares estimator is\n\n\\[\\operatorname{Var}(\\widehat{\\boldsymbol\\beta}) =(X^\\top V^{-1}X)^{-1}.\\]\n\nIn practice, \\(V\\) is estimated. Simply substituting its estimate produces familiar model-based standard errors, but the resulting normal or Wald approximations may be inaccurate when there are few groups, highly unequal group sizes, or weakly identified variance components. The relevant effective information is specific to the coefficient being tested.\n\nFor Gaussian linear mixed models, Satterthwaite-type approximations assign approximate denominator degrees of freedom to tests. Kenward–Roger methods additionally adjust the estimated covariance of fixed effects and approximate the distribution of test statistics. These methods address finite-sample behavior rather than changing the underlying scientific question. Their availability and implementation vary across model classes and software, and no single denominator degrees of freedom applies automatically to every coefficient in a complex model.\n\nLikelihood-ratio tests compare nested models through twice the difference in maximized log likelihood. For a regular fixed-effect comparison fitted by maximum likelihood, a large-sample chi-squared distribution may provide a useful approximation. Testing a variance component against zero is different because zero lies on the boundary of the allowable parameter space. In a simple setting with one tested variance component and suitable regularity conditions, the asymptotic null distribution is a half-and-half mixture of a point mass at zero and a chi-squared distribution with one degree of freedom. More complicated random-effects tests need not follow that mixture.\n\nParametric bootstrap procedures approximate a test or interval distribution by simulating datasets under a fitted model and refitting them. They can incorporate the actual group sizes and covariance structure, although their validity still depends on the assumed data-generating model. A nonparametric bootstrap must preserve the relevant dependence structure: independently resampling individual observations is generally inappropriate when those observations are clustered. Even cluster resampling becomes difficult when very few independent clusters are available.\n\nA confidence interval for an average slope, a prediction interval for a group-specific slope, and an interval for a variance component answer different questions. Testing an average slope of zero does not test whether every group has zero slope; an average effect can vanish while substantial heterogeneity remains. Conversely, a nonzero average does not establish that the effect has the same sign in every group. The interpretation of hypothesis tests therefore depends on whether the target is a population mean, a particular group, or the distribution of group-specific effects."
    },
    {
      "heading": "Generalized linear and nonlinear mixed models",
      "text": "A generalized linear mixed model extends a generalized linear model by adding random effects to its linear predictor. Conditional on \\(\\mathbf b\\), outcomes commonly follow Bernoulli, binomial, Poisson, or another exponential-family distribution, with\n\n\\[g(\\mu_{ij}) =\\mathbf x_{ij}^{\\top}\\boldsymbol\\beta +\\mathbf z_{ij}^{\\top}\\mathbf b,\\]\n\nwhere \\(g\\) is a link function and \\(\\mu_{ij}=E(y_{ij}\\mid\\mathbf b)\\). Conditional independence is often assumed after accounting for the included random effects, although additional dependence structures are possible. A random-intercept logistic model describes variation in conditional log odds; a Poisson mixed model with a log link describes multiplicative heterogeneity in event rates.\n\nUnlike the Gaussian linear case, the marginal likelihood usually requires numerical integration:\n\n\\[L(\\boldsymbol\\beta,\\boldsymbol\\theta) =\\int \\left\\{\\prod_i p(\\mathbf y_i\\mid\\mathbf b_i,\\boldsymbol\\beta)\\right\\} p(\\mathbf b\\mid\\boldsymbol\\theta)\\,d\\mathbf b.\\]\n\nMethods include Laplace approximation, adaptive Gaussian quadrature, and simulation-based integration. Their accuracy depends on the shape of the integrand and the available information per random effect. Binary outcomes with small cluster sizes can provide particularly limited information about group-specific intercepts. Penalized quasi-likelihood is another historical approach, but its approximations can produce appreciable bias in some binary-data settings.\n\nNonlinearity creates an important distinction between conditional and population-average effects. In a logistic mixed model,\n\n\\[E\\!\\left[ \\operatorname{logit}^{-1}(X\\boldsymbol\\beta+Z\\mathbf b) \\right] \\ne \\operatorname{logit}^{-1}(X\\boldsymbol\\beta)\\]\n\nin general. Integrating over random effects does not preserve the same logistic regression equation. Consequently, the exponentiated coefficient in a random-intercept logistic model is a conditional odds ratio, not generally the population-average odds ratio. Population-average probabilities require integration over the random-effects distribution, with any further averaging over predictors defined explicitly.\n\nGeneralized estimating equations provide a related but distinct approach. They specify a marginal mean and a working association structure rather than necessarily defining a full latent-variable likelihood. Under suitable conditions, robust variance estimation protects inference against misspecification of that working association. Their regression coefficients and mixed-model coefficients can therefore target different quantities, particularly with nonlinear links; differences between them are not automatically evidence that one method is incorrect.\n\nA nonlinear mixed-effects model places random effects inside a genuinely nonlinear response function. For example, a simplified concentration curve may take the form\n\n\\[C_i(t)=A_i\\exp(-k_it)+\\varepsilon_i(t),\\]\n\nwith positive parameters represented through \\(\\log A_i=\\alpha_A+b_{Ai}\\) and \\(\\log k_i=\\alpha_k+b_{ki}\\). Such models are widely used in pharmacokinetics and growth analysis. Because averaging and nonlinear transformation do not commute, a curve evaluated at average parameters is generally not the average curve across individuals. Estimation, interpretation, and prediction must preserve that distinction."
    },
    {
      "heading": "Bayesian hierarchical formulation",
      "text": "A Bayesian mixed-effects model specifies distributions for unknown population coefficients and covariance parameters as well as for group effects. A basic hierarchy is\n\n\\[\\mathbf y\\mid\\boldsymbol\\beta,\\mathbf b,\\boldsymbol\\theta \\sim p(\\mathbf y\\mid\\boldsymbol\\beta,\\mathbf b,\\boldsymbol\\theta),\\]\n\n\\[\\mathbf b\\mid\\boldsymbol\\theta\\sim p(\\mathbf b\\mid\\boldsymbol\\theta), \\qquad (\\boldsymbol\\beta,\\boldsymbol\\theta)\\sim p(\\boldsymbol\\beta,\\boldsymbol\\theta).\\]\n\nThe resulting posterior distribution combines the likelihood with the prior distribution. This formulation makes uncertainty in the covariance parameters part of the inferential calculation, rather than treating estimated covariance parameters as fixed during later prediction.\n\nThe distribution of group effects supplies the same basic pooling mechanism as in frequentist mixed models. For a Gaussian random-intercept model with known parameters, the posterior mean of a group effect has the familiar shrinkage form. Fully Bayesian analysis additionally averages over uncertainty in the population mean and variances. Empirical Bayes methods instead estimate hyperparameters from the observations and use those estimates in conditional calculations. They are closely connected to empirical BLUPs, although plug-in intervals need not capture all hyperparameter uncertainty.\n\nPriors matter most where the data contain little information. With only a few groups, the between-group standard deviation may remain weakly determined even when each group has many observations. Priors are often specified separately for random-effect standard deviations and their correlation matrix. This separation makes it possible to express different assumptions about plausible scales and correlations. A numerically broad prior is not automatically weak in its implications for outcomes: a wide prior on a logistic-scale coefficient, for example, can imply probabilities concentrated near zero and one.\n\nPosterior computation commonly uses Markov chain Monte Carlo, including Hamiltonian methods. Parameterization can strongly affect efficiency. A centered representation samples \\(\\mathbf b\\) directly from its hierarchical distribution; a noncentered representation writes\n\n\\[\\mathbf b=L\\mathbf z,\\qquad \\mathbf z\\sim N(\\mathbf 0,I),\\]\n\nwhere \\(LL^\\top=G\\). Noncentering often improves sampling when group effects are weakly informed, while centering can work better when each group is strongly informed. Neither representation changes the intended probability model when implemented equivalently.\n\nBayesian reporting distinguishes uncertainty about parameters from uncertainty about future observations. A credible interval describes posterior uncertainty conditional on the model and data. A posterior predictive distribution additionally generates future group effects or residuals as required by the prediction target. Posterior predictive checks compare observed summaries with replicated data, such as within-group variation, extreme group means, or frequencies of zero counts. Agreement with selected checks supports those model features but does not establish that every assumption is correct."
    },
    {
      "heading": "Assumptions, diagnostics, and model adequacy",
      "text": "The assumptions of a mixed model concern several distinct components: the mean relationship, the random-effects distribution, the residual distribution, dependence remaining after conditioning, and the relation between predictors and latent group effects. Normality of the raw outcome is not a general mixed-model requirement. A Gaussian linear mixed model assumes Gaussian conditional residuals and random effects, whereas a binary mixed model assigns a discrete conditional distribution. Confusing the marginal outcome distribution with the model’s conditional distributions can lead to inappropriate diagnostics.\n\nFor a linear mixed model, residual plots can reveal nonlinear mean patterns, unequal residual variances, or outlying observations. Residuals ordered by time can reveal dependence not captured by random intercepts and slopes. A random intercept induces a common covariance within a group, but it does not generally reproduce a correlation that decays with time separation. An autoregressive residual structure may therefore be needed alongside random effects. Conversely, adding a complicated residual structure cannot substitute for a substantively missing mean relationship.\n\nChecking the random-effects distribution is difficult because the group effects are latent. Their fitted predictions are shrunken, have unequal precision, and are not independent observations from the assumed distribution. A normal quantile plot of predicted random effects therefore does not have the same interpretation as a normal quantile plot of directly observed independent measurements. Simulation-based comparisons can reproduce the estimation and shrinkage process, allowing the observed diagnostic to be compared with what the fitted model would generate.\n\nA singular fit means that the estimated random-effects covariance matrix has less than full rank. It can arise through a zero variance or through a linear dependence among random coefficients, sometimes appearing as a correlation near \\(1\\) or \\(-1\\). Such a fit may reflect a genuinely negligible component, insufficient replication, an overly elaborate random-effects structure, or poor scaling. Numerical nonconvergence is a separate issue: an optimizer may fail to locate a suitable solution even when the model is identifiable, while a singular boundary solution may be found successfully.\n\nOther difficulties concern how observations enter the dataset. Under an appropriate missing-at-random assumption and correctly specified observed-data likelihood, likelihood methods can use incomplete response records without filling in every missing value. This does not make mixed models immune to missing-data bias. Missingness depending on unobserved outcomes may require a joint or sensitivity model, while missing predictors need separate treatment. Informative cluster size can also matter: if group size depends on latent outcomes, an analysis that ignores this sampling relationship may target a different population quantity than intended."
    },
    {
      "heading": "Study design, applications, and predictive assessment",
      "text": "Mixed-effects models are closely connected to experimental design because the randomization and measurement structure determine which effects can be distinguished. In a cluster-randomized study, treatment is assigned to groups such as schools rather than to individual pupils. Pupils within a school do not provide independent treatment assignments. A random school intercept can represent their outcome dependence, but the number of randomized schools remains central to treatment-effect precision and to the reliability of inferential approximations.\n\nFor equal cluster size \\(m\\) and an exchangeable within-cluster correlation \\(\\rho\\), the variance inflation relative to independent observations is often summarized by\n\n\\[D=1+(m-1)\\rho.\\]\n\nFor example, \\(m=20\\) and \\(\\rho=0.05\\) give \\(D=1.95\\). Under the assumptions behind this calculation, 200 clustered observations provide roughly the mean-estimation information of \\(200/1.95\\), or about 103 independent observations. This is a design approximation, not a universal formula for mixed-model information. Unequal cluster sizes, treatment allocation, covariate adjustment, and random slopes can materially change the calculation.\n\nIn longitudinal data analysis, mixed models represent individual trajectories and accommodate unequal numbers or timings of measurements. A study with several visits per person may include random intercepts, random time slopes, and a fixed treatment-by-time interaction. The interaction describes how the average time relationship differs by treatment; the random slope describes variation among individuals around their group’s average relationship. Nonlinear time effects, such as splines or piecewise slopes, can be added when a straight trajectory is inadequate.\n\nOther applications emphasize different sources of replication. Animal breeding uses relatedness matrices to model covariance among genetic effects. Multi-environment agricultural trials distinguish genotype differences, environment differences, and their interaction. Language experiments often require crossed participant and stimulus effects. In meta-analysis, a random-effects model represents study estimates as noisy measurements of study-specific true effects distributed around an overall mean. A mixed-effects meta-regression adds study-level predictors, with remaining heterogeneity represented by a variance component.\n\nPredictive evaluation must match the intended use. Randomly splitting individual records can leave observations from the same group in both training and test data. That evaluates prediction partly informed by known groups, not necessarily performance for entirely new groups. Leaving complete groups out in cross-validation addresses a different and often harder task. In repeated-measures forecasting, time order also matters: estimating a subject’s random effect using later observations can leak information into predictions of earlier outcomes. Model comparison is meaningful only when the validation design represents the information that will actually be available at prediction time."
    },
    {
      "heading": "Historical development and computational practice",
      "text": "Mixed-effects modeling developed from several traditions rather than from a single invention. Early twentieth-century analysis of variance, particularly the work of Ronald A. Fisher, provided methods for separating sources of experimental variation. Agricultural experiments and breeding programs required distinctions among treatment effects, environmental variation, and persistent differences among animals or plots. Balanced designs permitted simple calculations from mean squares, but irregular records and unequal group sizes demanded more general methods.\n\nCharles R. Henderson’s work in animal breeding during the mid-twentieth century was especially influential in the development of variance-component estimation and prediction of random effects. His mixed-model equations can be written, when the relevant inverses exist, as\n\n\\[\\begin{pmatrix} X^\\top R^{-1}X & X^\\top R^{-1}Z\\\\ Z^\\top R^{-1}X & Z^\\top R^{-1}Z+G^{-1} \\end{pmatrix} \\begin{pmatrix} \\widehat{\\boldsymbol\\beta}\\\\ \\widehat{\\mathbf b} \\end{pmatrix} = \\begin{pmatrix} X^\\top R^{-1}\\mathbf y\\\\ Z^\\top R^{-1}\\mathbf y \\end{pmatrix}.\\]\n\nFor known covariance parameters, these equations jointly yield generalized least-squares estimates of fixed effects and linear unbiased predictions of random effects. Their block structure also connects statistical prediction with penalized least squares.\n\nH. D. Patterson and Robin Thompson introduced the restricted-likelihood approach for interblock information recovery in 1971, establishing a major foundation for modern REML estimation. The expectation–maximization algorithm presented by Arthur Dempster, Nan Laird, and Donald Rubin in 1977 supplied a general framework useful for latent-variable estimation. Laird and James Ware’s 1982 paper on random-effects models for longitudinal data helped establish the modern formulation for irregular repeated measurements. Parallel work in education and the social sciences developed the language of multilevel and hierarchical models.\n\nModern implementations depend heavily on sparse matrix methods. A random-effects design matrix often contains many zeros because each observation belongs to only a few groups. Sparse factorization can exploit this pattern rather than form a dense \\(n\\times n\\) marginal covariance matrix. This is particularly valuable for crossed effects and large numbers of groups. Computational cost is nevertheless influenced by the pattern of connections and the fill-in created during factorization, not merely by the number of observations.\n\nSoftware environments provide different parameterizations, approximations, and inferential defaults. Widely used systems include SAS procedures for mixed and generalized mixed models, the R packages nlme and lme4, and Bayesian frameworks such as Stan. Formulas such as (1 + time | subject) usually request a correlated random intercept and time slope, but details must be interpreted within the particular implementation. Comparisons across programs require alignment of maximum likelihood versus REML, covariance constraints, integration methods, treatment of residual dependence, and prediction targets; matching the written model formula alone does not ensure identical statistical calculations."
    }
  ],
  "links": [
    "probability-distribution",
    "linear-regression",
    "analysis-of-variance",
    "bayesian-inference",
    "matrix",
    "covariance-matrix",
    "covariance",
    "random-sampling",
    "econometrics",
    "endogeneity",
    "causal-inference",
    "likelihood-function",
    "maximum-likelihood-estimation",
    "cholesky-decomposition",
    "expectation-maximization-algorithm",
    "conditional-expectation",
    "regularization",
    "ridge-regression",
    "prediction-interval",
    "standard-error",
    "degrees-of-freedom",
    "chi-squared-distribution",
    "bootstrap-sampling",
    "confidence-interval",
    "statistical-hypothesis-testing",
    "generalized-linear-mixed-model",
    "generalized-linear-model",
    "odds-ratio",
    "posterior-distribution",
    "prior-distribution",
    "markov-chain-monte-carlo",
    "credible-interval",
    "posterior-predictive-distribution",
    "posterior-predictive-checking",
    "autoregressive-model",
    "experimental-design",
    "statistical-interaction",
    "meta-analysis",
    "cross-validation",
    "ronald-fisher",
    "sparse-matrix"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}