A hierarchical Bayesian model is a statistical model in which observations, unknown parameters, and parameters governing other parameters are organized into interconnected levels of probability distributions. It extends Bayesian inference by allowing related quantities—such as effects for different schools, regions, or experiments—to share a common probabilistic structure. This structure enables partial pooling: estimates for individual groups draw on information from other groups without requiring every group to have the same underlying value.
Hierarchical modeling is especially useful when data are grouped, repeated, spatially structured, or collected from multiple related sources. A hierarchy need not correspond to a literal organizational hierarchy; it may instead represent shared variation, latent processes, or successive levels of uncertainty.
Mathematical structure
Consider observations , where indexes groups and indexes observations within group . A basic hierarchical model has three levels:
The components have distinct roles:
- Observation model: describes how data arise within a group.
- Group-level distribution: describes variation among group parameters.
- Hyperprior: assigns a prior distribution to the shared parameters, or hyperparameters, .
For example, might be a school-specific mean, while contains the mean and spread of the distribution of school effects.
Under the assumed conditional independence structure, the joint distribution factors as
Applying Bayes’ theorem gives the posterior distribution:
Inference concerns this joint posterior or distributions obtained by integrating out unwanted quantities. For example,
Consequently, uncertainty about the population-level structure contributes to uncertainty about each group.
The hierarchy can be represented as a probabilistic graphical model. Although group parameters may be independent conditional on shared hyperparameters, they are generally dependent after those hyperparameters are integrated out. This dependence is how information passes between groups.
Exchangeability and information sharing
A common justification for a shared group-level distribution is exchangeability: before observing the outcomes, relabeling the groups does not change the joint distribution assigned to their parameters. Exchangeability is weaker than unconditional statistical independence and does not imply that groups are identical.
For example, school effects might be modeled as exchangeable after accounting for relevant school characteristics. If groups differ systematically, their distribution can depend on predictors:
Here contains group-level predictors, and the remaining deviations are modeled as exchangeable around the regression prediction.
Exchangeability is a modeling assumption, not an automatic consequence of having grouped data. Combining unrelated groups under an unsuitable common distribution can produce misleading information sharing.
Partial pooling and shrinkage
Hierarchical models occupy a middle position between two limiting approaches:
- Complete pooling: all groups share a common parameter, .
- No pooling: each group is analyzed separately, without learning about its parameter from the other groups.
- Partial pooling: group parameters may differ, but their distribution is learned jointly.
Partial pooling often produces shrinkage: a group estimate moves toward a shared mean or a predictor-dependent mean. The amount of movement depends on both the uncertainty within that group and the estimated variation between groups.
Normal–normal example
Suppose the estimated mean in group , denoted , has known sampling variance :
Here denotes the normal distribution, is the population mean, and measures between-group variation.
Conditional on and , the posterior mean is
where
The corresponding posterior variance is
These expressions show that:
- Precise group estimates, with small , receive relatively little shrinkage.
- Imprecise group estimates receive more shrinkage.
- Small between-group variation produces stronger pooling.
- Large between-group variation produces weaker pooling.
The formula is conditional on known hyperparameters. In a fully Bayesian analysis, and also have posterior distributions, and inference averages over their uncertainty.
Shrinkage is related to regularization: it discourages unstable estimates unsupported by sufficient evidence. It can improve estimation under appropriate loss criteria, but it does not guarantee greater accuracy for every individual group.
Common model families
Varying-intercept and varying-slope models
A hierarchical extension of linear regression allows both intercepts and slopes to vary:
The group coefficients may follow a multivariate normal distribution:
The covariance matrix describes variation in coefficients and their association across groups. For example, groups with higher intercepts may tend to have steeper or shallower slopes.
Hierarchical generalized linear models
The same structure extends to a generalized linear model for non-normal observations. Examples include hierarchical logistic regression for binary outcomes and models using a Poisson distribution for counts.
A binary-outcome model might use
with a shared distribution for the group intercepts .
Nested and crossed structures
In a nested model, observations may lie within classrooms, classrooms within schools, and schools within districts. Each level can contribute its own variation.
In a crossed model, grouping factors overlap rather than nest. For example, performance measurements may depend on both the participant and the task, with each participant completing multiple tasks. Hierarchical modeling accommodates both structures; a simple tree is not required.
Hierarchical models for study effects
In meta-analysis, study-specific estimates can be modeled as noisy observations of underlying study effects, which themselves follow a population distribution. This separates within-study sampling uncertainty from between-study heterogeneity.
Similar formulations occur in regional estimation, repeated-measures analysis, and models of related experiments.
Prior specification
Priors are required for unknown quantities at each relevant level. Important choices include priors for population means, regression coefficients, between-group scales, and correlation structures.
Scale parameters such as require particular care because they determine the strength of pooling. Common choices include half-normal and half-Student- distributions. Their scales must be interpreted relative to the measurement units and plausible variation in the application.
A prior described as “vague” is not necessarily harmless. Very broad scale priors can permit implausibly large group differences, while priors concentrated near zero can impose excessive pooling. Improper priors may also lead to an improper posterior in some hierarchical models.
Prior predictive simulation—generating parameters and observations from the model before conditioning on observed outcomes—helps reveal whether the implied data are plausible. Sensitivity analysis examines how substantive inferences change under reasonable alternative priors.
Fully Bayesian and empirical Bayes approaches
In a fully Bayesian analysis, hyperparameters receive priors and are inferred jointly with group parameters. Predictions and uncertainty summaries integrate over their posterior uncertainty.
In an empirical Bayes analysis, hyperparameters are estimated from the data, often by maximizing the marginal likelihood:
The resulting estimate is then used in group-level inference.
Both approaches can produce partial pooling. The important distinction is the treatment of uncertainty in the shared distribution. Plug-in empirical Bayes calculations typically omit some of that uncertainty unless additional adjustments are made. The difference can matter especially when there are few groups.
Hierarchical Bayesian models also overlap with classical mixed-effects models. Their observation models may be identical, while parameter treatment, estimation procedures, and uncertainty interpretation differ.
Computation
Some hierarchical models permit analytic calculations using a conjugate prior, but complex models usually require numerical methods.
Markov chain Monte Carlo methods approximate the joint posterior using simulated draws. Gibbs sampling is useful when conditional distributions are easy to sample. Hamiltonian Monte Carlo is widely used for continuous, differentiable models.
Variational inference provides another approach by fitting a tractable approximation to the posterior. It can be computationally attractive for large models, but approximation quality requires assessment, particularly for uncertainty estimates and posterior dependence.
Centered and non-centered parameterizations
The model
can be written equivalently as
The first is a centered parameterization; the second is non-centered. They define the same probability model but can have very different computational behavior.
When group-level data are weak, a non-centered formulation often reduces difficult posterior dependence between and . When group parameters are strongly informed by observations, a centered formulation may work better. Neither is universally superior.
Convergence diagnostics, effective sample sizes, and sampler-specific diagnostics remain necessary; a mathematically coherent hierarchy does not ensure reliable computation.
Prediction and model checking
Prediction differs according to whether the target belongs to an observed group or a new group.
For a future observation in an existing group,
For an observation in a new group, a new group parameter must also be generated:
The posterior predictive distribution therefore incorporates uncertainty about outcomes, group effects, and shared parameters. A new group typically introduces uncertainty that is absent when predicting for a well-observed existing group.
Posterior predictive checking compares observed features with those of data generated from the fitted model. Relevant features include within-group dispersion, between-group variation, extreme groups, and systematic residual patterns.
Cross-validation must match the prediction task. Holding out individual observations largely evaluates prediction within observed groups; holding out entire groups evaluates generalization to new groups. These are distinct tasks and can yield different assessments of predictive performance.
Historical development
Hierarchical Bayesian modeling grew from work on Bayesian inference, random-effects models, and the estimation of many related quantities. Mid-twentieth-century research on empirical Bayes methods and shrinkage clarified the potential benefits of combining group-specific evidence with information from a population.
The expansion of practical simulation methods, especially Markov chain Monte Carlo in the late twentieth century, made substantially more complex hierarchies computationally feasible. Later developments in probabilistic programming, automatic differentiation, and gradient-based sampling broadened their routine use.
Limitations and interpretation
Limited information about heterogeneity. Many observations within a few groups can precisely estimate those groups while still providing limited evidence about the distribution of group effects. Between-group variation depends strongly on the number and diversity of groups, not merely the total number of observations.
Inappropriate pooling. A shared distribution can obscure meaningful subpopulations, outliers, or structural differences. Alternative group-level distributions, explicit predictors, or separate hierarchies may be needed.
Identification problems. Complex hierarchies can contain parameters whose separate contributions are difficult to determine. Statistical identification depends on both the model structure and the available information; a proper prior does not by itself show that the data distinguish competing explanations.
Model dependence. Posterior estimates are conditional on the specified observation model, hierarchy, and priors. A narrow credible interval can still be misleading if important model assumptions are wrong.
Selection and missingness. Hierarchical structure does not automatically correct selection bias, unmodeled missing-data mechanisms, or unrepresentative groups.
Association versus causation. Group-specific effects are not automatically causal effects. Causal inference requires additional assumptions about assignment, confounding, and the data-generating process; adding hierarchical levels does not establish those assumptions.