{
  "id": "posterior-predictive-distribution",
  "title": "Posterior Predictive Distribution",
  "lang": "en",
  "source_lang": "en",
  "summary": "A posterior predictive distribution describes unobserved outcomes by averaging their conditional distributions over Bayesian posterior uncertainty about model parameters.",
  "category": "mathematics",
  "subcategory": "probability-statistics",
  "type": "concept",
  "aliases": [
    "Posterior predictive",
    "Bayesian posterior predictive distribution",
    "Posterior predictive density"
  ],
  "generated_at": "2026-10-04",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/posterior-predictive-distribution/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/posterior-predictive-distribution/",
      "json": "https://aiwiki.page/zh/posterior-predictive-distribution.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "The posterior predictive distribution is the probability distribution of unobserved outcomes conditional on observed data in Bayesian inference. It combines uncertainty about model parameters with the variability of outcomes at any given parameter value. Unlike a posterior distribution, which concerns unknown parameters, it concerns observable quantities: future measurements, missing observations, or hypothetical replications of an experiment. It therefore provides a distribution of possible outcomes rather than only a point prediction. (mc-stan.org)"
    },
    {
      "heading": "Mathematical definition",
      "text": "Let \\(y\\) denote observed data, \\(\\theta\\) the model parameters, and \\(\\tilde y\\) an unobserved outcome. A Bayesian model specifies a prior distribution \\(p(\\theta)\\) and a likelihood function \\(p(y\\mid\\theta)\\). Through Bayes’ theorem, these determine \\(p(\\theta\\mid y)\\). The predictive distribution is\n\n\\[p(\\tilde y\\mid y) =\\int p(\\tilde y\\mid\\theta,y)\\, p(\\theta\\mid y)\\,d\\theta.\\]\n\nIf \\(\\tilde y\\) and \\(y\\) have conditional independence given \\(\\theta\\), this becomes\n\n\\[p(\\tilde y\\mid y) =\\int p(\\tilde y\\mid\\theta)\\, p(\\theta\\mid y)\\,d\\theta.\\]\n\nThe integration removes the parameters from the joint distribution of parameters and predictions. Discrete parameters require summation instead. The notation \\(p\\) may represent a probability density or a probability mass, depending on the outcome. The resulting distribution is a posterior-weighted mixture of conditional outcome distributions. (statproofbook.github.io)\n\nFor regression with observed predictors \\(x\\) and new predictors \\(\\tilde x\\), the corresponding expression is\n\n\\[p(\\tilde y\\mid\\tilde x,x,y) =\\int p(\\tilde y\\mid\\tilde x,\\theta) p(\\theta\\mid x,y)\\,d\\theta.\\]\n\nThus predictions depend on both the fitted model and the conditions under which the new outcome is generated. (mc-stan.org)"
    },
    {
      "heading": "Sources of predictive uncertainty",
      "text": "Predictive uncertainty includes both parameter uncertainty and conditional outcome variability. When the relevant moments exist, the predictive expected value satisfies\n\n\\[\\mathbb E[\\tilde Y\\mid y] =\\mathbb E_{\\theta\\mid y} \\!\\left[\\mathbb E(\\tilde Y\\mid\\theta,y)\\right],\\]\n\nand its variance decomposes as\n\n\\[\\operatorname{Var}(\\tilde Y\\mid y) = \\mathbb E_{\\theta\\mid y} [\\operatorname{Var}(\\tilde Y\\mid\\theta,y)] + \\operatorname{Var}_{\\theta\\mid y} [\\mathbb E(\\tilde Y\\mid\\theta,y)].\\]\n\nThe first term averages variability within the observation model; the second measures variation in conditional means across plausible parameter values. These are applications of conditional expectation and total variance. (sites.stat.columbia.edu)\n\nA plug-in prediction \\(p(\\tilde y\\mid\\hat\\theta)\\), using a maximum-likelihood estimate or another point estimate, does not perform this averaging. In particular, uncertainty about a conditional mean is not the same as uncertainty about an actual future observation. Even precisely known parameters can imply substantial outcome variability. (mc-stan.org)"
    },
    {
      "heading": "Conjugate examples",
      "text": "Suppose \\(s\\) successes are observed in \\(n\\) trials with a common success probability \\(\\theta\\). A beta distribution is a conjugate prior for the binomial model:\n\n\\[\\theta\\sim\\operatorname{Beta}(\\alpha,\\beta), \\qquad \\theta\\mid y\\sim \\operatorname{Beta}(\\alpha+s,\\beta+n-s).\\]\n\nFor one additional trial, the predictive Bernoulli distribution has success probability\n\n\\[\\Pr(\\tilde Y=1\\mid y) =\\frac{\\alpha+s}{\\alpha+\\beta+n}.\\]\n\nFor \\(m\\) additional trials, integrating their binomial distribution over this posterior produces a beta-binomial distribution. A single shared parameter draw governs the entire future batch; drawing a separate parameter independently for each trial would define a different joint predictive model. (statproofbook.github.io)\n\nFor a continuous example, suppose observations follow a normal distribution with unknown mean \\(\\mu\\) and known variance \\(\\sigma^2\\). If\n\n\\[\\mu\\mid y\\sim N(m_n,v_n),\\]\n\nthen a new observation has distribution\n\n\\[\\tilde Y\\mid y\\sim N(m_n,\\sigma^2+v_n).\\]\n\nIts variance explicitly combines observation variability and posterior uncertainty about the mean. Consequently, a credible interval for \\(\\mu\\) differs from a prediction interval for \\(\\tilde Y\\). (tensorflow.org)"
    },
    {
      "heading": "Computation and summaries",
      "text": "When analytical integration is unavailable, the Monte Carlo method provides a practical approximation. Parameter draws, often obtained through Markov chain Monte Carlo, are followed by simulated outcomes:\n\n\\[\\theta^{(r)}\\sim p(\\theta\\mid y), \\qquad \\tilde y^{(r)}\\sim p(\\tilde y\\mid\\theta^{(r)},y).\\]\n\nThe second step is essential: retaining only conditional means omits outcome variability. Predictive samples support estimates of quantiles, event probabilities, and other summaries. (mc-stan.org)\n\nAlternatively, the predictive density at a specified outcome can be estimated by averaging conditional densities:\n\n\\[p(\\tilde y\\mid y) \\approx\\frac1R\\sum_{r=1}^{R} p(\\tilde y\\mid\\theta^{(r)},y).\\]\n\nDensity evaluation and outcome simulation are related but distinct operations. For very small densities, stable log-sum-exp calculations help prevent numerical underflow. (mc-stan.org)"
    },
    {
      "heading": "Model checking and predictive evaluation",
      "text": "In posterior predictive checking, replicated datasets generated from the fitted model are compared with observed data. Comparisons may examine dispersion, extremes, proportions of zeros, or other features. Such checks assess whether the model can reproduce selected aspects of the observations; they are not themselves tests of performance on genuinely unseen data. (mc-stan.org)\n\nOut-of-sample evaluation instead uses withheld observations, for example through cross-validation. The relevant predictive distribution conditions on the training subset rather than the full dataset. Predictive means minimize expected squared error, whereas other losses can favor different summaries. Prediction intervals describe uncertainty conditional on the specified model and data; their practical adequacy still depends on the observation model and the prediction setting. (mc-stan.org)"
    }
  ],
  "links": [
    "probability-distribution",
    "bayesian-inference",
    "posterior-distribution",
    "prior-distribution",
    "likelihood-function",
    "bayes-theorem",
    "conditional-independence",
    "integral",
    "joint-probability-distribution",
    "probability-density-function",
    "probability-mass-function",
    "expected-value",
    "variance",
    "maximum-likelihood-estimation",
    "beta-distribution",
    "conjugate-prior",
    "binomial-distribution",
    "bernoulli-distribution",
    "normal-distribution",
    "credible-interval",
    "prediction-interval",
    "monte-carlo-method",
    "markov-chain-monte-carlo",
    "posterior-predictive-checking",
    "cross-validation"
  ],
  "sources": [
    {
      "title": "Stan User’s Guide",
      "url": "https://mc-stan.org/docs/2_31/stan-users-guide-2_31.pdf"
    },
    {
      "title": "2 Computing the posterior predictive distribution",
      "url": "https://mc-stan.org/docs/2_29/stan-users-guide/computing-the-posterior-predictive-distribution.html"
    },
    {
      "title": "3 Sampling from the posterior predictive distribution",
      "url": "https://mc-stan.org/docs/2_31/stan-users-guide/sampling-from-the-posterior-predictive-distribution.html"
    },
    {
      "title": "Posterior Predictive Sampling",
      "url": "https://mc-stan.org/docs/2_38/stan-users-guide/posterior-prediction.html"
    },
    {
      "title": "Posterior predictive distribution — The Book of Statistical Proofs",
      "url": "https://statproofbook.github.io/D/post-pred.html"
    },
    {
      "title": "Posterior distribution for binomial observations",
      "url": "https://statproofbook.github.io/P/bin-post.html"
    },
    {
      "title": "Bayesian Data Analysis, third edition",
      "url": "https://sites.stat.columbia.edu/gelman/book/BDA3.pdf"
    },
    {
      "title": "A conservation law for posterior predictive variance",
      "url": "https://arxiv.org/abs/2406.11806"
    },
    {
      "title": "STAT415 Handouts — Beta-Binomial Model",
      "url": "https://bookdown.org/kevin_davisross/stat415-handouts/beta-binomial.html"
    },
    {
      "title": "Posterior Predictive Distribution for Beta-Binomial model",
      "url": "https://www.cs.ubc.ca/~murphyk/pmtk/pmtk1/doc/examples/binom_betaPostPredDemo.html"
    },
    {
      "title": "tfp.distributions.normal_conjugates_known_scale_predictive",
      "url": "https://www.tensorflow.org/probability/api_docs/python/tfp/distributions/normal_conjugates_known_scale_predictive"
    }
  ],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}