{
  "id": "conjugate-prior",
  "title": "Conjugate Prior",
  "lang": "en",
  "source_lang": "en",
  "summary": "A conjugate prior is a prior distribution whose family is preserved when it is updated by a specified likelihood.",
  "category": "mathematics",
  "subcategory": "probability-statistics",
  "type": "concept",
  "aliases": [
    "Conjugate prior distribution"
  ],
  "generated_at": "2026-10-04",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/conjugate-prior/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/conjugate-prior/",
      "json": "https://aiwiki.page/zh/conjugate-prior.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "A conjugate prior is a prior distribution that, under a specified statistical model, produces a posterior distribution belonging to the same family of probability distributions. In Bayesian inference, this property makes updating especially convenient: observations change the distribution’s parameters without changing its functional form. Conjugacy is a relationship between a prior family and a likelihood function, not an intrinsic property of a distribution considered alone. (math.mit.edu)"
    },
    {
      "heading": "Definition and mathematical structure",
      "text": "Let \\(x\\) denote observed data, \\(\\theta\\) an unknown parameter, and \\(\\pi(\\theta\\mid\\lambda)\\) a prior density indexed by hyperparameters \\(\\lambda\\). By Bayes’ theorem,\n\n\\[\\pi(\\theta\\mid x,\\lambda) = \\frac{p(x\\mid\\theta)\\pi(\\theta\\mid\\lambda)} {\\int p(x\\mid\\vartheta)\\pi(\\vartheta\\mid\\lambda)\\,d\\vartheta}.\\]\n\nThe denominator is the marginal likelihood. The prior family is conjugate if every admissible update can be written as \\(\\pi(\\theta\\mid\\lambda')\\), where \\(\\lambda'\\) depends on the observations and original hyperparameters. Thus, posterior calculation often reduces to multiplying kernels and recognizing a familiar probability density function. The prior and posterior need not have the same parameter values, and neither must belong to the distribution family used for the observations. (stat210a.berkeley.edu)"
    },
    {
      "heading": "Beta–binomial conjugacy",
      "text": "Suppose \\(S\\), the number of successes in \\(n\\) trials, follows a binomial distribution with unknown success probability \\(\\theta\\). Its likelihood is proportional to\n\n\\[\\theta^S(1-\\theta)^{n-S}.\\]\n\nA beta distribution with positive parameters \\(\\alpha,\\beta\\) has density proportional to \\(\\theta^{\\alpha-1}(1-\\theta)^{\\beta-1}\\). Multiplying these expressions gives\n\n\\[\\theta\\mid S \\sim\\operatorname{Beta}(\\alpha+S,\\beta+n-S).\\]\n\nThe same update applies to a sequence of conditionally independent Bernoulli observations. The prior hyperparameters are updated by adding successes and failures, respectively. (ocw.mit.edu)\n\nThe posterior mean is\n\n\\[\\mathbb E[\\theta\\mid S] =\\frac{\\alpha+S}{\\alpha+\\beta+n} =\\frac{\\alpha+\\beta}{\\alpha+\\beta+n} \\frac{\\alpha}{\\alpha+\\beta} +\\frac{n}{\\alpha+\\beta+n}\\frac{S}{n},\\]\n\nfor \\(n>0\\). This expresses the estimate as a weighted average of the prior mean and observed success proportion. In this mean-based interpretation, \\(\\alpha+\\beta\\) acts as an effective prior sample size. Hyperparameters need not be integers or represent actual previous observations. (stat210a.berkeley.edu)\n\nFor example, a \\(\\operatorname{Beta}(2,2)\\) prior and seven successes in ten trials give a \\(\\operatorname{Beta}(9,5)\\) posterior, with mean \\(9/14\\), approximately \\(0.643\\). This is a direct application of the update formula. (ocw.mit.edu)"
    },
    {
      "heading": "Other standard conjugate pairs",
      "text": "For conditionally independent counts \\(X_1,\\ldots,X_n\\) from a Poisson distribution with rate \\(\\lambda\\), a gamma prior is conjugate. Using shape \\(a>0\\) and rate \\(b>0\\), its kernel is \\(\\lambda^{a-1}e^{-b\\lambda}\\), and\n\n\\[\\lambda\\mid x \\sim\\operatorname{Gamma} \\left(a+\\sum_{i=1}^{n}x_i,\\ b+n\\right).\\]\n\nThe rate convention matters: formulations using a scale parameter require different-looking updates. (math.mit.edu)\n\nFor observations from a normal distribution with unknown mean \\(\\mu\\) and known variance \\(\\sigma^2\\), a normal prior \\(\\mu\\sim N(m_0,v_0)\\) yields\n\n\\[\\mu\\mid x\\sim N(m_n,v_n), \\qquad v_n^{-1}=v_0^{-1}+n\\sigma^{-2},\\]\n\n\\[m_n=v_n\\left(v_0^{-1}m_0+ \\sigma^{-2}\\sum_{i=1}^{n}x_i\\right).\\]\n\nPosterior precision—the reciprocal of variance—is the sum of prior and data precisions. When both mean and variance are unknown, standard joint conjugate families include the normal–inverse-gamma distribution, rather than an arbitrary pair of independent priors. (cs.ubc.ca)"
    },
    {
      "heading": "Connection with exponential families",
      "text": "A broad construction is available for exponential-family models. Write a sampling density as\n\n\\[p(x\\mid\\eta) =h(x)\\exp\\{\\eta^\\top T(x)-A(\\eta)\\},\\]\n\nwhere \\(\\eta\\) is the natural parameter, \\(T(x)\\) a statistic, and \\(A(\\eta)\\) the log-normalizing function. For independent observations, \\(\\sum_iT(x_i)\\) is a sufficient statistic. A conjugate prior density over \\(\\eta\\), relative to a specified reference measure, has kernel\n\n\\[\\pi(\\eta\\mid\\chi,\\nu) \\propto\\exp\\{\\eta^\\top\\chi-\\nu A(\\eta)\\}.\\]\n\nMultiplication by the likelihood gives the additive updates\n\n\\[\\chi'=\\chi+\\sum_iT(x_i), \\qquad \\nu'=\\nu+n.\\]\n\nThese formulas apply where the prior and posterior normalizing integrals are finite. They explain why many familiar conjugate updates accumulate counts, sums, or other low-dimensional statistics. However, the construction does not guarantee an elementary expression for the normalizing constant. (stat.berkeley.edu)"
    },
    {
      "heading": "Prediction and sequential updating",
      "text": "Conjugacy concerns uncertainty about parameters, whereas the posterior predictive distribution describes future observations:\n\n\\[p(\\widetilde x\\mid x) =\\int p(\\widetilde x\\mid\\theta) \\pi(\\theta\\mid x)\\,d\\theta.\\]\n\nIntegrating a binomial sampling distribution against a beta posterior produces a beta–binomial distribution. The predictive family therefore need not match either the posterior family or the original sampling family. (lancaster.ac.uk)\n\nWith observations independent conditional on the parameter, updating sequentially is equivalent to updating with the entire dataset at once. The posterior after one batch becomes the prior for the next; additive sufficient statistics make such updates compact. (ocw.mit.edu)"
    },
    {
      "heading": "Scope and limitations",
      "text": "Conjugate families are not unique: mixtures of conjugate priors can also remain closed under updating, with both component parameters and mixture weights changing. Their mathematical convenience does not establish that their shapes adequately represent substantive prior information. Conjugacy also does not guarantee computational tractability when normalizing constants are difficult to evaluate. (stat.berkeley.edu)\n\nIn multiparameter models, convenient full conditional distributions may permit Gibbs sampling, a Markov chain Monte Carlo method, even when directly sampling the joint posterior is difficult. Such conditional updating is distinct from obtaining a complete joint posterior through a single conjugate-family update. (sites.stat.columbia.edu)"
    }
  ],
  "links": [
    "prior-distribution",
    "posterior-distribution",
    "probability-distribution",
    "bayesian-inference",
    "likelihood-function",
    "hyperparameter",
    "bayes-theorem",
    "marginal-likelihood",
    "probability-density-function",
    "binomial-distribution",
    "beta-distribution",
    "bernoulli-distribution",
    "expected-value",
    "poisson-distribution",
    "normal-distribution",
    "variance",
    "sufficient-statistic",
    "posterior-predictive-distribution",
    "markov-chain-monte-carlo"
  ],
  "sources": [
    {
      "title": "math.mit.edu/~rmd/650/bayestopix.pdf",
      "url": "https://math.mit.edu/~rmd/650/bayestopix.pdf"
    },
    {
      "title": "stat210a.berkeley.edu/fall-2025/reader/bayes-estimation.html",
      "url": "https://stat210a.berkeley.edu/fall-2025/reader/bayes-estimation.html"
    },
    {
      "title": "ocw.mit.edu/courses/15-097-prediction-machine-learning-and-statistics-spring-20…",
      "url": "https://ocw.mit.edu/courses/15-097-prediction-machine-learning-and-statistics-spring-2012/553a0822984b08bc611306c93533a0a3_MIT15_097S12_lec15.pdf"
    },
    {
      "title": "cs.ubc.ca/~murphyk/Papers/bayesGauss.pdf",
      "url": "https://www.cs.ubc.ca/~murphyk/Papers/bayesGauss.pdf"
    },
    {
      "title": "stat.berkeley.edu/~bartlett/courses/241A-spring2007/handout_1_expfam.pdf",
      "url": "https://www.stat.berkeley.edu/~bartlett/courses/241A-spring2007/handout_1_expfam.pdf"
    },
    {
      "title": "lancaster.ac.uk/~prendivs/accessible/math331/lectures.tex/Ch3.S1.html",
      "url": "https://www.lancaster.ac.uk/~prendivs/accessible/math331/lectures.tex/Ch3.S1.html"
    },
    {
      "title": "sites.stat.columbia.edu/gelman/bayescomputation/bdachapters10and11.pdf",
      "url": "https://sites.stat.columbia.edu/gelman/bayescomputation/bdachapters10and11.pdf"
    }
  ],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}