A credible interval is an interval that contains a specified proportion of the posterior distribution of an unknown quantity in Bayesian inference. It summarizes uncertainty after observed data have been combined with a statistical model and a prior distribution. For example, a 95% credible interval assigns 95% posterior probability to the parameter lying between its endpoints, conditional on the data and assumptions. It is a Bayesian counterpart to a confidence interval, but the two have different interpretations. (web.stanford.edu)
Definition and mathematical basis
Let (\theta) denote an unknown scalar parameter and (y) the observed data. By Bayes’ theorem, its posterior density is
[ p(\theta\mid y) =\frac{p(y\mid\theta)p(\theta)}{p(y)}, ]
where (p(y\mid\theta)) is the likelihood function and (p(y)) is the marginal likelihood, which normalizes the distribution. The parameter is represented by a random variable to express uncertainty; this does not require the underlying physical quantity to fluctuate randomly. (web.stanford.edu)
For (0<\alpha<1), an interval ([L,U]) with credibility level (1-\alpha) satisfies
[ \Pr(L\leq\theta\leq U\mid y)=1-\alpha. ]
For a continuous posterior with probability density function (p(\theta\mid y)), this means
[ \int_L^U p(\theta\mid y),d\theta=1-\alpha. ]
The integral measures probability mass, not the height of the density at particular points. Intervals generally are not unique: different endpoint choices can enclose the same mass. For discrete parameters, exact equality may be impossible, so a credible set commonly contains at least the stated probability. (web.stanford.edu)
Principal constructions
Equal-tailed intervals
An equal-tailed interval excludes probability (\alpha/2) from each tail. If (F) is the posterior cumulative distribution function, its endpoints are the quantiles
[ L=F^{-1}(\alpha/2),\qquad U=F^{-1}(1-\alpha/2). ]
Thus, a 95% equal-tailed interval extends from the 2.5th to the 97.5th percentile. It is also called a central interval, although it need not be geometrically centered on the posterior mean or mode. Equal-tailed endpoints transform consistently under strictly monotonic transformations, with their order reversed for decreasing transformations. (stata.com)
Highest-posterior-density regions
A highest-posterior-density region consists of parameter values whose posterior density exceeds a threshold chosen to enclose the desired probability:
[ H_c={\theta:p(\theta\mid y)\geq c}. ]
For a continuous, unimodal posterior, this commonly forms an interval and minimizes length among intervals containing the same posterior mass. For a multimodal posterior, however, it can comprise disconnected pieces rather than one interval. Equal-tailed and highest-density intervals coincide for a normal distribution, but can differ substantially for skewed distributions. (stata.com)
Highest-density constructions depend on parameterization because probability densities change under nonlinear transformations. Consequently, transforming an HPD interval for a standard deviation by squaring its endpoints need not produce the HPD interval for the corresponding variance. Both constructions contain the specified probability mass; their difference concerns which values are included, not how much probability they contain. (statmodeling.stat.columbia.edu)
Example: an unknown success probability
Suppose 12 independent Bernoulli trials produce 11 successes and one failure. Let their common success probability be (\theta), with a beta distribution prior,
[ \theta\sim\operatorname{Beta}(7,3). ]
The beta family is a conjugate prior for this likelihood, giving
[ \theta\mid y\sim\operatorname{Beta}(18,4). ]
The posterior’s 10th and 90th percentiles are approximately 0.7090 and 0.9142. Therefore, ([0.7090,0.9142]) is an 80% equal-tailed credible interval. Conditional on the specified prior and sampling model, the posterior probability that (\theta) lies within these bounds is 0.80. The interval describes uncertainty about the common success probability, not a range containing 80% of future binary outcomes. (web.stanford.edu)
Contrast with confidence and prediction intervals
A frequentist confidence interval is defined through the sampling distribution of an interval-producing procedure. A 95% confidence procedure covers a fixed true parameter in 95% of repeated samples under its assumptions. Its defining probability concerns the random intervals before observation, rather than a posterior probability assigned to the parameter after observation. (itl.nist.gov)
A credible interval instead conditions on the observed data. Its posterior credibility does not by itself guarantee the same repeated-sampling coverage at every fixed parameter value. Bayesian and frequentist endpoints may nevertheless agree in particular models; identical numerical bounds do not make their interpretations identical. (web.stanford.edu)
A prediction interval addresses an unobserved or future outcome. In Bayesian analysis, it is obtained from the posterior predictive distribution,
[ p(\widetilde y\mid y) =\int p(\widetilde y\mid\theta)p(\theta\mid y),d\theta. ]
This combines parameter uncertainty with variability in outcomes conditional on the parameter. It therefore answers a different question from an interval for the parameter itself. (mc-stan.org)
Computation and model dependence
When posterior quantiles are available analytically or numerically, equal-tailed intervals can be calculated directly. Otherwise, Markov chain Monte Carlo draws provide empirical estimates of posterior quantiles. These endpoints have computational uncertainty: correlated draws reduce the effective information available, and inadequate exploration of the posterior can make tail estimates unreliable. Quantile-specific Monte Carlo error and effective sample size distinguish numerical accuracy from the uncertainty represented by the interval itself. (web.stanford.edu)
Credible intervals depend on the prior, likelihood, and conditioning assumptions. Prior influence can be substantial when observations provide limited information. A narrow interval does not establish that the model captures relevant features of the data; posterior predictive checking examines that separate issue by comparing observed data with model-generated replications. (web.stanford.edu)
References
- STATS 200: Introduction to Statistical Inference, Lecture 20web.stanford.edu
- Chapter 8. Statistical Inference: Credible Intervalsweb.stanford.edu
- BAYES] Bayesian Analysisstata.com
- Bayesian Inference 2019vioshyvo.github.io
- 1.4. What are confidence intervals?itl.nist.gov
- Statistical Inferencepages.stat.wisc.edu
- Posterior Predictive Samplingmc-stan.org
- Posterior Analysismc-stan.org
- Package index — posteriormc-stan.org
- Modern Statistics for Modern Biology: Statistical Modelingweb.stanford.edu
- Posterior and Prior Predictive Checksmc-stan.org