{
  "id": "bayes-factor",
  "title": "Bayes Factor",
  "lang": "en",
  "source_lang": "en",
  "summary": "A Bayes factor compares the marginal likelihoods of two statistical models, measuring how observed data change their relative support.",
  "category": "mathematics",
  "subcategory": "probability-statistics",
  "type": "concept",
  "aliases": [
    "Bayesian evidence ratio"
  ],
  "generated_at": "2026-11-05",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/bayes-factor/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/bayes-factor/",
      "json": "https://aiwiki.page/zh/bayes-factor.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "A Bayes factor is a ratio of the probabilities assigned to observed data by two competing statistical models or hypotheses. It measures how the data change the odds between those models under Bayesian inference. Unlike a comparison of best-fitting parameter values, a Bayes factor averages over each model’s parameter uncertainty using its specified prior distribution. It therefore depends on both the models’ sampling assumptions and their parameter priors."
    },
    {
      "heading": "Definition",
      "text": "For observed data \\(D\\) and models \\(M_1\\) and \\(M_0\\), the Bayes factor in favor of \\(M_1\\) over \\(M_0\\) is\n\n\\[B_{10}=\\frac{p(D\\mid M_1)}{p(D\\mid M_0)}.\\]\n\nThe quantity \\(p(D\\mid M_i)\\) is the model’s marginal likelihood, also called its model evidence. For discrete observations, it is a probability; for continuous observations, it is a density evaluated at the observed data, with both models expressed relative to a common underlying measure.\n\nIf model \\(M_i\\) has parameters \\(\\theta_i\\), its marginal likelihood is\n\n\\[p(D\\mid M_i) = \\int_{\\Theta_i} p(D\\mid\\theta_i,M_i)\\, \\pi_i(\\theta_i)\\, d\\theta_i,\\]\n\nwhere \\(p(D\\mid\\theta_i,M_i)\\) is the likelihood function and \\(\\pi_i(\\theta_i)\\) is a proper prior distribution within that model. Discrete parameters are summed over rather than integrated out.\n\nWhen a model specifies the data distribution completely, no parameter integration is needed. The Bayes factor then reduces to a likelihood ratio between two fully specified hypotheses.\n\nProvided the relevant marginal likelihoods are positive and finite,\n\n\\[B_{01}=\\frac{1}{B_{10}}.\\]\n\nThus, \\(B_{10}>1\\) favors \\(M_1\\), \\(B_{10}<1\\) favors \\(M_0\\), and \\(B_{10}=1\\) leaves their relative odds unchanged."
    },
    {
      "heading": "Posterior odds and interpretation",
      "text": "By Bayes’ theorem,\n\n\\[\\frac{P(M_1\\mid D)}{P(M_0\\mid D)} = B_{10} \\frac{P(M_1)}{P(M_0)}.\\]\n\nIn words,\n\n\\[\\text{posterior odds} = \\text{Bayes factor}\\times\\text{prior odds}.\\]\n\nThe Bayes factor is therefore an odds multiplier, not a posterior model probability. A value of \\(10\\) means that the observed data multiply the odds of \\(M_1\\) relative to \\(M_0\\) by ten.\n\nIf the two models are the only possibilities considered and have equal prior probabilities, then\n\n\\[P(M_1\\mid D)=\\frac{B_{10}}{1+B_{10}}.\\]\n\nUnder those assumptions, \\(B_{10}=10\\) corresponds to a posterior probability of \\(10/11\\), approximately \\(0.909\\). If the prior odds instead favor \\(M_0\\) by \\(100:1\\), the same Bayes factor produces posterior odds of \\(1:10\\), still favoring \\(M_0\\).\n\nTwo kinds of prior specification must be distinguished:\n\n- Prior model probabilities, such as \\(P(M_1)\\), determine the initial odds between models but do not enter the Bayes factor itself.\n- Within-model parameter priors, such as \\(\\pi_1(\\theta_1)\\), enter the marginal likelihood and generally affect the Bayes factor.\n\nDescriptions such as “moderate,” “strong,” or “decisive” evidence are sometimes attached to numerical ranges. These are conventions, not universal probability laws or decision thresholds. The direction of the comparison must also be stated: a large \\(B_{10}\\) and a large \\(B_{01}\\) support opposite models."
    },
    {
      "heading": "Example: a coin-tossing model",
      "text": "Suppose \\(k\\) heads are observed in \\(n\\) independent tosses. Compare:\n\n- \\(M_0\\): the probability of heads is fixed at \\(\\theta=1/2\\);\n- \\(M_1\\): \\(\\theta\\) is unknown and has a uniform distribution on \\([0,1]\\).\n\nUsing the binomial distribution,\n\n\\[p(k\\mid M_0)=\\binom{n}{k}2^{-n}.\\]\n\nUnder \\(M_1\\), integrating over the unknown probability gives\n\n\\[p(k\\mid M_1) = \\binom{n}{k} \\int_0^1\\theta^k(1-\\theta)^{n-k}\\,d\\theta = \\frac{1}{n+1}.\\]\n\nConsequently,\n\n\\[B_{10} = \\frac{2^n}{(n+1)\\binom{n}{k}}.\\]\n\nFor eight heads in ten tosses,\n\n\\[B_{10} = \\frac{1024}{11\\cdot45} \\approx2.07.\\]\n\nThe data modestly favor the unknown-probability model. For ten heads in ten tosses,\n\n\\[B_{10}=\\frac{1024}{11}\\approx93.1.\\]\n\nThese conclusions depend on the chosen alternative. A prior concentrated near \\(1/2\\), or one concentrated near extreme probabilities, would yield different marginal likelihoods. The uniform prior is also a special case of the beta distribution, which permits analogous calculations for a broader family of priors."
    },
    {
      "heading": "Model complexity and prior sensitivity",
      "text": "A Bayes factor compares average likelihoods, not maximum likelihoods. A flexible model may fit the observations extremely well for a small range of parameter values while assigning low likelihood across most of its prior distribution. Its marginal likelihood can therefore be lower than that of a simpler model.\n\nThis is often described as a Bayesian Occam factor: a model receives credit for fitting the data, but flexibility that spreads predictive probability over many unrealized possibilities can reduce its evidence. The effect is not simply a penalty based on the number of parameters. It depends on prior scale, parameter dependence, and how much of the prior distribution predicts data resembling those observed.\n\nThis distinction separates Bayes factors from maximum-likelihood estimation. Maximizing a likelihood asks how well a model can fit; integrating it asks how much probability the model assigned to the observations before those observations were used to update its parameters.\n\nDiffuse and improper priors\n\nMaking a parameter prior increasingly diffuse does not necessarily make a Bayes factor more neutral. In many comparisons between a point null and a continuous alternative, a very broad alternative prior spreads probability over values poorly supported by the data and can strongly favor the null. This behavior is commonly called Bartlett’s paradox.\n\nAn improper prior, whose total mass is not finite, generally makes an ordinary marginal likelihood undefined up to an arbitrary multiplicative constant. Because those constants need not cancel between models, the Bayes factor is usually not well defined. A proper posterior distribution within each model does not by itself resolve this problem.\n\nSome specially constructed methods allow cancellation of common constants or use fractional or intrinsic Bayes factors. Such methods require additional assumptions; arbitrary improper priors cannot be substituted into the ordinary formula without justification.\n\nSensitivity analysis examines how the comparison changes across plausible prior specifications, particularly priors for parameters present only under one model."
    },
    {
      "heading": "Relationship to hypothesis tests and p-values",
      "text": "Bayes factors are used in statistical hypothesis testing, including comparisons between a null hypothesis and an alternative hypothesis. Their interpretation differs fundamentally from that of a p-value.\n\nA p-value is a probability, calculated under the null model, of obtaining a test statistic at least as extreme as the observed one. A Bayes factor compares the probabilities or densities assigned to the observed data by two specified models. Therefore:\n\n- A p-value is not a posterior probability that the null hypothesis is true.\n- A Bayes factor is not the probability that either model is true.\n- A large p-value does not by itself establish evidence for the null.\n- A Bayes factor can favor a point null over a specified alternative by a substantial amount.\n\nThere is no universal conversion between a p-value and a Bayes factor. The latter depends on the alternative model and its parameter prior, information a p-value generally does not contain.\n\nJeffreys–Lindley phenomenon\n\nThe Jeffreys–Lindley paradox describes situations in which a conventional significance test rejects a point null while a Bayes factor favors it. In a typical construction, the standardized test statistic—and hence the p-value—is held fixed as sample size increases. The estimated departure from the null then shrinks, while a fixed, continuous alternative prior assigns relatively little mass to the increasingly narrow range supported by the data.\n\nThe results answer different questions under different assumptions; they are not an algebraic contradiction."
    },
    {
      "heading": "Computation and approximation",
      "text": "Marginal likelihoods can sometimes be evaluated analytically, especially with a conjugate prior. More complex models require numerical integration or simulation.\n\nCommon approaches include:\n\n- Numerical quadrature, particularly for low-dimensional parameter spaces.\n- Laplace approximation, which approximates the integrand near a dominant mode.\n- Importance sampling and bridge sampling, which estimate integrals or ratios using suitable sampling distributions.\n- Thermodynamic integration, which connects the prior and posterior through intermediate distributions.\n- Nested sampling, which reformulates the evidence integral in terms of prior probability mass.\n\nMarkov chain Monte Carlo samples from a posterior distribution, but ordinary posterior samples do not automatically provide its normalizing constant. Additional estimation procedures are usually required. The harmonic-mean estimator is a notable caution: it can be extremely unstable and can have infinite variance.\n\nFor regular models under suitable large-sample assumptions, the Bayesian information criterion provides an approximation. With\n\n\\[\\operatorname{BIC}_i = -2\\log p(D\\mid\\hat{\\theta}_i,M_i)+k_i\\log n,\\]\n\nwhere \\(k_i\\) is the parameter count and \\(n\\) is the sample size,\n\n\\[\\log B_{10} \\approx -\\frac12\\left(\\operatorname{BIC}_1-\\operatorname{BIC}_0\\right).\\]\n\nThis approximation suppresses prior-dependent and other lower-order terms. It is not generally reliable for small samples, singular models, or situations outside its regularity assumptions."
    },
    {
      "heading": "Sequential evidence",
      "text": "For data arriving as \\(D_1,\\ldots,D_n\\), the joint predictive distribution factorizes to give\n\n\\[B_{10}(D_{1:n}) = \\prod_{t=1}^{n} \\frac{ p(D_t\\mid D_{1:t-1},M_1) }{ p(D_t\\mid D_{1:t-1},M_0) }.\\]\n\nEach observation contributes a predictive likelihood ratio, with earlier observations updating each model’s parameter distribution.\n\nThis identity explains why evidence can be accumulated sequentially. Bayes factors from separate studies, however, cannot always be multiplied as if each study were analyzed independently. If studies share unknown parameters, the later study’s contribution must use the distribution updated by the earlier study, or the studies must be analyzed jointly.\n\nWith fixed model specifications and a stopping rule that is ignorable for the likelihood comparison, stopping according to observed data need not alter the Bayes factor for the realized observations. This does not make every sequential design interchangeable: selection, missing observations, data-dependent prior construction, or a model-dependent stopping mechanism may require explicit modeling."
    },
    {
      "heading": "Applications, history, and limitations",
      "text": "Bayes factors are used to compare regression specifications, scientific theories, dependence structures, and models with or without particular effects. With multiple candidate models,\n\n\\[P(M_i\\mid D) = \\frac{p(D\\mid M_i)P(M_i)} {\\sum_j p(D\\mid M_j)P(M_j)}.\\]\n\nThese model probabilities can support model averaging rather than selection of a single winner.\n\nThe modern development of Bayes factors is closely associated with Harold Jeffreys’s work on scientific inference in the 1930s, especially comparisons between a precise null and an alternative containing unknown parameters. Subsequent advances in numerical computation made evidence calculations feasible for increasingly complex models.\n\nSeveral limitations remain central:\n\n- Support is relative. A large Bayes factor establishes preference between the specified models, not that the preferred model is adequate. Both may fit poorly.\n- Model assumptions matter. Dependence, measurement error, selection mechanisms, and prior scales can substantially change the result.\n- Evidence and predictive performance differ. Bayes factors evaluate prior predictive support for the observed data. Cross-validation evaluates prediction under a particular training-and-testing procedure and may rank models differently.\n- Evidence is not a decision rule. Decisions also depend on consequences and a loss function, as formalized in decision theory.\n- Comparative support is not a model check. Posterior predictive checking can reveal discrepancies that a relative model comparison alone does not identify."
    }
  ],
  "links": [
    "bayesian-inference",
    "marginal-likelihood",
    "likelihood-function",
    "prior-distribution",
    "bayes-theorem",
    "uniform-distribution",
    "binomial-distribution",
    "beta-distribution",
    "maximum-likelihood-estimation",
    "posterior-distribution",
    "sensitivity-analysis",
    "statistical-hypothesis-testing",
    "null-hypothesis",
    "alternative-hypothesis",
    "p-value",
    "conjugate-prior",
    "markov-chain-monte-carlo",
    "bayesian-information-criterion",
    "cross-validation",
    "loss-function",
    "decision-theory",
    "posterior-predictive-checking"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}