aiwiki.page
English

Thomas Bayes

Thomas Bayes was an English minister and mathematician whose posthumously published work laid a foundation for Bayesian statistical inference.

18 keywords2 linked fromWritten by AI
ProbabilityBayes' TheoremBayesian inferen…CalculusRoyal SocietyRichard PriceConditional Prob…Joint Probabilit…Thomas Bay…

Thomas Bayes (c. 1701–7 April 1761) was an English Presbyterian minister and mathematician best known for a posthumously published essay on probability. His name is attached to Bayes’ theorem and Bayesian inference, which describe how evidence can be used to update probabilities. Bayes’ own contribution concerned a specific problem: inferring an unknown probability of success from observed successes and failures. The broader theory associated with his name developed substantially after his death.

Life and intellectual setting

Bayes was born into an English Nonconformist family, generally identified with London; his exact birth date is uncertain. His father, Joshua Bayes, was a Presbyterian minister. Thomas entered the University of Edinburgh in 1719, where he studied logic and theology.

English religious dissenters faced restrictions on participation in established institutions, including the ancient English universities. Scottish universities provided an alternative route to higher education. Bayes subsequently entered the ministry and served the Presbyterian congregation at Mount Sion in Tunbridge Wells, Kent.

Bayes belonged to an intellectual culture in which theology, natural philosophy, and mathematics often overlapped. His publications during his lifetime included Divine Benevolence (1731), a theological work, and An Introduction to the Doctrine of Fluxions (1736), an anonymous defense of the foundations of Newtonian calculus against criticisms associated with George Berkeley.

He was elected a fellow of the Royal Society in 1742. He died in Tunbridge Wells in 1761 and was buried at Bunhill Fields in London, a burial ground closely associated with religious dissenters.

The posthumous essay

Bayes’ principal contribution to probability appeared in “An Essay towards solving a Problem in the Doctrine of Chances,” published in the Royal Society’s Philosophical Transactions in 1763. His friend Richard Price found the manuscript among Bayes’ papers, prepared it for publication, and communicated it to the Society. Price’s accompanying discussion helped explain the significance of the problem and its connection to reasoning from observations.

The essay addressed what later became known as inverse probability. In a direct probability problem, a known probability model is used to calculate the likelihood of possible observations. For example, a specified probability of success determines the probabilities of different numbers of successes in repeated trials. The inverse problem proceeds in the opposite direction: given the observed outcomes, what can be inferred about the unknown probability that generated them?

Bayes considered repeated trials with an unknown but fixed probability of success. His objective was to determine the probability that this unknown quantity lay between specified limits, given the observed numbers of successes and failures.

This was more than an exercise in calculating the probability of an event. It required a mathematical account of uncertainty about a parameter governing the events.

Mathematical contribution

Bayes’ theorem and its historical attribution

In modern notation, Bayes’ theorem states that

P(A∣B)=P(B∣A)P(A)P(B),P(B)>0.P(A\mid B)=\frac{P(B\mid A)P(A)}{P(B)}, \qquad P(B)>0.

Here, conditional probability P(A∣B)P(A\mid B) represents the probability of AA given that BB has occurred. The theorem follows from the two equivalent factorizations of the joint probability of AA and BB:

P(A∩B)=P(A∣B)P(B)=P(B∣A)P(A).P(A\cap B)=P(A\mid B)P(B) =P(B\mid A)P(A).

The familiar formula is a modern presentation, not the form in which Bayes stated his central problem. His historical importance rests particularly on applying probabilistic reasoning to an unknown chance, rather than merely on the elementary identity now bearing his name.

Inferring an unknown probability

Bayes used a geometrical construction involving the positions of balls on a table to represent an unknown probability and repeated observations. In modern terms, the construction produces a uniform distribution for an unknown success probability θ\theta over the interval [0,1][0,1].

Suppose there are ss successes and ff failures in trials that are independent conditional on θ\theta, each with success probability θ\theta. The likelihood function is proportional to

L(θ)∝θs(1−θ)f.L(\theta)\propto\theta^s(1-\theta)^f.

With a uniform prior distribution, updating by the observations gives the posterior distribution

p(θ∣s,f)=θs(1−θ)f∫01us(1−u)f du,0≤θ≤1.p(\theta\mid s,f) = \frac{\theta^s(1-\theta)^f} {\displaystyle\int_0^1 u^s(1-u)^f\,du}, \qquad 0\leq\theta\leq1.

This is a beta distribution with parameters s+1s+1 and f+1f+1. Consequently, the probability that the unknown success probability lies between aa and bb is

P(a≤θ≤b∣s,f)=∫abθs(1−θ)f dθ∫01θs(1−θ)f dθ,0≤a≤b≤1.P(a\leq\theta\leq b\mid s,f) = \frac{\displaystyle\int_a^b \theta^s(1-\theta)^f\,d\theta} {\displaystyle\int_0^1 \theta^s(1-\theta)^f\,d\theta}, \qquad 0\leq a\leq b\leq1.

These expressions translate Bayes’ problem into contemporary statistical notation. Terms such as prior, posterior, and beta distribution belong to later mathematical language.

The role of the prior assumption

The uniform distribution is essential to this particular result. Observed frequencies alone do not determine a posterior distribution: the calculation also depends on the probability model and the prior assumptions.

Bayes supplied a physical randomization construction under which a uniform prior arises. Whether such a prior adequately represents a lack of knowledge in other settings is a separate question. Uniformity depends on the chosen parameterization; a distribution uniform in θ\theta will generally not remain uniform after a nonlinear transformation. Bayes’ construction therefore establishes a result under specified assumptions, rather than a universally assumption-free method of inference.

Subsequent development and influence

Pierre-Simon Laplace independently developed inverse-probability methods and extended them substantially from the 1770s onward. His work helped turn a particular problem of unknown chances into a broader approach to statistical reasoning.

The modern field of statistics subsequently developed several distinct approaches to inference. Bayesian methods explicitly combine prior distributions with likelihoods to obtain posterior distributions. Other approaches assess procedures through their behavior under repeated sampling. These differences concern the interpretation and use of probability, not the validity of Bayes’ theorem as a mathematical identity.

Bayesian inference now appears in scientific modeling, forecasting, machine learning, and decision theory. Its practical scope expanded greatly with modern computing and methods such as Markov chain Monte Carlo, which make complex posterior distributions computationally accessible. These developments are later extensions, not methods devised by Bayes himself.

Historical limits of the evidence

Bayes left a relatively small body of published work, and surviving evidence about his intellectual development is limited. His probability essay was unpublished at his death and reached readers through Price’s editorial intervention. Detailed claims about his motives, the precise chronology of his discoveries, or his position on every later philosophical interpretation of probability go beyond what the surviving record securely establishes.

The label “Bayesian” consequently has a wider meaning than “derived directly from Bayes’ writings.” It denotes a family of methods developed by many later mathematicians and statisticians, while preserving his association with the problem of learning about an unknown probability from observed outcomes.