Probability is a measure of how likely an event is to happen. It is written as a number from 0, meaning the event is impossible, to 1, meaning it is certain. Probability theory is the branch of mathematics that studies these measures and the rules for combining them. It is the theoretical basis of statistics, and fields as varied as statistical mechanics, quantum mechanics, genetics, economics and machine learning rely on it. Mathematicians agree on how probability is calculated. What a probability statement actually means is still debated in philosophy.
History
People played dice games for thousands of years before anyone built a mathematical theory of chance. The Italian physician and mathematician Gerolamo Cardano wrote Liber de ludo aleae (Book on Games of Chance) around 1564. It provided the first explicit calculations of dice probabilities, assuming uniform outcomes and introducing concepts like odds, but it was not printed until 1663.
Most historians date the start of the theory to 1654. That year Blaise Pascal, a French mathematician, received a set of gambling questions from the Chevalier de Méré. Pascal began corresponding with Pierre de Fermat, and their letters worked out how to divide a prize fairly when a game is interrupted. This question, known as the "problem of points," led Pascal and Fermat to the idea of mathematical expectation. In 1657 the Dutch scientist Christiaan Huygens published De ratiociniis in ludo aleae (On Reasoning in Games of Chance), the first printed book on the subject.
During the 18th century the theory grew far beyond gambling. Jacob Bernoulli's Ars Conjectandi was published after his death, in 1713. It proved an early form of the law of large numbers. Abraham de Moivre's The Doctrine of Chances (1718) developed the subject systematically, and de Moivre later showed that the normal curve approximates binomial probabilities. An essay by Thomas Bayes, published in 1763 after his death, described how to reason backward from observed effects to their probable causes. Pierre-Simon Laplace brought these results together in his Théorie analytique des probabilités (1812). He developed what is now known as Bayes' theorem and worked on the central limit theorem. Carl Friedrich Gauss used probability to analyse errors in astronomical measurements, which helped establish the normal distribution in science.
By the early 20th century the field was widely used but lacked rigorous foundations. That changed in 1933, when Andrey Kolmogorov published Grundbegriffe der Wahrscheinlichkeitsrechnung (Foundations of the Theory of Probability), which provided the first rigorous axiomatic foundation for probability theory by framing it within the framework of measure theory. Kolmogorov's approach is still the standard one.
Mathematical foundations
In Kolmogorov's framework, a random experiment is described by a probability space with three parts:
- a sample space Ω, which is the set of all possible outcomes;
- a collection of events, which are subsets of Ω, closed under complements and countable unions (a σ-algebra);
- a probability measure P, which gives each event a real number.
The measure must satisfy three axioms. Probabilities are never negative. The whole sample space has probability 1. The probability of a countable union of mutually exclusive events equals the sum of their separate probabilities. Every other rule follows from these axioms, using the language of set theory and measure theory. For example, the probability that an event does not happen is 1 minus the probability that it does.
Conditional probability P(A | B) is the probability of A given that B has occurred. It is defined as P(A and B) divided by P(B). Two events are independent if knowing that one occurred does not change the probability of the other. Bayes' theorem uses conditional probability to update a probability when new evidence arrives.
A random variable is a function that assigns a number to each outcome, such as the total shown by two dice. Its behaviour is summarised by a probability distribution. Discrete distributions, such as the binomial and Poisson distributions, give probabilities to separate values. Continuous distributions, such as the normal and exponential distributions, are described by density functions, and probabilities come from integration. The expected value of a random variable is its probability-weighted average. Variance measures how spread out its values are.
Limit theorems and stochastic processes
Two results connect probability with what is observed in long series of trials. The law of large numbers says that the average of many independent repetitions of an experiment tends toward the expected value. This explains why frequencies settle down over many trials. The central limit theorem says that the sum of many independent random variables with finite variance is approximately normally distributed, whatever the distribution of each one. This is why the bell curve appears so often in measurement data. Both results are statements about limits and belong to mathematical analysis.
A stochastic process is a family of random variables indexed by time or space. Examples include Markov chains, introduced by Andrey Markov in the early 1900s, as well as random walks and Brownian motion. These processes model systems that change randomly over time, such as diffusing particles, queues, population changes and financial prices. Stochastic calculus extends calculus to such processes.
Interpretations
Everyone accepts the axioms, but there are several views about what a probability is:
- Classical: probability is the ratio of favourable outcomes to all equally possible outcomes. This is the view of Pascal, Fermat and Laplace.
- Frequentist: probability is the limiting relative frequency of an event over an indefinitely long series of repetitions. The frequentist interpretation based on geometric notions of area and limiting notions of repeated events began with the work of Cournot and Venn.
- Bayesian (subjective): probability is a rational degree of belief, which is updated by Bayes' theorem as evidence comes in.
- Propensity: probability is a physical tendency of a system to produce certain outcomes. Karl Popper proposed this view.
These views shape statistical practice, since frequentist and Bayesian methods differ. They also raise questions in epistemology, inductive reasoning and the philosophy of science.
Applications
Probability underlies statistical inference, including the design and analysis of randomized controlled trials. In physics, statistical mechanics explains thermodynamic properties as averages over enormous numbers of particles. Quantum mechanics uses probability more directly: the Born rule gives the probabilities of measurement outcomes. Other uses include:
- in genetics, predicting how traits are inherited and modelling random changes in populations;
- in insurance and finance, pricing risk;
- in cryptography, analysing security;
- in computer science, randomised algorithms and Monte Carlo simulation, a method that John von Neumann and his colleagues helped develop in the 1940s.
In artificial intelligence, probabilistic models such as Bayesian networks, and the statistical learning methods behind modern machine learning, rest on probability theory.
References
- History of probability — Grokipediagrokipedia.com
- History of Probability: Timeline, Key Mathematicians & Problemsstatisticsfundamentals.com
- Cox's Theorem and the Jaynesian Interpretation of Probabilityarxiv.org
- A Historical Survey of the Development of Classical ...diva-portal.org
- The Dawn of Probability: A Journey to the Heart of Chance 🎲✨jfgouyet.fr
- Probability. A (very) brief history. Ionut Florescupeople.math.aau.dk