{
  "id": "expected-value",
  "title": "Expected value",
  "lang": "en",
  "source_lang": "en",
  "summary": "The probability-weighted average of a random variable, a basic tool in probability, statistics, and decision-making.",
  "category": "mathematics",
  "subcategory": "probability-statistics",
  "type": "concept",
  "aliases": [
    "expectation",
    "mean (in the probabilistic sense)",
    "mathematical hope"
  ],
  "generated_at": "2026-10-08",
  "model": "pro/claude-haiku-5-5",
  "url": "https://aiwiki.page/en/expected-value/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/expected-value/",
      "json": "https://aiwiki.page/zh/expected-value.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "In probability theory, the expected value of a random variable is its probability-weighted average: each possible value is multiplied by the probability of that value, and the products are summed (or integrated, for continuous variables). It is also called the expectation or the mean, and it is the basic measure of a random quantity's central tendency. Its usefulness comes from its interpretation as the long-run average of many independent repetitions of the same experiment, a link formalized by the law of large numbers. The concept arose in the seventeenth century from problems about fair division of stakes in games of chance and was given its modern form through the work of later probabilists and, in the twentieth century, through measure-theoretic probability.\n\nThe entry treats expected value first as a computational object: its definition for discrete and continuous variables, its existence conditions, and the rules that make it tractable, above all linearity, which holds even when variables are dependent. It then turns to the historical development and to the interpretation of expected value as a long-run average, including where that interpretation can mislead. The remaining sections trace its use as a decision criterion and the paradoxes that criterion produced, its role in combinatorial counting and randomized algorithms, its generalization to conditional expectation and martingales, and its place in statistical estimation, machine learning, and physics, where the term \"expectation value\" is used for closely related but distinct quantities."
    },
    {
      "heading": "Discrete random variables",
      "text": "For a discrete random variable \\(X\\) that takes values in a finite or countable set, the expected value is\n\n\\[E[X] = \\sum_{x} x\\,P(X = x),\\]\n\nwhere the sum runs over the values \\(x\\) that \\(X\\) can take, and \\(P(X = x)\\) is its probability mass function. The weights are non-negative and sum to one, so the expression is a weighted average of the possible values, with each value counted in proportion to its probability. Because the formula uses only the probability-distribution of \\(X\\), any two variables with the same distribution share the same expected value.\n\nA fair six-sided die shows each face from 1 to 6 with probability \\(1/6\\), so\n\n\\[E[X] = \\sum_{k=1}^{6} k \\cdot \\tfrac{1}{6} = \\tfrac{21}{6} = 3.5.\\]\n\nThe value 3.5 is not a face the die can show. This illustrates that the expectation describes a long-run average, not a value that any single roll is expected to produce.\n\nA simple wager gives a second example. Suppose a bet gains 5 units with probability 0.3 and loses 2 units with probability 0.7. Then \\(E[X] = 5(0.3) + (-2)(0.7) = 0.1\\), so the bet has a small positive expected gain per play. If the loss were 3 units instead, the expectation would be \\(1.5 - 2.1 = -0.6\\), and the wager would be unfavorable to the player. The sign of the expectation therefore separates favorable from unfavorable bets, a point developed further in the section on decisions.\n\nWhen \\(X\\) takes countably many values \\(x_1, x_2, \\ldots\\), the finite sum becomes an infinite-series. For instance, if \\(P(X = k) = 2^{-k}\\) for \\(k = 1, 2, 3, \\ldots\\), then \\(E[X] = \\sum_{k=1}^{\\infty} k\\,2^{-k} = 2\\). In general, the expectation is well defined only when the series converges absolutely, that is, when\n\n\\[\\sum_{x} |x|\\,P(X = x) < \\infty.\\]\n\nThe condition matters because a series with terms of mixed sign can converge while its sum changes under rearrangement. As an example, let \\(X\\) take the value \\((-1)^k 2^k / k\\) with probability \\(2^{-k}\\) for \\(k \\ge 1\\). The probabilities sum to one, and \\(\\sum_x x\\,P(X = x) = \\sum_k (-1)^k / k\\) converges to \\(-\\ln 2\\). However, \\(\\sum_x |x|\\,P(X = x) = \\sum_k 1/k\\) diverges, so \\(E[X]\\) is undefined despite the convergent signed series. Without absolute convergence, the value would depend on the order in which the possible values are listed, which is why the definition requires it.\n\nThe expected value is written \\(E[X]\\) in most texts, while \\(E(X)\\) is also common, particularly where parentheses are preferred. In statistics the same quantity is often denoted \\(\\mu\\) or \\(\\mu_X\\). The terms \"expected value,\" \"expectation,\" and \"mean\" (when referring to a distribution rather than a data set) are used interchangeably. The mean of a sample is a different object, an estimator of this quantity, and is discussed in the section on estimation. The continuous analogue, which replaces the sum with an integral against a density, is treated in the following section."
    },
    {
      "heading": "General and continuous definitions",
      "text": "When a random variable \\(X\\) has a probability density function \\(f\\), the sum in the discrete definition becomes an integral:\n\n\\[E[X] = \\int_{-\\infty}^{\\infty} x\\, f(x)\\, dx,\\]\n\nprovided that \\(\\int_{-\\infty}^{\\infty} |x|\\, f(x)\\, dx\\) is finite. The quantity \\(f(x)\\,dx\\) plays the role that the probability mass plays in the discrete case: it is the weight given to a small interval around \\(x\\). The uniform distribution on \\([a,b]\\) has density \\(1/(b-a)\\) on that interval, and the integral gives \\(E[X] = (a+b)/2\\), the midpoint of the interval. For the exponential distribution with rate \\(\\lambda > 0\\), the integral \\(\\int_0^\\infty x\\,\\lambda e^{-\\lambda x}\\,dx\\) equals \\(1/\\lambda\\). The tail formula below gives the same value with less computation.\n\nMeasure-theoretic definition\n\nThe general definition is stated on a probability-space \\((\\Omega, \\mathcal F, P)\\), where \\(X\\colon \\Omega \\to \\mathbb R\\) is a measurable-function. The expectation is the lebesgue-integral of \\(X\\) with respect to \\(P\\):\n\n\\[E[X] = \\int_\\Omega X\\, dP.\\]\n\nThis one formula covers discrete and continuous variables alike. Both kinds induce a distribution on \\(\\mathbb R\\), and the density formula above is what the general integral reduces to when that distribution has a density with respect to lebesgue-measure.\n\nFor a variable taking both positive and negative values, one splits \\(X = X^+ - X^-\\), where \\(X^+ = \\max(X, 0)\\) and \\(X^- = \\max(-X, 0)\\) are both nonnegative. Then \\(E[X] = E[X^+] - E[X^-]\\), and this is meaningful as long as at least one of the two terms is finite.\n\nExistence\n\nThe expectation exists exactly when \\(E|X| = \\int_\\Omega |X|\\, dP\\) is finite, which is equivalent to both \\(E[X^+]\\) and \\(E[X^-]\\) being finite. A symmetric limit is not enough. The Cauchy distribution has density \\(\\frac{1}{\\pi(1+x^2)}\\). The integral of \\(|x|\\) against this density diverges logarithmically at both ends, so the expectation is undefined. The symmetric truncated integrals \\(\\int_{-R}^{R} x f(x)\\,dx\\) are zero for every \\(R\\), so a limit exists along symmetric truncations. That limit is a principal value, however, and it does not satisfy the definition of expectation.\n\nTail formula\n\nFor a nonnegative variable \\(X\\),\n\n\\[E[X] = \\int_0^\\infty P(X > t)\\, dt.\\]\n\nThe identity follows from writing \\(X = \\int_0^\\infty \\mathbf 1\\{X > t\\}\\, dt\\) pointwise and then exchanging the order of integration, which is justified for nonnegative integrands. No finiteness assumption is needed, so both sides may equal \\(+\\infty\\). This makes the formula a practical tool for infinite expectations: the expectation is infinite exactly when the tail probability decays too slowly for the integral to converge. For instance, if \\(P(X > t) = t^{-\\alpha}\\) for \\(t \\ge 1\\), then \\(E[X] = 1 + \\frac{1}{\\alpha-1} = \\frac{\\alpha}{\\alpha-1}\\) when \\(\\alpha > 1\\), while for \\(\\alpha \\le 1\\) the integral diverges and \\(E[X] = \\infty\\). Such heavy tails are closely related to power-law behaviour. For a variable of both signs whose two one-sided expectations are finite, the same idea gives \\(E[X] = \\int_0^\\infty P(X > t)\\,dt - \\int_0^\\infty P(X < -t)\\,dt\\)."
    },
    {
      "heading": "Basic properties and computational rules",
      "text": "Linearity\n\nFor finitely many random variables \\(X_1,\\dots,X_n\\), each with a finite expectation, and real constants \\(a_1,\\dots,a_n\\), the expectation of the weighted sum is the weighted sum of the expectations:\n\n\\[\\mathbb{E}\\Big[\\sum_{i=1}^{n} a_i X_i\\Big] = \\sum_{i=1}^{n} a_i\\,\\mathbb{E}[X_i].\\]\n\nThis identity does not depend on how the variables are related. For discrete variables it can be checked from the joint probability mass function: summing \\((x+y)\\,P(X=x,Y=y)\\) over all pairs splits into \\(\\sum_x x\\,P(X=x)\\) plus \\(\\sum_y y\\,P(Y=y)\\), because each marginal is recovered by summing out the other variable. In the general measure-theoretic setting, the same fact follows from additivity of the Lebesgue integral. Extending linearity to infinite sums is not automatic. It requires a dominating condition, such as \\(\\sum_n \\mathbb{E}|X_n| < \\infty\\), and this is where the dominated convergence theorem and the monotone convergence theorem are used.\n\nMonotonicity, constants and bounds\n\nIf \\(X \\ge Y\\) almost surely and both expectations exist, then \\(\\mathbb{E}[X] \\ge \\mathbb{E}[Y]\\). A constant shifts the expectation by the same amount, so \\(\\mathbb{E}[X+c]=\\mathbb{E}[X]+c\\), and scaling multiplies it: \\(\\mathbb{E}[cX]=c\\,\\mathbb{E}[X]\\). Applying monotonicity to \\(-|X| \\le X \\le |X|\\) gives\n\n\\[|\\mathbb{E}[X]| \\le \\mathbb{E}|X|,\\]\n\nand if \\(a \\le X \\le b\\) almost surely, then \\(a \\le \\mathbb{E}[X] \\le b\\). The inequality \\(\\mathbb{E}[X^2] \\ge (\\mathbb{E}[X])^2\\) is a special case of Jensen's inequality for the convex function \\(t \\mapsto t^2\\). It is also equivalent to the non-negativity of variance.\n\nExpectations of functions of a variable\n\nIf \\(Y = g(X)\\) for a measurable function \\(g\\), the expectation of \\(Y\\) can be computed directly from the distribution of \\(X\\), without first finding the distribution of \\(Y\\). For a discrete variable,\n\n\\[\\mathbb{E}[g(X)] = \\sum_x g(x)\\,P(X=x),\\]\n\nand for a continuous variable with density \\(f\\),\n\n\\[\\mathbb{E}[g(X)] = \\int_{-\\infty}^{\\infty} g(x)\\,f(x)\\,dx,\\]\n\nprovided \\(\\mathbb{E}|g(X)| < \\infty\\). This is often called the law of the unconscious statistician. For a fair die, \\(\\mathbb{E}[X^2] = (1+4+9+16+25+36)/6 = 91/6\\), which is obtained without writing down the distribution of \\(X^2\\). For \\(X\\) uniform on \\([0,1]\\), \\(\\mathbb{E}[e^X] = \\int_0^1 e^x\\,dx = e-1\\), while the density of \\(e^X\\) would have to be derived first if the rule were not used.\n\nProducts, covariance and variance\n\nIf \\(X\\) and \\(Y\\) are independent and both are integrable, then \\(\\mathbb{E}[XY] = \\mathbb{E}[X]\\,\\mathbb{E}[Y]\\). For discrete variables the proof is direct: \\(P(X=x, Y=y) = P(X=x)\\,P(Y=y)\\), so the double sum factorizes. The continuous case follows the same pattern with the joint density. The converse does not hold, and the product rule fails for dependent variables. For a fair die, take \\(Y = X\\). Then \\(\\mathbb{E}[XY] = \\mathbb{E}[X^2] = 91/6 \\approx 15.17\\), while \\(\\mathbb{E}[X]\\,\\mathbb{E}[Y] = 49/4 = 12.25\\).\n\nThe general identity is \\(\\mathbb{E}[XY] = \\mathbb{E}[X]\\,\\mathbb{E}[Y] + \\operatorname{Cov}(X,Y)\\), where \\(\\operatorname{Cov}(X,Y) = \\mathbb{E}[(X-\\mathbb{E}X)(Y-\\mathbb{E}Y)]\\) is the covariance. Independence forces covariance to be zero, but zero covariance does not imply independence. Let \\(X\\) be uniform on \\(\\{-1,0,1\\}\\) and \\(Y = X^2\\). Then \\(\\mathbb{E}[X] = 0\\) and \\(\\mathbb{E}[XY] = \\mathbb{E}[X^3] = 0\\), so the covariance is zero, yet \\(Y\\) is a function of \\(X\\) and the two are plainly dependent.\n\nVariance follows from the same tools. Expanding \\((X-\\mu)^2\\) and applying linearity with \\(\\mu = \\mathbb{E}[X]\\) gives\n\n\\[\\operatorname{Var}(X) = \\mathbb{E}[(X-\\mu)^2] = \\mathbb{E}[X^2] - \\mu^2,\\]\n\nso \\(\\operatorname{Var}(X) = \\mathbb{E}[X^2] - (\\mathbb{E}[X])^2\\). For a fair die this gives \\(91/6 - 49/4 = 35/12\\). Constants enter quadratically, so \\(\\operatorname{Var}(aX+b) = a^2 \\operatorname{Var}(X)\\). For any two variables, \\(\\operatorname{Var}(X+Y) = \\operatorname{Var}(X) + \\operatorname{Var}(Y) + 2\\operatorname{Cov}(X,Y)\\), which reduces to the additive rule when the covariance vanishes, in particular under independence. The formula \\(\\mathbb{E}[X^2] - \\mu^2\\) requires the second moment to be finite; when it is not, the variance is infinite even if the mean exists. The standard deviation is the square root of this quantity."
    },
    {
      "heading": "Origins and historical development",
      "text": "The concept of expected value grew out of seventeenth-century gambling problems, most prominently the problem of points: how to divide the stakes fairly when a game of chance is interrupted before it ends. The question had been debated for centuries before it gained new momentum in 1654, when the Chevalier de Méré, a French writer and amateur mathematician, put it to Blaise Pascal. Pascal then corresponded with Pierre de Fermat. Their letters reasoned from the possible continuations of an unfinished game rather than from the game's past results, and historians usually date the founding of probability theory to this exchange. The letters stayed private for a time, so their methods reached a wider public only through later work.\n\nChristiaan Huygens's De ratiociniis in ludo aleae (1657) was the first printed work to state the principle explicitly. Huygens had heard of the Paris discussions but worked out his own solutions, and he extended the method beyond two players to more complicated cases. His foundational proposition held that a prospect is worth the sum that would give the same expectation in a fair game. From it he derived the rule for two equally likely prospects: if a player expects either \\(a\\) or \\(b\\) with equal chance, the expectation is worth \\(\\frac{a+b}{2}\\). His justification was game-theoretic rather than based on long-run frequencies, and it fixed which divisions of stakes count as fair. The word \"expectation\" itself was not used in its modern sense by Pascal or Huygens; the mathematical meaning developed gradually afterward.\n\nJakob Bernoulli's Ars Conjectandi, published posthumously in 1713, marked the next step. Its first part reprinted Huygens's treatise with commentary and offered a simpler route to the same rule, using gamblers who draw from boxes whose contents are sums of money. Its fourth part contains the early statement of the law of large numbers, which connects long-run averages of observed outcomes with the underlying probabilities. Pierre-Simon Laplace then gave the quantity its explicit name. In the 1814 Essai philosophique sur les probabilités he defined the advantage of a game as the sum hoped for multiplied by the probability of obtaining it, and called this \"mathematical hope,\" tying it to the equitable division of stakes. Laplace also distinguished it from \"moral expectation,\" a rival notion based on the utility of wealth. One source places the first appearance of the adjective \"mathematical\" in his 1812 Théorie analytique des probabilités, so the exact date of the term is reported differently in different accounts.\n\nIn the nineteenth century, Pafnuty Chebyshev began to think systematically in terms of the expectations of random variables. The notation changed in 1901, when William Allen Whitworth's Choice and Chance used the letter E for expectation; the notation later became standard in English-language texts and sits alongside the German Erwartungswert, the French espérance mathématique, and the Spanish esperanza matemática. The modern axiomatic foundation came in 1933, when Andrey Kolmogorov defined probability as a measure on a probability space. Under that framework, the expectation of a random variable became an integral with respect to that measure, placing the concept on measure-theoretic grounds."
    },
    {
      "heading": "Long-run averages and the law of large numbers",
      "text": "The frequency interpretation\n\nLet \\(X_1, X_2, \\dots, X_n\\) be independent copies of a random variable \\(X\\) with finite mean \\(\\mu = E[X]\\), and let \\(\\bar X_n = \\frac{1}{n}\\sum_{i=1}^{n} X_i\\) be their sample average. The sample average tends to lie near \\(\\mu\\) when \\(n\\) is large. If \\(\\sigma^2 = \\operatorname{Var}(X)\\) is finite, Chebyshev's inequality gives a quantitative form of this statement:\n\n\\[P\\big(|\\bar X_n - \\mu| \\ge \\varepsilon\\big) \\le \\frac{\\sigma^2}{n\\varepsilon^2} \\qquad \\text{for every } \\varepsilon > 0.\\]\n\nThe bound shrinks toward zero as \\(n\\) grows. This is the precise sense in which a long series of repeated trials \"settles\" near the expected value. For a fair die, the average of many rolls approaches 3.5, although no single roll can show that value. The argument depends on the trials being independent and identically distributed, so each trial must be unaffected by the others (see statistical independence).\n\nWeak and strong laws\n\nThe law of large numbers comes in two main forms, which differ in the mode of convergence. The weak law states that \\(\\bar X_n\\) converges in probability to \\(\\mu\\): for each tolerance \\(\\varepsilon > 0\\), the probability that \\(\\bar X_n\\) lies farther than \\(\\varepsilon\\) from \\(\\mu\\) tends to zero. The strong law states that \\(\\bar X_n\\) converges almost surely to \\(\\mu\\), meaning the set of outcomes where the running average fails to reach \\(\\mu\\) has probability zero. Such failing sequences can exist; they simply form an infinitesimal fraction of the sample space. The strong form implies the weak form, but the weak form is the easier of the two to establish. Clearly the strong law implies the weak law, but the weak law is easier to prove, and it has somewhat better quantitative estimates.\n\nThe conditions differ as well. When the variance is finite, the weak law follows from a short argument based on Chebyshev's inequality, which is the most common route in textbooks. The proof is easy when the variables have a finite variance, and it is most often based on Chebyshev's Inequality. For the strong law, Kolmogorov's theorem requires only that the variables be identically distributed with a finite mean; no variance assumption is needed. The weak law for i.i.d. variables likewise holds under a finite mean alone, although the proof is harder without a variance bound. Without a finite mean, the averages need not converge to any constant at all.\n\nExpected value compared with the mode and the median\n\nThe expected value is a weighted average, and it can differ sharply from the most likely outcome or from the median. Consider a hypothetical lottery in which a ticket costs one unit and the net gain \\(X\\) takes three values: \\(X = -1\\) with probability 0.97, \\(X = 4\\) with probability 0.02, and \\(X = 1000\\) with probability 0.01. The expected value is well defined and positive:\n\n\\[E[X] = 0.97(-1) + 0.02(4) + 0.01(1000) = 9.11.\\]\n\nThe mode, the most probable value, is \\(-1\\). The median is also \\(-1\\), because \\(P(X \\le -1) = 0.97\\) exceeds one half. Thus 97 percent of players lose their ticket price, while the mean is pulled upward by a rare large prize. A player who buys many tickets would see the total average approach 9.11 per ticket, so the expected value describes the long-run average of the game, not the typical result for any one player. The same gap between mean and median appears in any strongly skewed distribution, such as income or insurance claim sizes.\n\nLimits of the interpretation\n\nThe expected value is not necessarily a value the variable can take. The mean of a fair die is 3.5, and no face shows that number. It also does not predict the outcome of any single trial: in the lottery above, a single ticket almost certainly loses. The law of large numbers concerns the average of many trials, and it does not imply that a run of losses will be offset by later gains. Deviations from the mean are not \"repaid\" by subsequent trials; they are simply diluted as \\(n\\) increases. When the mean does not exist, as for the Cauchy distribution, the sample average does not stabilize at all, so the interpretation breaks down entirely."
    },
    {
      "heading": "Decisions, utility and paradoxes",
      "text": "As a decision criterion, expected value ranks options by their probability-weighted payoffs. In decision theory, an option with a higher expected gain is the default choice when stakes are small and a decision is repeated many times. The same reasoning underlies insurance pricing. An insurer that charges premiums above the expected cost of claims, after administrative costs, earns a margin across a large pool of policyholders, and the law of large numbers makes that margin dependable. An individual who buys cover pays more than the expected loss, because the purchase converts an uncertain large loss into a small certain cost. Assessing a bet works the same way. A wager that wins 100 with probability 0.5 and loses 80 with probability 0.5 has an expected return of +10, so it is favourable on average. A game with negative expected return loses money in the long run, which is why casino games are built with a house edge.\n\nThe St. Petersburg paradox shows the limits of this criterion. A fair coin is tossed until it first lands heads. If the first heads appears on toss n, the player receives 2^n units. Since P(n) = 2^(-n), each term of the expectation equals 1, so the sum is an infinite series of ones and the expected payoff diverges. Nicolas Bernoulli posed the problem in the early eighteenth century, and Daniel Bernoulli published an analysis in 1738 in the commentaries of the St. Petersburg Academy, which gave the game its name. Few people would pay more than a modest sum to play, which suggested that the expected monetary value was not the quantity people actually compare. Daniel Bernoulli's response was to evaluate the expected change in utility instead of money. He assumed that the increase in utility from a small gain is inversely proportional to current wealth, which integrates to a logarithmic utility function u(w) = ln w. Under this function the expected utility is finite and the implied fair price is modest. Gabriel Cramer had proposed a square-root utility in 1728. Later analyses showed that the paradox can be rebuilt for logarithmic utility when payoffs grow fast enough, so some treatments require bounded utility.\n\nExpected utility theory generalizes this idea: a rational agent chooses the option that maximizes \\(E[u(X)]\\) for some utility function u. Its modern axiomatic form was given by John von Neumann and Oskar Morgenstern in their 1944 work on game theory, whose axioms include completeness, transitivity, continuity and independence. These axioms guarantee that a utility function representing preferences exists. Risk aversion appears as concavity of u. By Jensen's inequality, \\(E[u(X)] \\le u(E[X])\\) for concave u, so a person with such a utility function prefers a sure payment equal to the expected value of a gamble to the gamble itself. The gap between the two is a measure of risk aversion.\n\nExperimental evidence has challenged expected utility as a descriptive model. In 1953 Maurice Allais described pairs of choices in which many respondents chose a certain option in the first pair and a risky option in the second. For instance, many people prefer a sure one million over a gamble with a 10% chance of five million and an 89% chance of one million, yet prefer a gamble with a 10% chance of five million over one with an 11% chance of one million. These choices violate the independence axiom, because adding the same common outcome to both options of each pair should not reverse a preference. Daniel Ellsberg's 1961 urn experiment produced a related violation, in which people prefer betting on a known-probability colour over an ambiguous one, a pattern now called ambiguity aversion. In response, Daniel Kahneman and Amos Tversky proposed prospect theory in 1979. It assesses outcomes as gains or losses relative to a reference point, uses a value function that is concave for gains, convex for losses and steeper for losses, and applies probability weights that overweight small probabilities and near-certain outcomes. Related formal models such as rank-dependent expected utility retain a single weighted average structure while changing how probabilities are transformed."
    },
    {
      "heading": "Linearity and indicator variables in combinatorics",
      "text": "Indicators turn counts into sums\n\nAn indicator variable is a random variable that takes the value 1 when an event \\(A\\) occurs and 0 otherwise. Its expectation is \\(E[I_A] = P(A)\\), because the mean of a variable that is either 0 or 1 is just the probability of the 1. Any count can then be written as a sum of indicators. If \\(X\\) counts how many of the events \\(A_1, \\dots, A_n\\) occur, then \\(X = \\sum_{i=1}^{n} I_{A_i}\\), and linearity of expectation gives \\(E[X] = \\sum_{i=1}^{n} P(A_i)\\). The calculation requires only the individual probabilities. It does not require computing the distribution of \\(X\\), which for combinatorial counts is often intractable.\n\nThe fixed points of a random permutation\n\nThe standard illustration is the number of fixed points of a uniformly random permutation of \\(\\{1, \\dots, n\\}\\), meaning elements that are sent to themselves. Let \\(I_i\\) indicate that element \\(i\\) is fixed. Exactly \\((n-1)!\\) of the \\(n!\\) permutations fix a given element, so \\(P(I_i = 1) = 1/n\\). Summing over the \\(n\\) elements gives \\(n \\cdot \\tfrac{1}{n} = 1\\). The expected number of fixed points is therefore 1 for every \\(n\\). Direct enumeration over all \\(n!\\) permutations would be laborious, while the indicator argument takes a few lines.\n\nThe argument never uses the fact that the indicators \\(I_i\\) are dependent. If element 1 is fixed, the probability that element 2 is also fixed changes, but linearity does not care. This is the general point: linearity holds for any random variables, so dependence is irrelevant to the mean of a sum.\n\nDependent events and counting patterns\n\nThe same technique handles counts whose terms overlap. Consider the number of inversions in a random permutation, the pairs \\((i, j)\\) with \\(i < j\\) whose order is reversed. Each pair is reversed with probability 1/2 by symmetry, and there are \\(\\binom{n}{2}\\) pairs, so the expected count is \\(\\binom{n}{2}/2\\). Similarly, in a sequence of \\(n\\) fair coin flips, the expected number of adjacent positions showing equal results is \\((n-1)/2\\). The indicators for neighbouring positions are not independent, since they share a flip, yet the expectation is unaffected. Such calculations are typically far simpler than attempts to count the exact distribution of patterns.\n\nThe usual caution applies: linearity gives the mean of a sum, not the probability that the sum is positive. Showing that a count is zero with positive probability requires further arguments, which is where the probabilistic method comes in.\n\nThe probabilistic method and Ramsey graphs\n\nThe probabilistic method uses expectation to prove that an object with a desired property exists, without constructing it. The key step is the first-moment principle: if the expected number of \"bad\" configurations is less than 1, then some configuration has no bad features at all.\n\nPaul Erdős applied this in 1947 to Ramsey numbers. The Ramsey number \\(R(k)\\) is the smallest \\(n\\) such that every red–blue colouring of the edges of \\(K_n\\) contains a monochromatic complete subgraph on \\(k\\) vertices. In a random colouring, a fixed set of \\(k\\) vertices is monochromatic with probability \\(2^{1-\\binom{k}{2}}\\). Summing over the \\(\\binom{n}{k}\\) sets of \\(k\\) vertices, the expected number of monochromatic \\(k\\)-cliques is \\(\\binom{n}{k} 2^{1-\\binom{k}{2}}\\). For \\(n\\) around \\(2^{k/2}\\) and \\(k \\ge 3\\), this quantity falls below 1, so some colouring has no monochromatic \\(k\\)-clique. This proves an exponential lower bound on \\(R(k)\\) without exhibiting such a colouring. The result is often regarded as the beginning of the probabilistic method, and it was for decades the main lower-bound technique for Ramsey numbers, with later refinements improving the bound only by constant factors.\n\nThe method is non-constructive in an essential way: it certifies existence while leaving the explicit object unknown. Later work by other mathematicians built on this approach, using the same basic idea of bounding the expected number of bad events. The method is developed further in the probabilistic method, and the counting tools used here rest on binomial coefficients and permutations."
    },
    {
      "heading": "Randomized algorithms and running time",
      "text": "For a randomized algorithm, the running time is itself a random variable, because the algorithm consumes random bits that determine which steps it takes. Its expected running time is the average over those internal random choices, with the input held fixed. This differs from the average-case analysis of a deterministic algorithm, which averages over a distribution of inputs. Because the randomness belongs to the algorithm, the expected cost on each input can be bounded without any assumption about how inputs are generated, so no fixed input can reliably force bad expected behaviour. The worst-case expected running time is the maximum, over all inputs, of this per-input expectation.\n\nRandomized quicksort is the standard worked example. Its pivot is chosen uniformly at random from the current subarray. Let the sorted values be \\(z_1 < \\dots < z_n\\) and let \\(X\\) be the total number of comparisons. Define the indicator \\(X_{ij}\\) to be 1 when \\(z_i\\) and \\(z_j\\) are compared and 0 otherwise. Two values are compared exactly when the first pivot chosen from the range \\(z_i,\\dots,z_j\\) is one of them, which happens with probability \\(2/(j-i+1)\\). Linearity of expectation then gives\n\n\\[E[X] = \\sum_{i<j} \\frac{2}{j-i+1} \\le 2n\\,H_n,\\]\n\nwhere \\(H_n\\) is the \\(n\\)th harmonic number. Since \\(H_n \\sim \\ln n\\), \\(E[X] = O(n \\log n)\\), and the total running time, which is \\(O(n+X)\\), is \\(O(n \\log n)\\) in expectation. The argument needs no independence among the indicators, only linearity of expectation, which is why it works despite the strong dependence among comparisons.\n\nLas Vegas and Monte Carlo algorithms are two classes distinguished by what the randomness affects. A Las Vegas algorithm always returns a correct answer, and only its running time is random, as in randomized quicksort. A Monte Carlo algorithm has a bounded running time but may return a wrong answer with small probability, as in Karger's randomized minimum-cut algorithm. The two classes are linked: a Las Vegas algorithm can be converted into a Monte Carlo algorithm by stopping it after a fixed time limit and outputting an arbitrary answer if it has not finished. Markov's inequality, which states that \\(P(T \\ge t) \\le E[T]/t\\) for a nonnegative running time \\(T\\), bounds the probability of exceeding the limit by the expected running time, so the expectation controls the error rate of the converted algorithm.\n\nHash tables give the most widely used example. Under simple uniform hashing with separate chaining, where keys are placed independently and uniformly into \\(m\\) slots, the load factor \\(\\alpha = n/m\\) is the average chain length. Successful and unsuccessful searches both take expected time \\(\\Theta(1+\\alpha)\\). Choosing the hash function at random from a universal family moves the analysis from assumptions about the data to properties of the random choice: two distinct keys collide with probability at most \\(1/m\\). Summing over the other stored keys shows that the expected number of collisions with a fixed key is at most \\(N/M\\), where \\(N\\) is the number of stored keys and \\(M\\) the table size, so each lookup, insertion or deletion has expected constant cost when the table size is proportional to \\(N\\).\n\nAmortized analysis complements these expectations. When a table grows, a single resize may cost \\(\\Theta(n)\\) time, but doubling the capacity spreads that cost over the many cheap insertions that precede it, giving \\(O(1)\\) amortized cost per update. Amortized analysis averages over a sequence of operations rather than over random choices, and the two averages are often combined, as in an expected amortized cost for a randomized structure."
    },
    {
      "heading": "Conditional expectation and martingales",
      "text": "Definition and interpretation\n\nFor a random variable \\(X\\) with \\(E|X|<\\infty\\) and a sub-σ-algebra \\(\\mathcal G\\subseteq\\mathcal F\\), the conditional expectation \\(E[X\\mid\\mathcal G]\\) is a random variable \\(Z\\) that is \\(\\mathcal G\\)-measurable, integrable, and satisfies\n\n\\[E[Z\\,\\mathbf 1_A]=E[X\\,\\mathbf 1_A]\\quad\\text{for every } A\\in\\mathcal G.\\]\n\nBecause \\(Z\\) is fixed only up to events of probability zero, the defining equalities hold almost surely. When \\(\\mathcal G\\) is generated by a discrete variable \\(Y\\), the object reduces to \\(E[X\\mid Y]=g(Y)\\), where \\(g(y)=E[X\\mid Y=y]\\) is computed from conditional probabilities. Its value is itself random, since it changes with \\(Y\\).\n\nIts central interpretation is as a prediction. Among all \\(\\mathcal G\\)-measurable variables \\(W\\) with finite second moment, \\(E[X\\mid\\mathcal G]\\) minimizes the mean-square error \\(E[(X-W)^2]\\). It is therefore the best mean-square predictor of \\(X\\) given the information in \\(\\mathcal G\\). The unconditional mean is the special case in which \\(\\mathcal G\\) is the trivial σ-algebra \\(\\{\\varnothing,\\Omega\\}\\), where no information is available and the best constant predictor is \\(E[X]\\).\n\nCore properties\n\n- Linearity and monotonicity carry over from ordinary expectation: \\(E[aX+bY\\mid\\mathcal G]=aE[X\\mid\\mathcal G]+bE[Y\\mid\\mathcal G]\\), and \\(X\\le Y\\) implies \\(E[X\\mid\\mathcal G]\\le E[Y\\mid\\mathcal G]\\) almost surely.\n- Tower property. If \\(\\mathcal H\\subseteq\\mathcal G\\), then\n\n\\[E\\big[E[X\\mid\\mathcal G]\\mid\\mathcal H\\big]=E[X\\mid\\mathcal H].\\]\n\nTaking \\(\\mathcal H\\) to be trivial yields the law of total expectation, \\(E\\big[E[X\\mid\\mathcal G]\\big]=E[X]\\). Informally, coarser information cannot be recovered by averaging over finer information in a way that changes the forecast.\n\n- Pulling out known factors. If \\(Z\\) is \\(\\mathcal G\\)-measurable and \\(XZ\\) is integrable, then \\(E[ZX\\mid\\mathcal G]=Z\\,E[X\\mid\\mathcal G]\\).\n- Independence. If \\(X\\) is independent of \\(\\mathcal G\\), then \\(E[X\\mid\\mathcal G]=E[X]\\), since the information in \\(\\mathcal G\\) says nothing about \\(X\\).\n- Conditional Jensen inequality. For a convex function \\(\\varphi\\), \\(\\varphi(E[X\\mid\\mathcal G])\\le E[\\varphi(X)\\mid\\mathcal G]\\).\n\nIn this framework a σ-algebra represents the information available at a given time, and a larger σ-algebra represents finer information.\n\nMartingales and optional stopping\n\nA sequence \\((M_n)\\) adapted to a filtration \\((\\mathcal F_n)\\) is a martingale if each \\(M_n\\) is integrable and\n\n\\[E[M_{n+1}\\mid\\mathcal F_n]=M_n.\\]\n\nConditional on the history, the expected change in the process is zero, which is the sense in which a martingale models a fair game. Replacing the equality with \\(\\ge\\) or \\(\\le\\) gives a submartingale or supermartingale. For any integrable \\(X\\), the sequence \\(M_n=E[X\\mid\\mathcal F_n]\\) is a martingale, which follows directly from the tower property.\n\nA stopping time \\(\\tau\\) is a random time whose event \\(\\{\\tau\\le n\\}\\) is determined by information available up to time \\(n\\). The optional stopping theorem asserts that \\(E[M_\\tau]=E[M_0]\\) under suitable hypotheses, which include boundedness of \\(\\tau\\), boundedness of the martingale together with \\(\\tau\\) being almost surely finite, or integrability of \\(\\tau\\) together with bounded increments. The hypotheses cannot be dropped. In a fair coin game, a doubling strategy that stops at the first win yields a net gain of 1 with probability one, so the expected terminal value is 1 while the starting value is 0. Here the stopping time is unbounded and the martingale is not uniformly integrable, so the conclusion fails. The general statement is treated in optional stopping theorem and its companion concept, the stopping time.\n\nExistence via the Radon–Nikodym theorem\n\nThe general definition requires a construction that does not rely on densities or elementary conditioning. For a nonnegative \\(X\\), define a measure on \\((\\Omega,\\mathcal G)\\) by \\(Q(A)=E[X\\mathbf 1_A]\\), and then treat the general integrable case by splitting \\(X\\) into its positive and negative parts. Since \\(P(A)=0\\) implies \\(Q(A)=0\\) for \\(A\\in\\mathcal G\\), \\(Q\\) is absolutely continuous with respect to \\(P\\) restricted to \\(\\mathcal G\\). The Radon–Nikodym theorem then guarantees a \\(\\mathcal G\\)-measurable density \\(Z=dQ/dP\\), unique up to null sets, and this density is \\(E[X\\mid\\mathcal G]\\). The finiteness of \\(P\\) ensures that the required σ-finiteness holds, so the construction applies in full generality on a probability space."
    },
    {
      "heading": "Estimation and statistics",
      "text": "The sample mean and bias\n\nLet \\(X_1,\\dots,X_n\\) be observations of a variable with expected value \\(\\mu\\). The sample mean \\(\\bar X = \\frac{1}{n}\\sum_{i=1}^n X_i\\) is the natural estimator of \\(\\mu\\). Applying linearity to the sum gives \\(E[\\bar X]=\\mu\\) for every \\(n\\), so the sample mean is an unbiased estimator. Formally, the expectation of the sample mean is computed by pulling the expectation inside the sum and reducing it to \\(E[X]\\).\n\nAn estimator \\(\\hat\\theta\\) of a parameter \\(\\theta\\) has bias \\(\\operatorname{Bias}(\\hat\\theta)=E[\\hat\\theta]-\\theta\\). An estimator whose expected value equals the parameter it estimates is called unbiased. In practical terms, repeating the estimation many times and averaging the results recovers the true value, and this holds at any sample size. Repeated estimation of the expectation value through the mean, on average, produces the correct value.\n\nUnbiasedness is not automatic for every natural construction. If the variance is estimated by averaging squared deviations from \\(\\bar X\\) and dividing by \\(n\\), the expectation of that estimator is \\(\\frac{n-1}{n}\\sigma^2\\), so it systematically underestimates \\(\\sigma^2\\). Dividing by \\(n-1\\) instead of \\(n\\) yields an unbiased estimator of the variance, since the factor \\(\\frac{n}{n-1}\\) corrects the shortfall. The intuition is that deviations measured from the sample mean rather than the true mean are smaller on average. Bias is one part of accuracy: the mean squared error splits into the variance of the estimator plus its squared bias.\n\nVariance of the sample mean and standard error\n\nWhen the observations are independent and share the same variance \\(\\sigma^2\\), the variance of \\(\\bar X\\) works out to \\(\\sigma^2/n\\). The spread of the estimate therefore shrinks as the sample grows, and its standard deviation shrinks at the rate \\(1/\\sqrt n\\). The standard deviation of an estimator is called its standard error. It is the standard deviation of the estimator and indicates how close one can expect the estimator to be to the parameter.\n\n\\[\\operatorname{SE}(\\bar X)=\\frac{\\sigma}{\\sqrt n}\\]\n\nThe formula requires \\(\\sigma\\), which is usually unknown. Using the variance of the sample mean to gauge its accuracy therefore requires knowing the population variance, which is rarely available. In practice the sample standard deviation \\(S\\) replaces \\(\\sigma\\), giving the estimated standard error \\(S/\\sqrt n\\). Note that linearity alone delivers the mean of \\(\\bar X\\); the \\(\\sigma^2/n\\) formula for the variance needs independence.\n\nMethod of moments\n\nThe method of moments is one of the oldest estimation techniques. Karl Pearson introduced it in 1894. For each positive integer \\(k\\), the population moment \\(\\mu_k'=E[X^k]\\) is matched with the sample moment \\(m_k'=\\frac{1}{n}\\sum_{i=1}^n X_i^k\\). The estimators are found by equating the first \\(k\\) sample moments to the corresponding \\(k\\) population moments and solving the resulting system of equations.\n\nFor an exponential distribution with rate \\(\\lambda\\), the first moment is \\(E(X)=1/\\lambda\\). Setting the sample mean equal to this population moment gives \\(\\lambda = 1/m_1\\). Hence \\(\\hat\\lambda = 1/\\bar X\\). For a normal distribution, the first two population moments are \\(E(X)=\\mu\\) and \\(E(X^2)=\\mu^2+\\sigma^2\\). Matching these gives \\(\\hat\\mu=\\bar X\\) and \\(\\hat\\sigma^2 = m_2' - \\bar X^2 = \\frac{1}{n}\\sum_i (X_i-\\bar X)^2\\). This variance estimate is the divide-by-\\(n\\) version discussed above, so method-of-moments estimators can be biased.\n\nPosterior expectation and the central limit theorem\n\nIn Bayesian inference, the parameter is treated as random and described by a posterior distribution obtained from a prior and the data. Under squared error loss, the optimal estimator from a Bayesian viewpoint is the posterior mean. Changing the loss function changes the summary: the Bayes estimate under absolute error loss is the posterior median.\n\nAs an example, a binomial observation of 13 successes in 20 trials, combined with a suitable prior, yields a posterior that is a beta distribution with \\(\\alpha=17\\) and \\(\\beta=11\\). A Beta\\((\\alpha,\\beta)\\) variable has mean \\(\\alpha/(\\alpha+\\beta)\\). The posterior mean is therefore \\(17/28\\approx 0.607\\), which serves as the point estimate of the success probability.\n\nThe central limit theorem describes the sampling distribution of \\(\\bar X\\) for large samples. Its distribution is approximately normal, centred at \\(\\mu\\), with variance \\(\\sigma^2/n\\). This approximation is the basis for confidence intervals and tests that use the standard error, as well as for interpreting the uncertainty of sample-mean estimates in general."
    },
    {
      "heading": "Machine learning and optimization",
      "text": "Expected risk and empirical risk\n\nIn supervised learning, a predictor \\(f\\) is scored by a loss function \\(L(y, f(x))\\). Its expected risk is \\(R(f) = \\mathbb{E}[L(Y, f(X))]\\), the expectation taken over the joint distribution of inputs and labels. This quantity is also called the true risk or generalization error. Because the data-generating distribution is almost always unknown, the expectation cannot be computed directly. Learning therefore substitutes the empirical risk, the sample average of the loss over \\(n\\) training examples:\n\n\\[\\hat R(f) = \\frac{1}{n}\\sum_{i=1}^{n} L(y_i, f(x_i)).\\]\n\nEach training loss is a draw from the same distribution, so this sample mean estimates the expected risk in the same way a sample mean estimates an expectation. Empirical risk minimization (ERM) selects the predictor that minimizes \\(\\hat R\\), using it as a proxy for the unreachable target.\n\nGeneralization gap and overfitting\n\nThe difference \\(\\Delta(f) = R(f) - \\hat R(f)\\) is called the generalization gap, and the identity \\(R(f) = \\hat R(f) + \\Delta(f)\\) holds by definition. Minimizing the empirical risk therefore controls the true risk only if the gap stays small. Predictors are usually better on examples they have seen than on new ones, so the gap is typically positive.\n\nThe gap is the formal side of overfitting. A model that memorizes its training set can drive \\(\\hat R\\) close to zero while its performance on unseen data stays poor. In the classical picture, increasing model complexity first lowers both the empirical and the expected risk, and beyond some point the expected risk rises again while the empirical risk keeps falling. Practitioners estimate the true risk with a held-out test set, whose average loss is another sample mean. The trade-off between fitting the sample and generalizing from it is discussed in the bias–variance trade-off and in generalization theory.\n\nStochastic gradient descent\n\nMinimizing the empirical risk by gradient descent requires the gradient of the average loss, which is the average of the per-example gradients. Computing it means touching the whole dataset at every step, which is impractical for large datasets. Stochastic gradient descent (SGD) instead draws one example \\(i\\) uniformly at random and uses \\(\\nabla \\ell_i(\\theta)\\) as the update direction. Because the draw is uniform, the expected value of this gradient equals the full gradient, \\(\\mathbb{E}[\\nabla \\ell_i(\\theta)] = \\nabla R(\\theta)\\), so the estimate is unbiased even though each individual step is noisy.\n\nMini-batch SGD uses a random subset \\(B\\) of \\(m\\) examples and the averaged gradient \\(g_B(\\theta) = \\frac{1}{m}\\sum_{i \\in B}\\nabla \\ell_i(\\theta)\\). It remains unbiased, while its variance falls in proportion to \\(1/m\\). The practical trade-off is between cost per step and gradient noise: a larger batch gives a more accurate direction but requires more computation. Because the noise has mean zero, it cancels over many iterations; convergence analyses typically require step sizes whose sum diverges while the sum of their squares stays finite. Mini-batch sizes in common use range from tens to a few hundred examples.\n\nExpected return in reinforcement learning\n\nIn reinforcement-learning, an agent acts in an environment and receives rewards. The discounted return from time \\(t\\) is \\(G_t = R_{t+1} + \\gamma R_{t+2} + \\gamma^2 R_{t+3} + \\cdots\\), where the discount factor \\(\\gamma\\) lies between 0 and 1. The return satisfies the recursion \\(G_t = R_{t+1} + \\gamma G_{t+1}\\). The state-value function is the expected return under a policy \\(\\pi\\) from a given state: \\(v_\\pi(s) = \\mathbb{E}_\\pi[G_t \\mid S_t = s]\\).\n\nTaking expectations of the recursion gives the Bellman expectation equation:\n\n\\[v_\\pi(s) = \\sum_a \\pi(a \\mid s) \\sum_{s', r} p(s', r \\mid s, a)\\,\\bigl[r + \\gamma\\, v_\\pi(s')\\bigr].\\]\n\nThe value of a state is thus the expected immediate reward plus the discounted expected value of the successor state. The equation relates values across neighboring states and turns a long-horizon problem into a one-step consistency condition. For optimal behavior, the Bellman optimality equation replaces the policy average with a maximum over actions, \\(Q^*(s,a) = \\mathbb{E}[R_{t+1} + \\gamma \\max_{a'} Q^*(S_{t+1}, a') \\mid S_t = s, A_t = a]\\). Value iteration and policy evaluation use these equations to compute values, and Monte Carlo methods estimate them by averaging sampled returns."
    },
    {
      "heading": "Expectation values in physics",
      "text": "Quantum expectation values and the Born rule\n\nIn quantum mechanics a measurable quantity is represented by a self-adjoint operator \\(\\hat A\\) acting on a hilbert-space of states. For a pure state described by a wave-function \\(\\psi\\) normalized so that \\(\\langle\\psi|\\psi\\rangle = 1\\), the expectation value is\n\n\\[\\langle A\\rangle = \\langle \\psi | \\hat A | \\psi \\rangle .\\]\n\nFor a mixed state with density operator \\(\\rho\\), the same quantity is \\(\\mathrm{Tr}(\\rho\\hat A)\\). If \\(\\hat A\\) has eigenvalues \\(a_i\\) with eigenprojectors \\(P_i\\), then \\(\\langle A\\rangle = \\sum_i a_i \\langle\\psi|P_i|\\psi\\rangle\\). This is a probability-weighted average of the possible measurement values, where the weights are the probabilities assigned by the Born rule. In the nondegenerate case, the weight of outcome \\(a_i\\) is \\(|\\langle a_i|\\psi\\rangle|^2\\). The expectation value therefore has the same algebraic form as the expected value of a random-variable with a discrete distribution, but the distribution itself is fixed by the state and the operator.\n\nQuantum and classical averages differ in origin\n\nIn classical statistical physics, an expectation value averages over a probability distribution that reflects incomplete knowledge. Each system has definite values of position and momentum, and the averaging compensates for not knowing which values apply. In quantum theory the state vector is treated as the complete description of the system, so the randomness is not attributed to ignorance of the initial state. Instead, it arises when a measurement is made.\n\nThis distinction has consequences for interpretation. Some authors stress that the quantum expectation is a formal quantity with a meaning that does not depend on any interpretation, and that the statement linking it to measurement results is a separate assumption.Note that the q-expectation value has a formal meaning independent of the interpretation; the measured expectation form of Born's rule just asserts that measurements result in a random variable whose expectation agrees with the formal q-expectation. In quantum statistical mechanics, where analyses often proceed without any explicit measurement, the expectation values that appear are usually treated as formal expressions.One usually makes a theoretical analysis of quantum processes where no actual measurement is made before the process is completed.\n\nEhrenfest's theorem\n\nPaul Ehrenfest showed in 1927 that expectation values of position and momentum obey equations with the same form as classical motion. For a particle with Hamiltonian \\(H = p^2/2m + V(x)\\), the result reads\n\n\\[\\frac{d\\langle x\\rangle}{dt} = \\frac{\\langle p\\rangle}{m}, \\qquad \\frac{d\\langle p\\rangle}{dt} = -\\left\\langle \\frac{dV}{dx} \\right\\rangle .\\]\n\nThe first equation is the quantum analogue of the velocity relation, and the second is an analogue of Newton's second law. However, the theorem does not say that the expected position follows a classical trajectory in general. The force term is the expectation of \\(-dV/dx\\), not the force evaluated at the expected position, so in general \\(\\langle V'(x)\\rangle \\neq V'(\\langle x\\rangle)\\). The gap between these two quantities is controlled by the spread of the wave packet. The statement is exact for a harmonic oscillator, where the force is linear, and it is approximately valid when the wave function is strongly localized. The approximation's dependence on the spread of the wave packet is noted in a standard treatment of the theorem.For general systems, if the wave function is highly concentrated around a point, then the expected position and expected momentum will be almost the same, since both will be approximately equal to that point.\n\nEnsemble averages in statistical mechanics\n\nIn statistical mechanics, a system held at fixed temperature \\(T\\) by a heat reservoir is described by the canonical ensemble. Each microstate \\(i\\) with energy \\(E_i\\) has probability\n\n\\[p_i = \\frac{e^{-\\beta E_i}}{Z}, \\qquad \\beta = \\frac{1}{k_{\\mathrm B} T}, \\qquad Z = \\sum_i e^{-\\beta E_i},\\]\n\nwhere \\(Z\\) is the partition-function. The average energy is the expectation of the Hamiltonian over this distribution, \\(\\langle E\\rangle = \\sum_i E_i p_i\\). It can be obtained from \\(Z\\) alone by differentiation:\n\n\\[\\langle E\\rangle = -\\frac{\\partial \\ln Z}{\\partial \\beta} .\\]\n\nThis derivative relation is the practical reason the partition function is central. Once \\(Z\\) is known as a function of temperature, the average energy, and from it other thermodynamic quantities, follow without summing over individual states. The average energy in this setting coincides with the internal-energy of the system. The energy itself is not fixed in the canonical ensemble, so its fluctuations are also measured by a variance, given by \\(\\langle E^2\\rangle - \\langle E\\rangle^2\\).\n\nThese ensemble averages, like the quantum expectation values above, are probability-weighted means. The difference lies in the source of the weights: in the canonical ensemble they come from the boltzmann-distribution and thermal equilibrium, and in quantum theory they come from the Born rule applied to a state."
    }
  ],
  "links": [
    "probability",
    "random-variable",
    "law-of-large-numbers",
    "probability-mass-function",
    "probability-distribution",
    "infinite-series",
    "probability-density-function",
    "uniform-distribution",
    "exponential-distribution",
    "probability-space",
    "measurable-function",
    "lebesgue-integral",
    "lebesgue-measure",
    "cauchy-distribution",
    "power-law",
    "dominated-convergence-theorem",
    "almost-surely",
    "jensens-inequality",
    "variance",
    "statistical-independence",
    "covariance",
    "standard-deviation",
    "blaise-pascal",
    "pierre-de-fermat",
    "andrey-kolmogorov",
    "convergence-in-probability",
    "decision-theory",
    "ramsey-theory",
    "probabilistic-method",
    "binomial-coefficient",
    "permutation",
    "quicksort",
    "randomized-algorithm",
    "hash-table",
    "amortized-analysis",
    "conditional-probability",
    "mean-squared-error",
    "sample-mean",
    "estimator",
    "estimator-bias",
    "standard-error",
    "bayesian-inference",
    "posterior-distribution",
    "prior-distribution",
    "loss-function",
    "central-limit-theorem",
    "empirical-risk-minimization",
    "overfitting",
    "bias-variance-tradeoff",
    "generalization",
    "gradient-descent",
    "stochastic-gradient-descent",
    "reinforcement-learning",
    "value-function",
    "hilbert-space",
    "wave-function",
    "born-rule",
    "partition-function",
    "internal-energy",
    "boltzmann-distribution"
  ],
  "sources": [
    {
      "title": "Properties of Expected Value - GeeksforGeeks",
      "url": "https://www.geeksforgeeks.org/engineering-mathematics/properties-of-expected-value/"
    },
    {
      "title": "Random Variables COS 341 Fall 2002, lecture 21",
      "url": "https://www.cs.princeton.edu/courses/archive/fall02/cs341/lec22.pdf"
    },
    {
      "title": "Variance 3.3.1 Linearity of Expectation with Indicator RVs",
      "url": "https://web.stanford.edu/class/archive/cs/cs109/cs109.1218/files/student_drive/3.3.pdf"
    },
    {
      "title": "6.2: Variance of Discrete Random Variables - Statistics LibreTexts",
      "url": "https://stats.libretexts.org/Bookshelves/Probability_Theory/Introductory_Probability_(Grinstead_and_Snell)/06:_Expected_Value_and_Variance/6.02:_Variance_of_Discrete_Random_Variables"
    },
    {
      "title": "Foundations of Vector Retrieval",
      "url": "https://arxiv.org/pdf/2401.09350"
    },
    {
      "title": "Covariance",
      "url": "https://en.wikipedia.org/wiki/Covariance"
    },
    {
      "title": "Expected value",
      "url": "https://en.wikipedia.org/wiki/Expected_value"
    },
    {
      "title": "Theory of Probability. A Historical Essay",
      "url": "https://arxiv.org/pdf/1802.09966"
    },
    {
      "title": "Chance Combinatorics: The Theory that History Forgot John D. Norton",
      "url": "https://sites.pitt.edu/~jdnorton/papers/chance_combinatorics_final.pdf"
    },
    {
      "title": "Theory of Probability. An Elementary Treatise against a Historical Background",
      "url": "https://arxiv.org/pdf/1802.03485"
    },
    {
      "title": "Expected values (mathematical concept)",
      "url": "https://www.ebsco.com/research-starters/science/expected-values-mathematical-concept/"
    },
    {
      "title": "Reading Theorie Analytique des Probabilites",
      "url": "https://arxiv.org/pdf/1203.6249"
    },
    {
      "title": "Definition:Expectation - ProofWiki",
      "url": "https://proofwiki.org/wiki/Definition:Expectation"
    },
    {
      "title": "Expected value - Wikiwand",
      "url": "https://www.wikiwand.com/en/Expected_value"
    },
    {
      "title": "Expected Value Formula: E(X) = Σx·P(x) — Definition, Examples & Calculator",
      "url": "https://statisticsfundamentals.com/statistics-and-probability/expected-value/"
    },
    {
      "title": "Faith Believes, Hope Expects: The Impact of Calvin's Theology on the Mathematics of Chance",
      "url": "https://arxiv.org/pdf/2407.13312"
    },
    {
      "title": "Weak and Strong Law of Large Numbers",
      "url": "https://davidbieber.com/snippets/2021-07-18-laws-of-large-numbers/"
    },
    {
      "title": "275A, Notes 3: The weak and strong law of large numbers",
      "url": "https://terrytao.wordpress.com/2015/10/23/275a-notes-3-the-weak-and-strong-law-of-large-numbers/"
    },
    {
      "title": "Law of Large Numbers",
      "url": "https://www.statlect.com/asymptotic-theory/law-of-large-numbers"
    },
    {
      "title": "The Laws of Large Numbers Compared Tom Verhoeff July 1993 1 Introduction",
      "url": "http://www.dklevine.com/archive/strong-law.pdf"
    },
    {
      "title": "St. Petersburg paradox",
      "url": "https://en.wikipedia.org/wiki/St._Petersburg_paradox"
    },
    {
      "title": "The St. Petersburg Paradox (Stanford Encyclopedia of Philosophy)",
      "url": "https://plato.stanford.edu/entries/paradox-stpetersburg/"
    },
    {
      "title": "Utility assessment",
      "url": "https://en.wikipedia.org/wiki/Utility_assessment"
    },
    {
      "title": "The Saint Petersburg Paradox and Its Solution[v1]",
      "url": "https://www.preprints.org/manuscript/202412.0526"
    },
    {
      "title": "The time resolution of the St Petersburg paradox",
      "url": "https://royalsocietypublishing.org/doi/10.1098/rsta.2011.0065"
    },
    {
      "title": "Exploring the Allais Paradox in Microeconomic Theory",
      "url": "https://umbrex.com/resources/economics-concepts/microeconomic-theory/allais-paradox/"
    },
    {
      "title": "A Contextual Risk Model for the Ellsberg Paradox",
      "url": "https://arxiv.org/pdf/1105.1814"
    },
    {
      "title": "Paradoxes and violations of normative decision theory Title",
      "url": "https://calhoun.nps.edu/server/api/core/bitstreams/60670540-3d93-4975-a7f5-14b473be26c7/content"
    },
    {
      "title": "Chapter 6 Alternatives to Expected Utility Theory",
      "url": "https://ocw.mit.edu/courses/14-123-microeconomic-theory-iii-spring-2015/2b03b8a02c2c864e2f27c0de603a1572_MIT14_123S15_Chap6.pdf"
    },
    {
      "title": "13 Challenges to Expected Utility",
      "url": "https://jonathanweisberg.org/vip/challenges-to-expected-utility.html"
    },
    {
      "title": "Expected number of fixed points of permutation equals one - Groupprops",
      "url": "https://groupprops.subwiki.org/wiki/Expected_number_of_fixed_points_of_permutation_equals_one"
    },
    {
      "title": "Expected Value and Linearity of Expectation - Probability for Quant Interviews",
      "url": "https://www.datainterview.com/courses/probability-quant-interviews/concept-expected-value"
    },
    {
      "title": "MITOCW",
      "url": "https://ocw.mit.edu/courses/18-226-probabilistic-methods-in-combinatorics-fall-2022/1Y0uagXz-GqUdxG-nQNHC-ewpw1qrDO98_transcript.pdf"
    },
    {
      "title": "Combinatorics/Bounds for Ramsey numbers - Wikibooks, open books for an open world",
      "url": "https://en.wikibooks.org/wiki/Combinatorics/Bounds_for_Ramsey_numbers"
    },
    {
      "title": "Upper bounds on diagonal Ramsey numbers [after Campos, Griffiths, Morris, and Sahasrabudhe]",
      "url": "https://arxiv.org/pdf/2411.09321"
    },
    {
      "title": "An exponential improvement for diagonal Ramsey",
      "url": "https://arxiv.org/pdf/2303.09521"
    },
    {
      "title": "Packing nearly optimal Ramsey R(3,t) graphs",
      "url": "https://arxiv.org/pdf/1711.05877"
    },
    {
      "title": "Examples in probability: Ramsey numbers and the pr",
      "url": "https://homepages.inf.ed.ac.uk/rmayr/ramsey-Ch7.pdf"
    },
    {
      "title": "Randomized Algorithms Part Two",
      "url": "https://web.stanford.edu/class/archive/cs/cs161/cs161.1138/lectures/10/Small10.pdf"
    },
    {
      "title": "Lecture 3 Probabilistic Analysis and Randomized Quicksort 3.1 Overview",
      "url": "https://www.cs.cmu.edu/~avrim/451f11/lectures/lect0906.pdf"
    },
    {
      "title": "Randomized algorithm",
      "url": "https://en.wikipedia.org/wiki/Randomized_algorithm"
    },
    {
      "title": "Las Vegas algorithm",
      "url": "https://en.wikipedia.org/wiki/Las_Vegas_algorithm"
    },
    {
      "title": "Lecture 10 Universal and Perfect Hashing 10.1 Overview",
      "url": "https://www.cs.cmu.edu/~avrim/451f11/lectures/lect1004.pdf"
    },
    {
      "title": "GitHub - smilank3/hashing_with_chaining · GitHub",
      "url": "https://github.com/smilank3/hashing_with_chaining"
    },
    {
      "title": "Universal Hash Function - an overview",
      "url": "https://www.sciencedirect.com/topics/computer-science/universal-hash-function"
    },
    {
      "title": "Introduction to Randomness and Statistics",
      "url": "https://arxiv.org/pdf/0910.4545"
    },
    {
      "title": "Unbiased Estimator - an overview",
      "url": "https://www.sciencedirect.com/topics/mathematics/unbiased-estimator"
    },
    {
      "title": "7.2: Sample Variance - Statistics LibreTexts",
      "url": "https://stats.libretexts.org/Courses/Saint_Mary's_College_Notre_Dame/DSCI_500B_Essential_Probability_Theory_for_Data_Science_(Kuter)/07:_The_Sample_Variance_and_Other_Distributions/7.2:_Sample_Variance"
    },
    {
      "title": "Moment Estimator - an overview",
      "url": "https://www.sciencedirect.com/topics/mathematics/moment-estimator"
    },
    {
      "title": "Chapter 9 Method of Moments, Maximum Likelihood Estimator (Lecture on 01/28/2020)",
      "url": "https://bookdown.org/jkang37/stat205b-notes/lecture07.html"
    },
    {
      "title": "Method of Moments - GeeksforGeeks",
      "url": "https://www.geeksforgeeks.org/maths/method-of-moments/"
    },
    {
      "title": "The Bayesian Way: Uncertainty, Learning, and Statistical Reasoning",
      "url": "https://arxiv.org/pdf/2512.05883"
    },
    {
      "title": "Squared Error Loss Function - an overview",
      "url": "https://www.sciencedirect.com/topics/mathematics/squared-error-loss-function"
    },
    {
      "title": "1 Estimating the uncertainty attached to a sample mean: s2 ...",
      "url": "https://web.stanford.edu/class/polisci100a/hyptests.pdf"
    },
    {
      "title": "8 Estimation 8.1 Introduction",
      "url": "https://archive.math.arizona.edu/tgk/466_s18/chap8.pdf"
    },
    {
      "title": "Estimation of the variance",
      "url": "https://www.statlect.com/fundamentals-of-statistics/variance-estimation"
    },
    {
      "title": "Efficiency (statistics)",
      "url": "https://en.wikipedia.org/wiki/Efficiency_(statistics)"
    },
    {
      "title": "Bayesian Estimate - an overview",
      "url": "https://www.sciencedirect.com/topics/mathematics/bayesian-estimate"
    },
    {
      "title": "Predicting the generalization gap in neural networks using topological data analysis",
      "url": "https://arxiv.org/pdf/2203.12330"
    },
    {
      "title": "2.4 Generalization Error and Empirical Risk Minimization",
      "url": "https://www.informit.com/articles/article.aspx?p=3222332&seqNum=4"
    },
    {
      "title": "Patterns, predictions, and actions: A story about machine learning",
      "url": "https://arxiv.org/pdf/2102.05242"
    },
    {
      "title": "Generalization",
      "url": "https://mlstory.org/generalization.html"
    },
    {
      "title": "Alice's Adventures in a Differentiable Wonderland -- Volume I, A Tour of the Land",
      "url": "https://arxiv.org/pdf/2404.17625"
    },
    {
      "title": "Learn Risk Minimization: Expected vs Empirical Risk",
      "url": "https://codefinity.com/courses/v2/abf891a8-0a13-4925-bcbc-0bafe55c27be/8aa87fb9-cab1-477a-a0b0-4c055d59908d/4ad62a5e-f271-42a2-bbc3-ae6c02b8cf33"
    },
    {
      "title": "Stochastic Gradient Descent: Theory, Numerical Examples, and Implementation",
      "url": "https://ekbanaml.github.io/applied%20machine%20learning/stochastic-gradient-descent/"
    },
    {
      "title": "Understanding Stochastic Gradient Descent: Estimating the Gradient from a Sample",
      "url": "https://www.statology.org/understanding-stochastic-gradient-descent-estimating-the-gradient-from-a-sample/"
    },
    {
      "title": "2/27 16.1 Stochastic gradient descent 16.2 SGD for Risk ...",
      "url": "https://web.stanford.edu/class/ee270/scribes/lecture16.pdf"
    },
    {
      "title": "Gradient Descent with Stochastic Subspaces via Persistence of Memory",
      "url": "https://arxiv.org/pdf/2609.18416"
    },
    {
      "title": "A Distributional Perspective on Reinforcement Learning",
      "url": "https://arxiv.org/pdf/1707.06887"
    },
    {
      "title": "An Introduction to Deep Reinforcement and Imitation Learning",
      "url": "https://arxiv.org/pdf/2512.08052"
    },
    {
      "title": "Deep Reinforcement Learning: From First Principles to Reasoning Models",
      "url": "https://arxiv.org/pdf/2608.00133"
    },
    {
      "title": "Reinforcement Learning Part 4: Expected Return, Value Functions, and Bellman Equations - Shawn Hymel",
      "url": "https://shawnhymel.com/3350/reinforcement-learning-part-4-expected-return-value-functions-and-bellman-equations/"
    },
    {
      "title": "Foundations of quantum physics I. A critique of the tradition",
      "url": "https://arxiv.org/pdf/1902.10778"
    },
    {
      "title": "The Born Rule—100 Years Ago and Today - PMC",
      "url": "https://pmc.ncbi.nlm.nih.gov/articles/PMC12026146/"
    },
    {
      "title": "Ehrenfest theorem",
      "url": "https://en.wikipedia.org/wiki/Ehrenfest_theorem"
    },
    {
      "title": "Measurement in quantum mechanics",
      "url": "https://en.wikipedia.org/wiki/Measurement_in_quantum_mechanics"
    },
    {
      "title": "Unifying Quantum and Classical Dynamics",
      "url": "https://arxiv.org/pdf/2601.10423"
    },
    {
      "title": "Ehrenfest's Theorem",
      "url": "https://www.maths.dur.ac.uk/users/kasper.peeters/mathphys/ehrenfest.html"
    },
    {
      "title": "17.3: The Average Ensemble Energy is Equal to the Observed Energy of a System - Chemistry LibreTexts",
      "url": "https://chem.libretexts.org/Bookshelves/Physical_and_Theoretical_Chemistry_Textbook_Maps/Physical_Chemistry_(LibreTexts)/17:_Boltzmann_Factor_and_Partition_Functions/17.03:_The_Average_Ensemble_Energy_is_Equal_to_the_Observed_Energy_of_a_System"
    },
    {
      "title": "Canonical Ensemble - an overview",
      "url": "https://www.sciencedirect.com/topics/physics-and-astronomy/canonical-ensemble"
    },
    {
      "title": "Partition function (statistical mechanics)",
      "url": "https://en.wikipedia.org/wiki/Partition_function_(statistical_mechanics)"
    }
  ],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}