{
  "id": "jensens-inequality",
  "title": "Jensen's Inequality",
  "lang": "en",
  "source_lang": "en",
  "summary": "Jensen’s inequality states that a convex function evaluated at an average cannot exceed the average of its values.",
  "category": "mathematics",
  "subcategory": "analysis",
  "type": "concept",
  "aliases": [
    "Jensen inequality"
  ],
  "generated_at": "2026-10-04",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/jensens-inequality/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/jensens-inequality/",
      "json": "https://aiwiki.page/zh/jensens-inequality.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Jensen’s inequality is a theorem relating convex functions to weighted averages and expected values. It states that applying a convex function after averaging gives a result no greater than averaging after applying the function. For concave functions, the inequality reverses. Its finite and probabilistic forms express the same underlying principle. (web.stanford.edu)"
    },
    {
      "heading": "Finite weighted form",
      "text": "Let \\(C\\) be a convex set in a real vector space, and let \\(f:C\\to\\mathbb R\\) be convex. For points \\(x_1,\\ldots,x_n\\in C\\) and weights \\(\\lambda_i\\geq0\\) satisfying \\(\\sum_i\\lambda_i=1\\), Jensen’s inequality states\n\n\\[f\\!\\left(\\sum_{i=1}^{n}\\lambda_i x_i\\right) \\leq \\sum_{i=1}^{n}\\lambda_i f(x_i).\\]\n\nThe argument on the left is a convex combination, which belongs to \\(C\\) because the domain is convex. Equal weights give the familiar comparison between the function of an arithmetic average and the arithmetic average of the function values. The statement applies to scalar or vector inputs, while the output remains scalar. (stanford.edu)\n\nFor two points, this is precisely the defining condition for convexity:\n\n\\[f(tx+(1-t)y)\\leq tf(x)+(1-t)f(y), \\qquad 0\\leq t\\leq1.\\]\n\nGeometrically, the graph lies below the chord joining two graph points. The finite inequality follows by repeatedly applying this two-point condition, or by mathematical induction. Nonnegative, normalized weights are essential; arbitrary signed coefficients do not give the same theorem. (web.stanford.edu)"
    },
    {
      "heading": "Expectation and integral forms",
      "text": "Let \\(X\\) be an integrable random variable taking values in an open interval \\(I\\), and let \\(f:I\\to\\mathbb R\\) be convex. A standard finite-valued formulation assumes that \\(f(X)\\) is also integrable. Then\n\n\\[f(\\mathbb E[X])\\leq\\mathbb E[f(X)].\\]\n\nThe finite weighted version is recovered when the probability distribution assigns mass \\(\\lambda_i\\) to \\(x_i\\). No assumption of independence is involved. The distinction is between transforming a mean and taking the mean of a transformation; these operations generally do not commute. (mit.edu)\n\nIn measure-theoretic notation, for a probability space \\((\\Omega,\\mathcal F,\\mu)\\),\n\n\\[f\\!\\left(\\int_\\Omega X\\,d\\mu\\right) \\leq \\int_\\Omega f(X)\\,d\\mu.\\]\n\nThese integrals use a measure of total mass one. For a finite positive measure of mass \\(M>0\\), normalization introduces \\(1/M\\) on both sides. Integrability hypotheses matter: an undefined mean cannot simply be substituted into the formula. (stanford.edu)\n\nThere is also a conditional version. Under suitable integrability assumptions, for a sub-sigma-algebra \\(\\mathcal G\\),\n\n\\[f(\\mathbb E[X\\mid\\mathcal G]) \\leq \\mathbb E[f(X)\\mid\\mathcal G] \\quad\\text{almost surely}.\\]\n\nThus the same comparison holds for conditional expectations, with the inequality interpreted almost surely rather than necessarily at every outcome. (dspace.mit.edu)"
    },
    {
      "heading": "Proof and equality conditions",
      "text": "A useful proof uses a supporting line. Write \\(m=\\mathbb E[X]\\). If \\(f\\) is differentiable and convex on an open interval containing \\(m\\), its derivative gives\n\n\\[f(x)\\geq f(m)+f'(m)(x-m).\\]\n\nTaking expectations makes the last term vanish, because \\(\\mathbb E[X-m]=0\\), leaving Jensen’s inequality. The proof also explains its direction: convexity places the function above its tangent line. (live.ocw.mit.edu)\n\nDifferentiability is not essential. At an interior point of a finite convex function’s domain, an appropriate supporting slope can replace the derivative. In several dimensions, the analogous argument uses a supporting hyperplane and, for differentiable functions, the gradient. This connects Jensen’s inequality with the first-order characterization of convexity used in convex optimization. (web.stanford.edu)\n\nIf \\(f\\) is strictly convex, equality in the finite form holds exactly when all points with positive weight coincide. In the probabilistic form, equality holds exactly when \\(X\\) is constant almost surely. For a non-strictly convex function, equality can also occur across an interval on which the function is affine. An affine function gives equality for every admissible distribution. (cs229.stanford.edu)"
    },
    {
      "heading": "Examples and the Jensen gap",
      "text": "Taking \\(f(x)=x^2\\) gives, for a variable with finite second moment,\n\n\\[(\\mathbb E[X])^2\\leq\\mathbb E[X^2].\\]\n\nThe difference is the variance of \\(X\\). As a concrete two-point example, averaging \\(1\\) and \\(3\\) before squaring gives \\(4\\), whereas averaging their squares gives \\(5\\). This illustrates the effect without requiring any probability notation. (mit.edu)\n\nBecause the logarithm is concave, positive numbers satisfy\n\n\\[\\sum_i\\lambda_i\\log x_i \\leq \\log\\!\\left(\\sum_i\\lambda_i x_i\\right).\\]\n\nExponentiation yields the weighted arithmetic–geometric mean inequality. Conversely, the convex exponential function gives \\(e^{\\mathbb E[X]}\\leq\\mathbb E[e^X]\\) whenever the expectations are defined. (cs.cmu.edu)\n\nThe nonnegative difference\n\n\\[J_f(X)=\\mathbb E[f(X)]-f(\\mathbb E[X])\\]\n\nis called the Jensen gap. Beyond its sign, bounds on its magnitude can depend on the function’s growth or curvature and on moments describing the distribution’s dispersion. Such bounds distinguish a nearly tight inequality from one with a substantial gap. (arxiv.org)"
    },
    {
      "heading": "Applications and historical origin",
      "text": "In machine learning, logarithmic Jensen inequalities construct lower bounds on otherwise difficult likelihood expressions. For a discrete latent variable \\(z\\) and a positive auxiliary distribution \\(q\\),\n\n\\[\\log p(x) = \\log\\mathbb E_q\\!\\left[\\frac{p(x,z)}{q(z)}\\right] \\geq \\mathbb E_q\\!\\left[\\log\\frac{p(x,z)}{q(z)}\\right].\\]\n\nWith appropriate support assumptions, the right side is an evidence lower bound. This construction underlies variational inference and the expectation–maximization algorithm, where choosing the current conditional distribution of the latent variable makes the bound tight. (cs229.stanford.edu)\n\nThe inequality bears the name of Johan Ludwig William Valdemar Jensen. His 1906 paper, Sur les fonctions convexes et les inégalités entre les valeurs moyennes, developed convex-function inequalities and their relationships with classical inequalities between means. (cs.cmu.edu)"
    }
  ],
  "links": [
    "theorem",
    "convex-function",
    "expected-value",
    "convex-set",
    "real-number",
    "vector-space",
    "convex-combination",
    "mathematical-induction",
    "random-variable",
    "probability-distribution",
    "statistical-independence",
    "measure-theory",
    "probability-space",
    "integral",
    "sigma-algebra",
    "conditional-expectation",
    "almost-surely",
    "mathematical-proof",
    "derivative",
    "hyperplane",
    "gradient",
    "convex-optimization",
    "affine-map",
    "variance",
    "arithmetic-geometric-mean-inequality",
    "exponential-function",
    "machine-learning",
    "evidence-lower-bound",
    "variational-inference",
    "expectation-maximization-algorithm"
  ],
  "sources": [
    {
      "title": "web.stanford.edu/~boyd/cvxbook/bv_cvxslides.pdf",
      "url": "https://web.stanford.edu/~boyd/cvxbook/bv_cvxslides.pdf"
    },
    {
      "title": "stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf",
      "url": "https://www.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf"
    },
    {
      "title": "mit.edu/~9.520/spring06/Classes/mathcamp02.pdf",
      "url": "https://www.mit.edu/~9.520/spring06/Classes/mathcamp02.pdf"
    },
    {
      "title": "dspace.mit.edu/bitstream/handle/1721.1/103814/18-466-spring-2003/contents/lectu…",
      "url": "https://dspace.mit.edu/bitstream/handle/1721.1/103814/18-466-spring-2003/contents/lecture-notes/c2s3.pdf"
    },
    {
      "title": "live.ocw.mit.edu/courses/14-30-introduction-to-statistical-methods-in-economics…",
      "url": "https://live.ocw.mit.edu/courses/14-30-introduction-to-statistical-methods-in-economics-spring-2009/fe7f0d9c5833858fc1ac48230e6dbcd2_MIT14_30s09_lec11.pdf"
    },
    {
      "title": "cs229.stanford.edu/summer2022/cs229-notes8.pdf",
      "url": "https://cs229.stanford.edu/summer2022/cs229-notes8.pdf"
    },
    {
      "title": "cs.cmu.edu/~suvrit/teach/papers/1906_jensen_sur_les_fonctions_convexes_et_les_i…",
      "url": "https://www.cs.cmu.edu/~suvrit/teach/papers/1906_jensen_sur_les_fonctions_convexes_et_les_inegalites_entre_valeurs_moyennes.pdf"
    },
    {
      "title": "arxiv.org/abs/1712.05267",
      "url": "https://arxiv.org/abs/1712.05267"
    }
  ],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}