{
  "id": "laplaces-method",
  "title": "Laplace’s Method",
  "lang": "en",
  "source_lang": "en",
  "summary": "Laplace’s method approximates integrals with sharply concentrated exponential factors by expanding the exponent near its dominant maxima, usually yielding a Gaussian leading term.",
  "category": "mathematics",
  "subcategory": "analysis",
  "type": "concept",
  "aliases": [
    "Laplace approximation",
    "Method of Laplace"
  ],
  "generated_at": "2026-11-05",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/laplaces-method/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/laplaces-method/",
      "json": "https://aiwiki.page/zh/laplaces-method.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Laplace’s method is a technique in asymptotic analysis for approximating integrals whose integrands contain a large positive parameter in an exponential factor. For an integral of the form\n\n\\[I(\\lambda)=\\int_a^b g(x)e^{\\lambda f(x)}\\,dx, \\qquad \\lambda\\to+\\infty,\\]\n\nthe dominant contribution usually comes from neighborhoods where \\(f\\) attains its largest value. When there is a unique interior maximum with nonzero negative second derivative, replacing the exponent locally by a quadratic function yields a Gaussian integral and an explicit leading approximation. The method is named after Pierre-Simon Laplace and is widely used in mathematical analysis, probability, statistics, and mathematical physics."
    },
    {
      "heading": "Basic approximation",
      "text": "Suppose \\(f\\) is real-valued and has a unique global maximum at an interior point \\(x_0\\in(a,b)\\), with\n\n\\[f'(x_0)=0,\\qquad f''(x_0)<0.\\]\n\nIf \\(g\\) is continuous at \\(x_0\\), \\(g(x_0)\\ne0\\), and suitable smoothness and tail conditions hold, then\n\n\\[I(\\lambda)\\sim g(x_0)e^{\\lambda f(x_0)} \\sqrt{\\frac{2\\pi}{\\lambda[-f''(x_0)]}}.\\]\n\nHere \\(A(\\lambda)\\sim B(\\lambda)\\) means that \\(A(\\lambda)/B(\\lambda)\\to1\\).\n\nThe three factors have distinct roles:\n\n- \\(e^{\\lambda f(x_0)}\\) gives the exponential scale.\n- \\(g(x_0)\\) gives the amplitude at the dominant point.\n- \\(\\sqrt{2\\pi/(\\lambda[-f''(x_0)])}\\) measures the effective width of the peak.\n\nThus the peak contributes over a width of order \\(\\lambda^{-1/2}\\), with narrower concentration when the curvature \\(-f''(x_0)\\) is larger.\n\nAn equivalent minimum formulation is often convenient:\n\n\\[J(\\lambda)=\\int_a^b g(x)e^{-\\lambda h(x)}\\,dx.\\]\n\nIf \\(h\\) has a unique nondegenerate interior minimum at \\(x_0\\), then\n\n\\[J(\\lambda)\\sim g(x_0)e^{-\\lambda h(x_0)} \\sqrt{\\frac{2\\pi}{\\lambda h''(x_0)}}.\\]"
    },
    {
      "heading": "Why the method works",
      "text": "The local argument follows from a Taylor expansion:\n\n\\[f(x)=f(x_0) +\\frac12 f''(x_0)(x-x_0)^2 +o\\!\\left((x-x_0)^2\\right).\\]\n\nWriting \\(a=-f''(x_0)>0\\) and introducing the scaled variable\n\n\\[u=\\sqrt{\\lambda a}\\,(x-x_0)\\]\n\ngives, for bounded \\(u\\),\n\n\\[\\lambda[f(x)-f(x_0)] =-\\frac{u^2}{2}+o(1).\\]\n\nConsequently,\n\n\\[I(\\lambda)\\approx \\frac{g(x_0)e^{\\lambda f(x_0)}}{\\sqrt{\\lambda a}} \\int_{-\\infty}^{\\infty}e^{-u^2/2}\\,du.\\]\n\nThe integral is \\(\\sqrt{2\\pi}\\), the normalization underlying the normal distribution.\n\nThe global part of the argument establishes that regions away from \\(x_0\\) are negligible. On a compact interval, continuity and uniqueness of the maximum imply that, outside any fixed neighborhood of \\(x_0\\),\n\n\\[f(x)\\le f(x_0)-\\eta\\]\n\nfor some \\(\\eta>0\\). Contributions from that region are therefore exponentially smaller than the peak contribution.\n\nOn an unbounded domain, uniqueness of the maximum alone is insufficient: the tails also require control. One useful sufficient set of conditions is a strict gap below the maximum outside a bounded neighborhood, together with\n\n\\[\\int |g(x)|e^{\\lambda_0 f(x)}\\,dx<\\infty\\]\n\nfor some fixed \\(\\lambda_0>0\\). Such conditions justify discarding the tails and applying a local convergence argument."
    },
    {
      "heading": "Higher-order terms and error estimates",
      "text": "Additional smoothness produces corrections in inverse powers of \\(\\lambda\\). Under suitable regularity and localization assumptions,\n\n\\[I(\\lambda)= e^{\\lambda f(x_0)} \\sqrt{\\frac{2\\pi}{\\lambda a}} \\left[ g(x_0)+\\frac{C_1}{\\lambda} +O(\\lambda^{-2}) \\right], \\qquad a=-f''(x_0).\\]\n\nThe remainder is expressed using big-O notation. With all derivatives evaluated at \\(x_0\\),\n\n\\[C_1= \\frac{g''}{2a} +\\frac{g'f'''}{2a^2} +\\frac{g f''''}{8a^2} +\\frac{5g(f''')^2}{24a^3}.\\]\n\nThese terms account for variation in the amplitude, asymmetry of the peak, and departures from quadratic curvature. In particular, the leading approximation generally has relative error \\(O(\\lambda^{-1})\\) when \\(g(x_0)\\ne0\\) and the required assumptions hold.\n\nThe full expansion is generally asymptotic rather than convergent. Adding terms may improve accuracy initially without producing a convergent series at fixed \\(\\lambda\\). If \\(g(x_0)=0\\), the displayed leading contribution vanishes, and the first nonzero term must be found from a more detailed local expansion."
    },
    {
      "heading": "Multidimensional form",
      "text": "For a domain \\(D\\subseteq\\mathbb R^d\\), consider\n\n\\[I(\\lambda)=\\int_D g(x)e^{\\lambda f(x)}\\,dx.\\]\n\nSuppose \\(f\\) has a unique interior maximizer \\(x_0\\), its gradient vanishes there, and\n\n\\[A=-\\nabla^2 f(x_0)\\]\n\nis a positive-definite matrix. Here \\(\\nabla^2 f\\) is the Hessian matrix. Subject to suitable smoothness and global control,\n\n\\[I(\\lambda)\\sim g(x_0)e^{\\lambda f(x_0)} \\left(\\frac{2\\pi}{\\lambda}\\right)^{d/2} \\frac{1}{\\sqrt{\\det A}},\\]\n\nwhere \\(\\det A\\) is the determinant of \\(A\\).\n\nThe local approximation is a multivariate normal distribution centered at \\(x_0\\), with covariance matrix\n\n\\[\\Sigma=\\frac{1}{\\lambda}A^{-1}.\\]\n\nEach locally curved direction contributes a factor of order \\(\\lambda^{-1/2}\\); the determinant combines the curvature in all directions."
    },
    {
      "heading": "Example: Stirling’s approximation",
      "text": "An important application is Stirling’s approximation for factorials. The gamma function gives\n\n\\[n!=\\int_0^\\infty t^n e^{-t}\\,dt.\\]\n\nSubstituting \\(t=nx\\) yields\n\n\\[n!=n^{n+1}\\int_0^\\infty e^{n(\\log x-x)}\\,dx.\\]\n\nFor\n\n\\[f(x)=\\log x-x,\\]\n\nthe unique maximum occurs at \\(x_0=1\\), where\n\n\\[f(1)=-1,\\qquad f''(1)=-1.\\]\n\nLaplace’s method therefore gives\n\n\\[n!\\sim n^{n+1}e^{-n}\\sqrt{\\frac{2\\pi}{n}} = \\sqrt{2\\pi n}\\left(\\frac ne\\right)^n.\\]\n\nThe first correction is\n\n\\[n!=\\sqrt{2\\pi n}\\left(\\frac ne\\right)^n \\left[1+\\frac{1}{12n}+O(n^{-2})\\right].\\]"
    },
    {
      "heading": "Bayesian and statistical applications",
      "text": "Posterior approximation\n\nIn Bayesian inference, a posterior density has the form\n\n\\[p(\\theta\\mid y) =\\frac{p(y\\mid\\theta)p(\\theta)}{p(y)}.\\]\n\nDefine the log unnormalized posterior\n\n\\[\\ell(\\theta)=\\log p(y\\mid\\theta)+\\log p(\\theta).\\]\n\nIf \\(\\ell\\) has a well-concentrated interior maximum at \\(\\hat\\theta\\), with positive-definite negative Hessian\n\n\\[H=-\\nabla^2\\ell(\\hat\\theta),\\]\n\nthen\n\n\\[p(\\theta\\mid y)\\approx \\mathcal N_d(\\theta;\\hat\\theta,H^{-1}).\\]\n\nThe center \\(\\hat\\theta\\) is a maximum a posteriori estimate.\n\nIn regular, fixed-dimensional statistical models, increasing sample size often supplies the large parameter: the log likelihood is a sum over observations and its curvature typically grows with sample size. The approximation is not automatically justified for arbitrary data sets or models.\n\nMarginal likelihood and expectations\n\nThe marginal likelihood is the normalizing integral\n\n\\[p(y)=\\int p(y\\mid\\theta)p(\\theta)\\,d\\theta.\\]\n\nUsing the same notation,\n\n\\[p(y)\\approx e^{\\ell(\\hat\\theta)} (2\\pi)^{d/2}(\\det H)^{-1/2}.\\]\n\nPosterior expectations can likewise be approximated as ratios of integrals:\n\n\\[\\mathbb E[h(\\theta)\\mid y] = \\frac{\\int h(\\theta)e^{\\ell(\\theta)}\\,d\\theta} {\\int e^{\\ell(\\theta)}\\,d\\theta}.\\]\n\nThe simplest leading approximation is \\(h(\\hat\\theta)\\), when the numerator and denominator have compatible dominant behavior. Higher-order methods capture posterior spread and asymmetry.\n\nThese deterministic approximations can be computationally cheaper than Markov chain Monte Carlo, but their accuracy depends strongly on the shape of the posterior and on finding the relevant modes."
    },
    {
      "heading": "Multiple peaks, boundaries, and degeneracy",
      "text": "Several dominant maxima\n\nIf finitely many isolated, nondegenerate interior maxima \\(x_1,\\ldots,x_k\\) share the same global height, their leading contributions add:\n\n\\[I(\\lambda)\\sim \\sum_{j=1}^k g(x_j)e^{\\lambda f(x_j)} \\sqrt{\\frac{2\\pi}{\\lambda[-f''(x_j)]}},\\]\n\nprovided the summed leading term does not cancel.\n\nLower local maxima are exponentially suppressed when their height gap remains fixed as \\(\\lambda\\to\\infty\\). Approximating a multimodal integral using only one peak can therefore miss a substantial contribution when several peaks have comparable heights.\n\nBoundary maxima\n\nIf the maximum occurs at the left endpoint and \\(f'(a)<0\\), the local exponent is linear rather than quadratic. Under suitable conditions,\n\n\\[I(\\lambda)\\sim \\frac{g(a)e^{\\lambda f(a)}}{\\lambda[-f'(a)]}.\\]\n\nThe leading power is \\(\\lambda^{-1}\\), not \\(\\lambda^{-1/2}\\).\n\nIf the endpoint is stationary, with \\(f'(a)=0\\) and \\(f''(a)<0\\), a smooth one-sided neighborhood contributes half the corresponding full Gaussian leading term.\n\nDegenerate maxima\n\nIf \\(f''(x_0)=0\\), the Gaussian formula generally fails. Suppose instead that\n\n\\[f(x)=f(x_0)-c(x-x_0)^{2m} +o\\!\\left(|x-x_0|^{2m}\\right), \\qquad c>0.\\]\n\nFor an interior maximum with contributions from both sides,\n\n\\[I(\\lambda)\\sim g(x_0)e^{\\lambda f(x_0)} \\frac{\\Gamma(1/(2m))} {m(\\lambda c)^{1/(2m)}}.\\]\n\nThe concentration width is then \\(\\lambda^{-1/(2m)}\\). A continuum of maxima, rather than isolated points, requires a different analysis that separates directions along the maximizing set from directions transverse to it."
    },
    {
      "heading": "Related methods and limitations",
      "text": "The method of stationary phase treats oscillatory integrals such as\n\n\\[\\int g(x)e^{i\\lambda f(x)}\\,dx.\\]\n\nThere, cancellation replaces exponential suppression, and stationary points need not be maxima. The method of steepest descent extends related ideas to complex integrals by deforming contours through suitable saddle points.\n\nLaplace’s method is distinct from the Laplace transform, although it can approximate such transforms in appropriate parameter limits.\n\nPractical implementation combines optimization with curvature calculations. Logarithms help avoid numerical overflow, while a Cholesky decomposition can evaluate the log determinant of a positive-definite Hessian and solve the required linear systems without explicitly forming its inverse.\n\nImportant limitations include:\n\n- Weak concentration: a local quadratic approximation may not represent much of the integral at moderate parameter values.\n- Nearly singular curvature: flat directions make the Gaussian scale unreliable.\n- Nonuniform behavior: moving or merging maxima can invalidate an approximation derived for fixed parameters.\n- Increasing dimension: fixed-dimensional error estimates need not remain small when dimension grows with sample size.\n- Reparameterization: a density’s mode and local Gaussian form depend on coordinates. Consistently transformed integral expansions preserve the integral’s asymptotic value, but finite-order Gaussian posterior approximations need not agree exactly across parameterizations.\n\nWhen maxima merge or approach a boundary, specialized uniform approximations may be necessary to describe the transition accurately."
    }
  ],
  "links": [
    "asymptotic-expansion",
    "integral",
    "pierre-simon-laplace",
    "taylor-series",
    "normal-distribution",
    "big-o-notation",
    "gradient",
    "positive-definite-matrix",
    "hessian-matrix",
    "determinant",
    "multivariate-normal-distribution",
    "covariance-matrix",
    "stirlings-approximation",
    "gamma-function",
    "bayesian-inference",
    "posterior-distribution",
    "maximum-a-posteriori-estimation",
    "marginal-likelihood",
    "expected-value",
    "markov-chain-monte-carlo",
    "gradient-descent",
    "laplace-transform",
    "mathematical-optimization",
    "cholesky-decomposition"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}