{
  "id": "inverse-transform-sampling",
  "title": "Inverse Transform Sampling",
  "lang": "en",
  "source_lang": "en",
  "summary": "Inverse transform sampling generates random values from a specified distribution by applying its quantile function to uniformly distributed random numbers.",
  "category": "mathematics",
  "subcategory": "probability-statistics",
  "type": "concept",
  "aliases": [
    "Inverse CDF sampling",
    "inversion sampling",
    "inverse transformation method",
    "quantile sampling"
  ],
  "generated_at": "2026-11-05",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/inverse-transform-sampling/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/inverse-transform-sampling/",
      "json": "https://aiwiki.page/zh/inverse-transform-sampling.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Inverse transform sampling is a method of generating a random variable with a specified probability distribution by applying that distribution’s quantile function to a uniformly distributed random number. It works for continuous, discrete, and mixed distributions when the inverse is interpreted appropriately. The method is a basic tool in random sampling and Monte Carlo methods, both as a practical algorithm and as a mathematical construction showing how arbitrary real-valued distributions can be obtained from uniform randomness."
    },
    {
      "heading": "Mathematical definition",
      "text": "Let \\(F\\) be the cumulative distribution function of a real-valued random variable:\n\n\\[F(x)=\\Pr(X\\le x).\\]\n\nFor \\(0<u<1\\), define its generalized inverse, or quantile function, by\n\n\\[Q(u)=\\inf\\{x\\in\\mathbb{R}:F(x)\\ge u\\}.\\]\n\nIf \\(U\\) has the uniform distribution on \\((0,1)\\), then\n\n\\[X=Q(U)\\]\n\nhas cumulative distribution function \\(F\\).\n\nWhen \\(F\\) is continuous and strictly increasing, \\(Q\\) is its ordinary inverse function, and the construction is usually written\n\n\\[X=F^{-1}(U).\\]\n\nThe generalized-inverse definition is essential when \\(F\\) has jumps or flat portions. Such functions need not have ordinary inverses, but they still define valid probability distributions."
    },
    {
      "heading": "Why the method works",
      "text": "The central identity is\n\n\\[Q(u)\\le x \\quad\\Longleftrightarrow\\quad u\\le F(x), \\qquad 0<u<1.\\]\n\nThe monotonicity and right-continuity of a cumulative distribution function ensure that this equivalence holds even at jumps. Consequently,\n\n\\[\\Pr(Q(U)\\le x) = \\Pr(U\\le F(x)) = F(x).\\]\n\nThus \\(Q(U)\\) has exactly the desired distribution.\n\nGeometrically, the method selects a random height \\(u\\) between zero and one and finds the leftmost horizontal coordinate at which the cumulative distribution reaches that height. A jump in \\(F\\) corresponds to an interval of uniform inputs that all produce the same output, creating a point mass. A flat interval in \\(F\\) corresponds to a gap that the generated variable crosses without taking values in its interior.\n\nRelationship to the probability integral transform\n\nThe probability integral transform proceeds in the opposite direction. If \\(X\\) has a continuous cumulative distribution function \\(F\\), then\n\n\\[F(X)\\sim \\operatorname{Uniform}(0,1).\\]\n\nContinuity is important: if \\(X\\) is discrete, \\(F(X)\\) is generally not uniform. Inverse transform sampling, by contrast, remains valid for discrete distributions because it uses the generalized inverse."
    },
    {
      "heading": "Basic algorithm",
      "text": "For a target distribution with cumulative distribution function \\(F\\):\n\n- Generate \\(u\\) uniformly from \\((0,1)\\).\n- Compute \\(x=Q(u)\\).\n- Return \\(x\\).\n\nRepeating this procedure with independent uniform inputs produces independent samples from the target distribution. Applying the same deterministic function separately to each input preserves statistical independence; it does not create independence if the inputs are dependent.\n\nThe practical cost depends primarily on how the quantile function is evaluated. Some distributions have simple inverse formulas. Others require numerical approximation or a search through cumulative probabilities."
    },
    {
      "heading": "Continuous examples",
      "text": "Uniform distribution on an interval\n\nFor a uniform distribution on \\([a,b]\\), where \\(a<b\\),\n\n\\[F(x)=\\frac{x-a}{b-a}, \\qquad a\\le x\\le b.\\]\n\nTherefore,\n\n\\[Q(u)=a+(b-a)u,\\]\n\nand a sample is obtained by the affine transformation\n\n\\[X=a+(b-a)U.\\]\n\nExponential distribution\n\nFor the exponential distribution with rate \\(\\lambda>0\\),\n\n\\[F(x)=1-e^{-\\lambda x}, \\qquad x\\ge0.\\]\n\nSolving \\(u=F(x)\\) gives\n\n\\[Q(u)=-\\frac{\\log(1-u)}{\\lambda}.\\]\n\nHence\n\n\\[X=-\\frac{\\log(1-U)}{\\lambda}.\\]\n\nSince \\(1-U\\) is also uniformly distributed, the alternative expression\n\n\\[X=-\\frac{\\log U}{\\lambda}\\]\n\nhas the same distribution. The two formulas are equal in distribution, but they do not give the same output for a fixed input \\(u\\).\n\nNormal distribution\n\nFor the standard normal distribution, write the cumulative distribution function as \\(\\Phi\\). Then\n\n\\[Z=\\Phi^{-1}(U)\\]\n\nis standard normal, and\n\n\\[X=\\mu+\\sigma Z, \\qquad \\sigma>0,\\]\n\nhas mean \\(\\mu\\) and standard deviation \\(\\sigma\\).\n\nThe normal quantile function has no elementary closed-form expression. Practical implementations use numerical approximations, often with different formulas for the central region and the tails."
    },
    {
      "heading": "Discrete distributions",
      "text": "Suppose a distribution takes ordered values\n\n\\[x_1<x_2<\\cdots<x_n\\]\n\nwith probabilities \\(p_1,\\ldots,p_n\\), where\n\n\\[p_i\\ge0, \\qquad \\sum_{i=1}^{n}p_i=1.\\]\n\nDefine the cumulative probabilities\n\n\\[c_0=0, \\qquad c_k=\\sum_{i=1}^{k}p_i.\\]\n\nInverse transform sampling selects the smallest index \\(k\\) such that\n\n\\[c_k\\ge U,\\]\n\nand returns \\(x_k\\). Equivalently, \\(x_k\\) is selected when\n\n\\[c_{k-1}<U\\le c_k.\\]\n\nThe length of this interval is \\(p_k\\), so the output has the required probability mass function.\n\nFor example, probabilities \\(0.2\\), \\(0.5\\), and \\(0.3\\) produce cumulative probabilities \\(0.2\\), \\(0.7\\), and \\(1\\). Uniform inputs in \\((0,0.2]\\), \\((0.2,0.7]\\), and \\((0.7,1)\\) select the corresponding outcomes.\n\nFor \\(n\\) tabulated outcomes, constructing cumulative probabilities takes \\(O(n)\\) time. Each subsequent sample can use a linear scan or binary search; binary search takes \\(O(\\log n)\\) comparisons. These bounds use big O notation and assume the cumulative table is already available.\n\nThe same principle applies to countably infinite distributions, such as the Poisson distribution, although searching through many cumulative probabilities may be expensive."
    },
    {
      "heading": "Numerical implementation",
      "text": "Numerical inversion\n\nWhen \\(F\\) can be evaluated but \\(Q\\) is unavailable, sampling can be reduced to finding where \\(F\\) reaches a specified value \\(u\\).\n\nFor a continuous, strictly increasing \\(F\\), this means solving\n\n\\[F(x)-u=0.\\]\n\nA bracketing method such as bisection is robust when an interval containing the solution is known. If the distribution has a probability density function \\(f\\), Newton’s method gives the iteration\n\n\\[x_{k+1} = x_k-\\frac{F(x_k)-u}{f(x_k)}.\\]\n\nNewton’s method can converge rapidly, but may fail when the initial value is poor, the density is small, or an iteration leaves the support. Safeguarded implementations combine fast iterations with a maintained bracket.\n\nFor discontinuous distributions, solving the equality \\(F(x)=u\\) is not generally appropriate: the cumulative distribution may jump over \\(u\\). The task is instead to locate the first point where \\(F(x)\\ge u\\).\n\nApproximation and interpolation\n\nFor repeated sampling, a quantile function may be approximated using tables, piecewise polynomials, or interpolation. The approximation should preserve monotonicity and account for tail behavior.\n\nA useful measure of inversion error is the probability-scale residual\n\n\\[|F(\\widehat Q(u))-u|.\\]\n\nFor continuous distributions, this describes how closely the approximate output corresponds to its intended cumulative probability. It is not the same as the value error \\(|\\widehat Q(u)-Q(u)|\\).\n\nWhere \\(F\\) is differentiable and its density is positive, the derivative of the quantile function is\n\n\\[Q'(u)=\\frac{1}{f(Q(u))}.\\]\n\nConsequently, small errors in probability can produce large errors in the sampled value where the density is small, especially in the tails.\n\nFinite precision and endpoints\n\nAn ideal continuous uniform variable equals neither zero nor one with positive probability. Computer-generated uniforms, however, lie on a finite grid whose endpoint convention depends on the implementation.\n\nEndpoint handling matters because quantile functions may be infinite there. For example, \\(-\\log U\\) is undefined at \\(U=0\\), while the normal quantile tends to opposite infinities as \\(u\\) approaches zero or one.\n\nFloating-point arithmetic also limits the probabilities and tail values that can be represented. A finite grid of uniform inputs cannot reproduce an unbounded continuous distribution exactly.\n\nNumerical stability matters even when an inverse formula is available. For the exponential inverse, evaluating \\(\\log(1-u)\\) with a dedicated log1p(-u) routine is typically more accurate when \\(u\\) is small than explicitly forming \\(1-u\\) and then taking its logarithm."
    },
    {
      "heading": "Extensions and applications",
      "text": "Truncated distributions\n\nInverse sampling adapts naturally to truncation. To sample from the distribution of \\(X\\) conditional on\n\n\\[a<X\\le b,\\]\n\nprovided \\(F(b)>F(a)\\), generate\n\n\\[V=F(a)+U\\bigl(F(b)-F(a)\\bigr)\\]\n\nand return\n\n\\[Q(V).\\]\n\nThis restricts the uniform draw to the cumulative-probability interval corresponding to the conditioning event. Different inclusion conventions at the boundaries require care when the original distribution has point masses.\n\nCoupled simulation\n\nUsing one uniform variable with several quantile functions produces dependent samples with prescribed marginal distributions. For example,\n\n\\[X=Q_X(U), \\qquad Y=Q_Y(U)\\]\n\ncouples both outputs monotonically to the same input.\n\nSuch constructions are useful for comparing simulated systems and for establishing stochastic-ordering results. Using shared random inputs may reduce the variance of a simulated difference, but variance reduction depends on the quantities being compared and is not automatic.\n\nMultivariate sampling\n\nA joint probability distribution generally cannot be sampled by independently applying each marginal quantile function: that would discard its dependence structure.\n\nAn extension uses successive conditional distributions. For a bivariate distribution, one may generate independent uniforms \\(U_1,U_2\\) and set\n\n\\[X_1=Q_1(U_1), \\qquad X_2=Q_{2\\mid X_1}(U_2).\\]\n\nHere \\(Q_{2\\mid X_1}\\) is the conditional quantile function evaluated at the generated value of \\(X_1\\). The construction extends to more variables, but computing the required conditional distributions may be difficult."
    },
    {
      "heading": "Advantages and limitations",
      "text": "Inverse transform sampling has a direct mathematical justification, works with arbitrary real-valued distributions, and uses one uniform input per univariate output in its basic form. Its monotone mapping is particularly useful when the relationship between inputs and outputs matters, rather than only the output distribution.\n\nIts main limitation is computational: evaluating the quantile function may be substantially harder than evaluating a density or cumulative distribution. Repeated numerical inversion can be expensive, discrete searches can be slow for large supports, and accurate tail sampling requires careful numerical design.\n\nAlternative methods, including rejection sampling and specialized distribution-specific algorithms, may be faster when inversion is difficult. Their relative efficiency depends on the distribution, required accuracy, preprocessing cost, and number of samples."
    }
  ],
  "links": [
    "random-variable",
    "probability-distribution",
    "random-sampling",
    "monte-carlo-method",
    "cumulative-distribution-function",
    "quantile",
    "uniform-distribution",
    "inverse-function",
    "statistical-independence",
    "exponential-distribution",
    "normal-distribution",
    "standard-deviation",
    "probability-mass-function",
    "big-o-notation",
    "poisson-distribution",
    "probability-density-function",
    "newtons-method",
    "interpolation",
    "derivative",
    "floating-point-arithmetic",
    "numerical-stability",
    "variance",
    "joint-probability-distribution",
    "rejection-sampling"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}