{
  "id": "empirical-distribution-function",
  "title": "Empirical Distribution Function",
  "lang": "en",
  "source_lang": "en",
  "summary": "The empirical distribution function records the fraction of sample observations at or below each value, providing a nonparametric estimate of a population’s cumulative distribution function.",
  "category": "mathematics",
  "subcategory": "probability-statistics",
  "type": "concept",
  "aliases": [
    "Empirical cumulative distribution function",
    "ECDF",
    "EDF",
    "Sample distribution function"
  ],
  "generated_at": "2026-11-05",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/empirical-distribution-function/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/empirical-distribution-function/",
      "json": "https://aiwiki.page/zh/empirical-distribution-function.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "The empirical distribution function is the cumulative distribution function obtained by assigning equal probability to every observation in a sample. At any real number \\(x\\), it gives the proportion of observations less than or equal to \\(x\\). In statistics, it is a fundamental nonparametric estimator of a population distribution: it requires no assumption that the population belongs to a particular family, such as the normal or exponential distributions."
    },
    {
      "heading": "Definition and construction",
      "text": "For real-valued observations \\(X_1,\\ldots,X_n\\), where \\(n\\geq 1\\), the empirical distribution function is\n\n\\[F_n(x)=\\frac{1}{n}\\sum_{i=1}^{n}\\mathbf{1}_{\\{X_i\\leq x\\}}, \\qquad x\\in\\mathbb{R},\\]\n\nwhere \\(\\mathbf{1}_{\\{X_i\\leq x\\}}\\) is an indicator function equal to \\(1\\) if \\(X_i\\leq x\\), and \\(0\\) otherwise.\n\nThe definition applies to any finite sample. Its usual statistical properties assume that the observations are independent and identically distributed, with common population distribution function\n\n\\[F(x)=\\Pr(X_i\\leq x).\\]\n\nBefore the data are observed, \\(F_n\\) is a random function; after observation, it is a fixed step function.\n\nExample\n\nFor the sample \\(2,3,3,7\\),\n\n\\[F_4(x)= \\begin{cases} 0, & x<2,\\\\ 1/4, & 2\\leq x<3,\\\\ 3/4, & 3\\leq x<7,\\\\ 1, & x\\geq 7. \\end{cases}\\]\n\nThe jump at \\(3\\) has size \\(2/4\\), because two observations equal \\(3\\). In particular, \\(F_4(3)=3/4\\): observations equal to the threshold are included."
    },
    {
      "heading": "Structural properties",
      "text": "Every empirical distribution function is:\n\n- nondecreasing;\n- right-continuous;\n- equal to \\(0\\) below the smallest observation;\n- equal to \\(1\\) at and above the largest observation.\n\nAt a distinct observed value \\(a\\), its jump is\n\n\\[F_n(a)-F_n(a^-) = \\frac{\\#\\{i:X_i=a\\}}{n},\\]\n\nwhere \\(F_n(a^-)\\) denotes the left-hand limit. Thus, an observation appearing \\(m\\) times produces a jump of size \\(m/n\\).\n\nIf the sorted observations, or order statistics, are denoted by\n\n\\[X_{(1)}\\leq X_{(2)}\\leq\\cdots\\leq X_{(n)},\\]\n\nthen, in the absence of ties,\n\n\\[F_n(x)=\\frac{k}{n} \\quad\\text{for}\\quad X_{(k)}\\leq x<X_{(k+1)}.\\]\n\nWith ties, some intermediate levels are skipped.\n\nThe empirical distribution is discrete even when the population distribution is continuous. Consequently, the ordinary derivative of the ECDF is zero between jumps and undefined at jumps; it is not a probability density function representing the empirical distribution."
    },
    {
      "heading": "Empirical measure and sample summaries",
      "text": "The distribution underlying the ECDF is the empirical measure\n\n\\[P_n=\\frac{1}{n}\\sum_{i=1}^{n}\\delta_{X_i},\\]\n\nwhere \\(\\delta_a\\) assigns probability \\(1\\) to the single point \\(a\\). The ECDF is its cumulative representation:\n\n\\[F_n(x)=P_n((-\\infty,x]).\\]\n\nFor a real-valued function \\(g\\) that is finite at the observations,\n\n\\[\\int g(t)\\,dF_n(t) = \\frac{1}{n}\\sum_{i=1}^{n}g(X_i).\\]\n\nThus, integration against the empirical distribution turns population expectations into sample averages. For example, its expected value is the sample mean,\n\n\\[\\int t\\,dF_n(t)=\\overline X.\\]\n\nIts variance is\n\n\\[\\int (t-\\overline X)^2\\,dF_n(t) = \\frac{1}{n}\\sum_{i=1}^{n}(X_i-\\overline X)^2.\\]\n\nThis is the variance of the empirical probability distribution, not the usual unbiased estimator of population variance, which uses denominator \\(n-1\\).\n\nThe ECDF also determines empirical interval probabilities:\n\n\\[P_n((a,b])=F_n(b)-F_n(a).\\]\n\nThe interval endpoints matter because the empirical distribution places positive probability on observed values."
    },
    {
      "heading": "Sampling properties",
      "text": "Assume throughout this section that the observations are independent and identically distributed with distribution function \\(F\\).\n\nPointwise expectation and variance\n\nFor each fixed \\(x\\), the count \\(nF_n(x)\\) has a binomial distribution:\n\n\\[nF_n(x)\\sim\\operatorname{Binomial}(n,F(x)).\\]\n\nIt follows that\n\n\\[\\mathbb E[F_n(x)]=F(x), \\qquad \\operatorname{Var}(F_n(x)) = \\frac{F(x)(1-F(x))}{n}.\\]\n\nHence the ECDF is pointwise unbiased. Its standard error at \\(x\\) is at most \\(1/(2\\sqrt n)\\), with the maximum attained when \\(F(x)=1/2\\).\n\nValues of the ECDF at different thresholds are dependent. Their covariance is\n\n\\[\\operatorname{Cov}(F_n(x),F_n(y)) = \\frac{F(\\min\\{x,y\\})-F(x)F(y)}{n}.\\]\n\nPointwise convergence and normal approximation\n\nBy the law of large numbers, for each fixed \\(x\\),\n\n\\[F_n(x)\\longrightarrow F(x) \\quad\\text{almost surely}.\\]\n\nWhen \\(0<F(x)<1\\), the central limit theorem gives\n\n\\[\\sqrt n\\bigl(F_n(x)-F(x)\\bigr) \\xrightarrow{d} N\\!\\left(0,F(x)(1-F(x))\\right).\\]\n\nThis is a statement about a fixed threshold. It does not by itself provide simultaneous uncertainty control across all thresholds."
    },
    {
      "heading": "Uniform convergence and uncertainty bands",
      "text": "Glivenko–Cantelli theorem\n\nThe Glivenko–Cantelli theorem, established in 1933, strengthens pointwise convergence to uniform convergence:\n\n\\[\\sup_{x\\in\\mathbb R}|F_n(x)-F(x)| \\longrightarrow 0 \\quad\\text{almost surely}.\\]\n\nThus, with probability one, the largest discrepancy anywhere between the empirical and population distribution functions tends to zero. The result applies to continuous, discrete, and mixed distributions.\n\nDvoretzky–Kiefer–Wolfowitz inequality\n\nThe Dvoretzky–Kiefer–Wolfowitz inequality, with its sharp constant, provides a finite-sample bound:\n\n\\[\\Pr\\!\\left( \\sup_x|F_n(x)-F(x)|>\\varepsilon \\right) \\leq 2e^{-2n\\varepsilon^2}, \\qquad \\varepsilon>0.\\]\n\nFor \\(0<\\alpha<1\\), define\n\n\\[\\varepsilon_{n,\\alpha} = \\sqrt{\\frac{\\log(2/\\alpha)}{2n}}.\\]\n\nWith probability at least \\(1-\\alpha\\), the inequalities\n\n\\[\\max\\{0,F_n(x)-\\varepsilon_{n,\\alpha}\\} \\leq F(x)\\leq \\min\\{1,F_n(x)+\\varepsilon_{n,\\alpha}\\}\\]\n\nhold simultaneously for every \\(x\\). These are simultaneous confidence bands, rather than separate pointwise confidence intervals.\n\nEmpirical-process limit\n\nAt the process level, the centered and rescaled ECDF\n\n\\[\\alpha_n(x)=\\sqrt n\\bigl(F_n(x)-F(x)\\bigr)\\]\n\nconverges in distribution, in the standard empirical-process formulation, to\n\n\\[B(F(x)),\\]\n\nwhere \\(B\\) is a Brownian bridge: a mean-zero Gaussian process on \\([0,1]\\) with covariance\n\n\\[\\operatorname{Cov}(B(s),B(t)) = \\min(s,t)-st.\\]\n\nThis functional limit describes fluctuations of the entire estimated distribution and underlies many distributional tests."
    },
    {
      "heading": "Applications",
      "text": "Distribution comparison and goodness-of-fit testing\n\nAn ECDF plot displays location, spread, ties, and tail behavior without choosing histogram bins. Multiple ECDFs can be plotted together to compare samples.\n\nThe one-sample Kolmogorov–Smirnov test uses the maximum discrepancy from a specified distribution function \\(F_0\\):\n\n\\[D_n=\\sup_x|F_n(x)-F_0(x)|.\\]\n\nFor an independent sample under a fully specified continuous null distribution, its null distribution does not depend on the particular form of \\(F_0\\). This distribution-free property generally does not hold unchanged for discrete null distributions or when parameters of \\(F_0\\) are estimated from the tested sample.\n\nThe two-sample version compares two ECDFs, \\(F_n\\) and \\(G_m\\), through\n\n\\[D_{n,m}=\\sup_x|F_n(x)-G_m(x)|.\\]\n\nQuantile estimation\n\nA natural empirical quantile function is the generalized inverse\n\n\\[Q_n(p)=\\inf\\{x:F_n(x)\\geq p\\}, \\qquad 0<p\\leq1.\\]\n\nIt satisfies\n\n\\[Q_n(p)=X_{(\\lceil np\\rceil)}.\\]\n\nSoftware also uses interpolated quantile definitions. These may return values between observations and need not coincide with the generalized inverse of the ECDF.\n\nBootstrap resampling\n\nIn the ordinary nonparametric bootstrap, observations are sampled independently with replacement from the empirical distribution. Conditional on the original data, each indexed observation receives probability \\(1/n\\). Repeated observed values therefore receive probability equal to their relative frequency.\n\nThis makes the empirical distribution a substitute for the unknown population distribution when approximating the sampling behavior of many statistics. Bootstrap validity still depends on the statistic and the sampling assumptions."
    },
    {
      "heading": "Variants and limitations",
      "text": "A weighted empirical distribution function has the form\n\n\\[F_w(x)= \\frac{\\sum_{i=1}^{n}w_i\\mathbf{1}_{\\{X_i\\leq x\\}}} {\\sum_{i=1}^{n}w_i},\\]\n\nfor nonnegative weights with positive total weight. Such functions arise with unequal sampling probabilities and importance weighting. Their uncertainty is not generally described by the ordinary unweighted formulas.\n\nFor vector-valued observations, a multivariate ECDF counts observations whose every coordinate is at most the corresponding threshold. It estimates a joint distribution, but its geometry and distributional theory differ from the univariate case.\n\nSeveral limitations are important:\n\n- Dependence: serially or otherwise dependent observations invalidate the simple binomial variance formula and can change convergence rates and limiting distributions.\n- Sampling bias: the ECDF describes the observed sample; it does not automatically correct selection bias, measurement error, or missing data.\n- Tail resolution: it assigns no probability below the observed minimum or above the observed maximum. Reliable extrapolation beyond the sample range requires additional assumptions.\n- Censoring and truncation: applying the ordinary ECDF to incompletely observed values generally does not estimate the distribution of the underlying uncensored variable.\n- Lack of smoothness: the ECDF estimates cumulative probabilities directly, not a smooth density. Density estimation requires a different construction, often with additional smoothing choices.\n\nComputationally, sorting the sample permits a compact representation by distinct observed values and their cumulative counts. Subsequent evaluations can use binary search, while empirical quantiles can be obtained directly from the sorted observations."
    }
  ],
  "links": [
    "cumulative-distribution-function",
    "statistics",
    "estimator",
    "indicator-function",
    "derivative",
    "probability-density-function",
    "expected-value",
    "sample-mean",
    "binomial-distribution",
    "standard-error",
    "covariance",
    "law-of-large-numbers",
    "central-limit-theorem",
    "uniform-convergence",
    "confidence-interval",
    "kolmogorov-smirnov-test",
    "quantile",
    "bootstrap-sampling",
    "joint-probability-distribution",
    "selection-bias",
    "density-estimation"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}