{
  "id": "consistent-estimator",
  "title": "Consistent Estimator",
  "lang": "en",
  "source_lang": "en",
  "summary": "A consistent estimator converges in probability to its target parameter as the sample size tends to infinity.",
  "category": "mathematics",
  "subcategory": "probability-statistics",
  "type": "concept",
  "aliases": [
    "Asymptotically consistent estimator",
    "weakly consistent estimator"
  ],
  "generated_at": "2026-10-04",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/consistent-estimator/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/consistent-estimator/",
      "json": "https://aiwiki.page/zh/consistent-estimator.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "In statistics, a consistent estimator is an estimator whose error converges to zero in probability as the sample size increases without bound. For any fixed positive tolerance, the probability that the estimate differs from its target by more than that tolerance tends to zero. Consistency describes large-sample behavior, rather than guaranteeing accuracy for any particular finite sample. Strictly, it is a property of a sequence of estimators indexed by sample size. (ocw.mit.edu)"
    },
    {
      "heading": "Formal definition",
      "text": "Let \\(X_1,\\ldots,X_n\\) be observations from a statistical model with parameter \\(\\theta\\in\\Theta\\), and let\n\n\\[\\widehat{\\theta}_n=T_n(X_1,\\ldots,X_n)\\]\n\nestimate \\(\\theta\\). The sequence is consistent at \\(\\theta\\) if, for every \\(\\varepsilon>0\\),\n\n\\[\\lim_{n\\to\\infty} P_\\theta\\!\\left( |\\widehat{\\theta}_n-\\theta|>\\varepsilon \\right)=0.\\]\n\nHere \\(P_\\theta\\) denotes probability under the model with parameter \\(\\theta\\). This is written\n\n\\[\\widehat{\\theta}_n\\xrightarrow{P_\\theta}\\theta,\\]\n\nusing the notation for convergence in probability. An estimator is consistent on \\(\\Theta\\) if this condition holds for every fixed \\(\\theta\\in\\Theta\\). When the target is a function \\(g(\\theta)\\), the definition replaces \\(\\theta\\) by \\(g(\\theta)\\). (ocw.mit.edu)\n\nFor vector-valued parameters, absolute error is replaced by a norm, or more generally by distance in a metric space. Consistency concerns concentration near the target; it does not require the realized error to decrease whenever another observation is added. (ocw.mit.edu)"
    },
    {
      "heading": "Forms of consistency",
      "text": "Weak consistency is the convergence-in-probability definition above. Unless otherwise specified, “consistent” ordinarily means weakly consistent. Strong consistency instead requires convergence almost surely:\n\n\\[P_\\theta\\!\\left( \\lim_{n\\to\\infty}\\widehat{\\theta}_n=\\theta \\right)=1.\\]\n\nStrong consistency implies weak consistency, but the converse does not hold in general. (statlect.com)\n\nMean-square consistency requires\n\n\\[E_\\theta\\!\\left[ (\\widehat{\\theta}_n-\\theta)^2 \\right]\\longrightarrow0.\\]\n\nIt implies weak consistency because\n\n\\[P_\\theta(|\\widehat{\\theta}_n-\\theta|>\\varepsilon) \\leq \\frac{E_\\theta[(\\widehat{\\theta}_n-\\theta)^2]} {\\varepsilon^2}.\\]\n\nThis is stronger than merely requiring small errors with high probability: rare, very large errors can prevent mean squared error from approaching zero. (bpb-us-w2.wpmucdn.com)\n\nUniform consistency strengthens the quantifier over parameter values:\n\n\\[\\sup_{\\theta\\in\\Theta} P_\\theta(|\\widehat{\\theta}_n-\\theta|>\\varepsilon) \\longrightarrow0.\\]\n\nOrdinary consistency applies separately to each fixed parameter; uniform consistency controls the worst-case error probability over the entire parameter space. This is a different distinction from weak versus strong convergence. (ocw.mit.edu)"
    },
    {
      "heading": "Consistency, bias, and mean squared error",
      "text": "Bias concerns the difference between an estimator’s expected value and its target:\n\n\\[\\operatorname{Bias}_\\theta(\\widehat{\\theta}_n) =E_\\theta[\\widehat{\\theta}_n]-\\theta.\\]\n\nUnbiasedness requires this difference to be zero at each sample size. Consistency instead concerns a limit of error probabilities. Neither property, by itself, implies the other. (bpb-us-w2.wpmucdn.com)\n\nFor example, suppose \\(X_1,X_2,\\ldots\\) are independent observations with mean \\(\\mu\\) and positive finite variance. The estimator \\(T_n=X_1\\) is unbiased for \\(\\mu\\), but its distribution does not change with \\(n\\); it therefore does not become concentrated at \\(\\mu\\). This illustrates why unbiasedness alone cannot establish consistency. (bpb-us-w2.wpmucdn.com)\n\nWhen the second moment exists, mean squared error satisfies\n\n\\[E_\\theta[(\\widehat{\\theta}_n-\\theta)^2] = \\operatorname{Var}_\\theta(\\widehat{\\theta}_n) + \\operatorname{Bias}_\\theta(\\widehat{\\theta}_n)^2.\\]\n\nConsequently, bias tending to zero together with variance tending to zero is a sufficient condition for consistency. It is not a necessary condition for weak consistency: convergence in probability alone need not imply convergence of expectations or second moments. (bpb-us-w2.wpmucdn.com)"
    },
    {
      "heading": "Basic examples",
      "text": "Sample mean\n\nLet \\(X_1,X_2,\\ldots\\) be independent and identically distributed, with \\(E|X_1|<\\infty\\) and mean \\(\\mu\\). The sample mean\n\n\\[\\overline X_n=\\frac1n\\sum_{i=1}^nX_i\\]\n\nis consistent for \\(\\mu\\) by the law of large numbers. Finite variance is not necessary for this conclusion. (ocw.mit.edu)\n\nIf \\(\\operatorname{Var}(X_1)=\\sigma^2<\\infty\\), there is a direct proof using Chebyshev’s inequality:\n\n\\[P(|\\overline X_n-\\mu|>\\varepsilon) \\leq\\frac{\\sigma^2}{n\\varepsilon^2} \\longrightarrow0.\\]\n\nIn this case the sample mean is also mean-square consistent, since its mean squared error is \\(\\sigma^2/n\\). (bpb-us-w2.wpmucdn.com)\n\nA biased but consistent endpoint estimator\n\nFor independent observations from the uniform distribution on \\([0,\\theta]\\), where \\(\\theta>0\\), consider the sample maximum \\(M_n\\). Its expectation is\n\n\\[E_\\theta[M_n]=\\frac{n}{n+1}\\theta,\\]\n\nso it is biased downward. Nevertheless, for \\(0<\\varepsilon<\\theta\\),\n\n\\[P_\\theta(|M_n-\\theta|>\\varepsilon) = \\left(1-\\frac{\\varepsilon}{\\theta}\\right)^n \\longrightarrow0.\\]\n\nThus the maximum is consistent despite its finite-sample bias. (ocw.mit.edu)"
    },
    {
      "heading": "Establishing consistency",
      "text": "Many proofs combine a law of large numbers with a continuous transformation. If \\(\\widehat{\\theta}_n\\) is consistent for \\(\\theta\\), and \\(g\\) is a function continuous at \\(\\theta\\), the continuous mapping theorem gives\n\n\\[g(\\widehat{\\theta}_n)\\xrightarrow{P}g(\\theta).\\]\n\nThis underlies many plug-in estimators: estimated moments or probabilities are substituted into a continuous formula for the target. (ocw.mit.edu)\n\nEstimators defined by optimization require additional reasoning. An M-estimator minimizes or maximizes a sample criterion. A common sufficient framework is that the criterion converges uniformly to a deterministic population criterion, whose optimum is uniquely separated from values outside every neighborhood of the target. Suitable approximate optimizers can also be consistent. Pointwise convergence of the criterion alone generally does not justify convergence of its optimizer. (ocw.mit.edu)\n\nMaximum likelihood estimation fits this framework through the average log-likelihood. Its consistency requires appropriate model and regularity conditions; maximization of a likelihood is not, by itself, a guarantee. Identification is fundamental: distinct target values must be distinguishable through the observational distribution. (ocw.mit.edu)\n\nIn linear regression, the ordinary least squares estimator is consistent under suitable laws of large numbers, a nonsingular limiting regressor second-moment matrix, and population orthogonality between regressors and errors. These assumptions are separate from assumptions used to establish its asymptotic normality. (statlect.com)"
    },
    {
      "heading": "Rates and limits of interpretation",
      "text": "Consistency does not specify a convergence rate or a limiting distribution for scaled errors. For an independent, identically distributed sample with positive finite variance, the central limit theorem supplies the stronger result\n\n\\[\\sqrt n(\\overline X_n-\\mu) \\xrightarrow{d}N(0,\\sigma^2).\\]\n\nSuch distributional results, together with consistent variance estimation, support asymptotic confidence intervals and hypothesis tests. Consistency alone does not provide their calibration. (ocw.mit.edu)\n\nA consistency claim is conditional on its sampling model and assumptions. Dependence, lack of identification, or failure of required moment conditions can invalidate a particular proof. For regression, increasing the sample size does not replace the orthogonality condition needed for consistency of ordinary least squares. Finite-sample performance remains a separate question: an estimator can be consistent while having substantial bias, variability, or computational cost at practically available sample sizes. (statlect.com)"
    }
  ],
  "links": [
    "statistics",
    "estimator",
    "convergence-in-probability",
    "norm-mathematics",
    "metric-space",
    "almost-surely",
    "estimator-bias",
    "expected-value",
    "mean-squared-error",
    "variance",
    "sample-mean",
    "law-of-large-numbers",
    "chebyshevs-inequality",
    "uniform-distribution",
    "continuous-function",
    "maximum-likelihood-estimation",
    "statistical-identification",
    "linear-regression",
    "ordinary-least-squares",
    "central-limit-theorem",
    "confidence-interval",
    "statistical-hypothesis-testing"
  ],
  "sources": [
    {
      "title": "M-estimators and their consistency",
      "url": "https://ocw.mit.edu/courses/18-465-topics-in-statistics-nonparametrics-and-robustness-spring-2005/3f81a744094897380591c3d2b21201e8_m_estimates.pdf"
    },
    {
      "title": "More about Estimators and Mathematical Digressions",
      "url": "https://stat135.berkeley.edu/spring-2026/lectures/lecture-6.html"
    },
    {
      "title": "Mathematical Statistics, Lecture 16 Asymptotics: Consistency and Delta Method",
      "url": "https://ocw.mit.edu/courses/18-655-mathematical-statistics-spring-2016/2ca793c6ea6c4a6fece39c0da3440ba2_MIT18_655S16_LecNote16.pdf"
    },
    {
      "title": "Point estimation: Theory and examples",
      "url": "https://www.statlect.com/fundamentals-of-statistics/point-estimation"
    },
    {
      "title": "Lecture: Consistency",
      "url": "https://bpb-us-w2.wpmucdn.com/sites.brown.edu/dist/1/427/files/2023/11/APMA-1650-Lecture-32.pdf"
    },
    {
      "title": "MITOCW: 14.310x Lecture 12 transcript",
      "url": "https://ocw.mit.edu/courses/14-310x-data-analysis-for-social-scientists-spring-2023/1PojOjDPcm1F7LClHqbLBH8ziL8VvWyeY_transcript.pdf"
    },
    {
      "title": "Consistency of the Maximum Likelihood Estimator",
      "url": "https://stat135.berkeley.edu/spring-2026/lectures/lecture-13.html"
    },
    {
      "title": "Lecture 7: Maximum Likelihood Estimation",
      "url": "https://ocw.mit.edu/courses/14-381-statistical-method-in-economics-fall-2018/2eb7af31864f5f340b0b62ec8d3368ab_MIT14_381F18_lec7.pdf"
    },
    {
      "title": "Properties of the OLS estimator: Consistency, asymptotic normality",
      "url": "https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties"
    }
  ],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}