{
  "id": "type-i-error",
  "title": "Type I Error",
  "lang": "en",
  "source_lang": "en",
  "summary": "A Type I error occurs when a statistical test rejects a true null hypothesis; its probability is controlled by the test’s significance level.",
  "category": "mathematics",
  "subcategory": "probability-statistics",
  "type": "concept",
  "aliases": [
    "Error of the first kind",
    "False-positive error",
    "Alpha error"
  ],
  "generated_at": "2026-11-04",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/type-i-error/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/type-i-error/",
      "json": "https://aiwiki.page/zh/type-i-error.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "A Type I error occurs in statistical hypothesis testing when a test rejects a null hypothesis that is true. It is often called a false positive, although what counts as “positive” depends on how the hypotheses are formulated. The probability of this error is controlled through the test’s significance level, conventionally denoted by \\(\\alpha\\). This is a property of a testing procedure under specified assumptions, not the probability that a particular rejected hypothesis is true."
    },
    {
      "heading": "Definition and mathematical formulation",
      "text": "A statistical test uses observed data to choose between rejecting a null hypothesis, \\(H_0\\), and not rejecting it. An alternative hypothesis, \\(H_1\\), describes parameter values or models outside the null hypothesis.\n\nLet \\(X\\) represent the data, let \\(\\theta\\) be the parameter governing their probability distribution, and write the null hypothesis as\n\n\\[H_0:\\theta\\in\\Theta_0.\\]\n\nIf \\(R\\) is the test’s rejection region—the set of possible datasets that lead to rejection—the Type I error probability at a particular null parameter value is\n\n\\[\\alpha(\\theta)=P_\\theta(X\\in R), \\qquad \\theta\\in\\Theta_0.\\]\n\nHere \\(P_\\theta\\) means probability calculated under the distribution specified by \\(\\theta\\). The parameter is held fixed; the randomness comes from the data.\n\nFor a simple null hypothesis, which specifies a single distribution, there is one Type I error probability. For a composite null hypothesis, which allows several parameter values, the error probability can vary across those values. The test’s size is\n\n\\[\\sup_{\\theta\\in\\Theta_0}P_\\theta(X\\in R).\\]\n\nA test has level \\(\\alpha\\) if its size is no greater than \\(\\alpha\\). Thus, describing a procedure as a “5% test” generally means that its Type I error probability is at most \\(0.05\\) throughout the null hypothesis, under the assumptions used to justify the procedure. It need not equal \\(0.05\\) at every null parameter value."
    },
    {
      "heading": "Relationship to other testing outcomes",
      "text": "The two possible decisions and the two possible states of the null hypothesis produce four outcomes:\n\nState of the null hypothesis | Reject \\(H_0\\) | Do not reject \\(H_0\\)\n\\(H_0\\) is true | Type I error | Correct non-rejection\n\\(H_0\\) is false | Correct rejection | Type II error\n\nA Type II error is failure to reject a false null hypothesis. The probability of rejection when a specified alternative is true is the test’s statistical power. The distinction between Type I and Type II errors concerns different possible states of the underlying model; they are not complementary events under a single fixed parameter value.\n\n“Do not reject” is generally more precise than “accept.” An inconclusive test may reflect limited information rather than evidence that the null hypothesis is true. Tests designed to establish equivalence or noninferiority use differently formulated null hypotheses and decision criteria."
    },
    {
      "heading": "Example: a test of a population mean",
      "text": "Suppose \\(X_1,\\ldots,X_n\\) are independent observations from a normal distribution with unknown mean \\(\\mu\\) and known standard deviation \\(\\sigma\\). Consider the two-sided test\n\n\\[H_0:\\mu=\\mu_0, \\qquad H_1:\\mu\\ne\\mu_0.\\]\n\nUsing the sample mean \\(\\bar X\\), define the test statistic\n\n\\[Z=\\frac{\\bar X-\\mu_0}{\\sigma/\\sqrt n}.\\]\n\nUnder \\(H_0\\), \\(Z\\) has a standard normal distribution. A level-\\(0.05\\) test rejects when\n\n\\[|Z|>\\Phi^{-1}(0.975)\\approx1.96,\\]\n\nwhere \\(\\Phi\\) is the standard normal cumulative distribution function.\n\nIf \\(\\mu\\) really equals \\(\\mu_0\\), the probability that the statistic falls into either rejection tail is \\(0.05\\). Such a rejection is a Type I error.\n\nIn repeated independent applications under this null model, the proportion of rejections approaches \\(0.05\\) as the number of applications grows. This does not mean that exactly five out of every hundred tests will reject, nor that a particular rejection has a 5% probability of being mistaken.\n\nFor a one-sided alternative, such as \\(\\mu>\\mu_0\\), the rejection region lies in one tail instead. Choosing the direction after inspecting the observed effect does not generally preserve the stated error rate."
    },
    {
      "heading": "Significance levels, p-values, and confidence intervals",
      "text": "Significance level versus observed p-value\n\nA significance level is a threshold defining a procedure’s error control. A p-value is calculated from the observed data and measures how incompatible those data are with the null model, using a specified ordering of results.\n\nFor a valid p-value,\n\n\\[P_\\theta(p\\le u)\\le u \\quad \\text{for every }\\theta\\in\\Theta_0 \\text{ and }0\\le u\\le1.\\]\n\nConsequently, rejecting whenever \\(p\\le\\alpha\\) gives a level-\\(\\alpha\\) test.\n\nUnder some continuous null models, the p-value has an exactly uniform distribution. In other settings, particularly discrete tests, its distribution can be conservative, so that rejection occurs less often than the nominal level.\n\nAn observed p-value of \\(0.03\\) is not:\n\n- the probability that the null hypothesis is true;\n- the probability that the particular rejection is a Type I error;\n- a measure of the magnitude or practical importance of an effect.\n\nPractical importance is a separate question, often addressed through an effect size and its uncertainty.\n\nConfidence-interval duality\n\nTests and confidence intervals can be constructed by inverting one another. In a matched construction, a level-\\(\\alpha\\) test rejects a proposed parameter value precisely when the corresponding \\(100(1-\\alpha)\\%\\) confidence set excludes it.\n\nThe coverage guarantee,\n\n\\[P_\\theta\\{\\theta\\in C(X)\\}\\ge1-\\alpha,\\]\n\ncorresponds to control of the probability of rejecting the true parameter value. This relationship requires compatible testing and interval procedures; arbitrary tests and intervals need not give matching decisions."
    },
    {
      "heading": "Controlling error in practice",
      "text": "Nominal and actual error rates\n\nA nominal significance level is justified only to the extent that the procedure’s assumptions hold. The actual Type I error rate can differ because of dependence among observations, incorrect distributional assumptions, unsuitable approximations, or a mismatch between the analysis and the experimental design.\n\nSome tests have exact finite-sample guarantees under specified assumptions. Others provide asymptotic guarantees that become accurate as sample size increases. A large sample alone does not repair every form of model misspecification.\n\nData-dependent choices also matter. Selecting an outcome, subgroup, model, or stopping point after examining results can change the probability of rejection. Sequential procedures can allow repeated examination of accumulating data while controlling Type I error, but their decision boundaries must account for that examination.\n\nMultiple testing\n\nWhen many hypotheses are tested, controlling each test separately does not generally control the chance of at least one false rejection.\n\nIf \\(m\\) tests are independent, all their null hypotheses are true, and each has Type I error probability exactly \\(\\alpha\\), then\n\n\\[P(\\text{at least one Type I error}) =1-(1-\\alpha)^m.\\]\n\nFor \\(m=20\\) and \\(\\alpha=0.05\\), this probability is approximately \\(0.642\\).\n\nMultiple-testing procedures specify which collective error rate they control. The Bonferroni correction, for example, tests each of \\(m\\) hypotheses at level \\(\\alpha/m\\) to control the probability of one or more false rejections at no more than \\(\\alpha\\), without requiring independence.\n\nThe false discovery rate is different: it is the expected proportion of false rejections among all rejections, with that proportion defined as zero when nothing is rejected."
    },
    {
      "heading": "Trade-offs and interpretation",
      "text": "For a fixed testing problem and a family of nested rejection regions, lowering \\(\\alpha\\) makes rejection harder and generally reduces power. Increasing sample size can improve power while retaining the same significance level. The balance also depends on the alternatives of interest and the consequences of the two kinds of error.\n\nThe Neyman–Pearson framework, developed in the early twentieth century, formalized the strategy of seeking powerful tests subject to a specified bound on Type I error. In broader decision theory, error consequences can instead be represented through a loss function. No single significance level is universally appropriate merely because it is conventional.\n\nType I error control also does not determine the proportion of reported findings that are false. That proportion depends on how often the tested null hypotheses are true, the tests’ power, and how analyses and results are selected. For example, among 1,000 tests, suppose 900 null hypotheses are true and 100 are false. If each true-null test has error probability \\(0.05\\) and each false-null test has power \\(0.80\\), the expected counts are 45 false rejections and 80 correct rejections. The ratio of these expected counts is \\(45/125=36\\%\\), not 5%; this ratio is not generally identical to the expected false-discovery proportion.\n\nAssigning a probability to a hypothesis itself requires an additional framework, such as Bayesian inference, together with specified models and a prior distribution. A frequentist Type I error guarantee concerns how a procedure behaves under true null hypotheses, not the posterior probability that an individual conclusion is wrong."
    }
  ],
  "links": [
    "statistical-hypothesis-testing",
    "null-hypothesis",
    "significance-level",
    "alternative-hypothesis",
    "probability-distribution",
    "statistical-power",
    "type-i-and-type-ii-errors",
    "normal-distribution",
    "sample-mean",
    "test-statistic",
    "cumulative-distribution-function",
    "p-value",
    "effect-size",
    "confidence-interval",
    "experimental-design",
    "sample-size",
    "statistical-independence",
    "multiple-testing",
    "bonferroni-correction",
    "false-discovery-rate",
    "decision-theory",
    "loss-function",
    "bayesian-inference",
    "prior-distribution"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}