{
  "id": "statistical-hypothesis-testing",
  "title": "Statistical Hypothesis Testing",
  "lang": "en",
  "source_lang": "en",
  "summary": "Statistical hypothesis testing evaluates claims about populations or models using sample data and procedures with specified error properties.",
  "category": "mathematics",
  "subcategory": "probability-statistics",
  "type": "concept",
  "aliases": [
    "Hypothesis testing",
    "Statistical testing"
  ],
  "generated_at": "2026-10-04",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/statistical-hypothesis-testing/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/statistical-hypothesis-testing/",
      "json": "https://aiwiki.page/zh/statistical-hypothesis-testing.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Statistical hypothesis testing is a framework in statistics for evaluating claims about populations or data-generating processes using observed data. A test specifies competing hypotheses, a measure of discrepancy, and a rule for deciding whether the observations warrant rejecting a designated hypothesis. Its conclusions depend on the statistical model and sampling assumptions; they do not establish truth with certainty. (itl.nist.gov)"
    },
    {
      "heading": "Hypotheses and statistical models",
      "text": "A statistical hypothesis specifies a property of a population or model, such as a mean, a proportion, or the relationship between variables. The null hypothesis, denoted \\(H_0\\), is the claim tested. The alternative hypothesis, denoted \\(H_1\\) or \\(H_a\\), describes the competing possibilities. A null hypothesis need not assert “no effect”: it may specify a threshold or an inequality. (itl.nist.gov)\n\nFor example, a two-sided test of a population mean compares \\(H_0:\\mu=\\mu_0\\) with \\(H_1:\\mu\\ne\\mu_0\\). A one-sided test instead targets departures in a specified direction. A simple hypothesis completely specifies the data’s probability distribution, whereas a composite hypothesis permits more than one distribution. Hypotheses and assumptions jointly determine how a random variable representing the data behaves under the null model. (stats.org.uk)"
    },
    {
      "heading": "Test statistics and decision rules",
      "text": "A test statistic reduces the observations to a quantity relevant to the hypotheses. Its sampling distribution describes its behavior across hypothetical repetitions of the sampling procedure. A rejection region contains outcomes that lead to rejection of \\(H_0\\); its boundaries are often expressed as critical values. (itl.nist.gov)\n\nThe significance level \\(\\alpha\\) specifies an upper bound on the probability of rejecting a true null hypothesis, under the model. More formally, if \\(R\\) is the rejection region and \\(\\Theta_0\\) the set of parameter values allowed by \\(H_0\\), a level-\\(\\alpha\\) test satisfies\n\n\\[\\sup_{\\theta\\in\\Theta_0}P_\\theta(X\\in R)\\leq\\alpha.\\]\n\nThe bound need not be attained exactly, particularly for discrete data. The significance level is a property of the procedure, not the probability that a particular conclusion is wrong. (stats.org.uk)\n\nA p-value is the probability, under the specified null model, of obtaining a test statistic at least as extreme as the observed value, with “extreme” defined by the test. A conventional decision rule rejects when \\(p\\leq\\alpha\\). The p-value is neither the probability that \\(H_0\\) is true nor the probability that the observations arose “by chance alone.” (doi.org)"
    },
    {
      "heading": "Errors and statistical power",
      "text": "A Type I error occurs when a true null hypothesis is rejected. A Type II error occurs when a false null hypothesis is not rejected. For a particular alternative parameter value \\(\\theta\\), its probability is conventionally denoted \\(\\beta(\\theta)\\). The power at that alternative is\n\n\\[\\pi(\\theta)=1-\\beta(\\theta) =P_\\theta(\\text{reject }H_0).\\]\n\nPower therefore varies across alternatives rather than being a single universal attribute of a test. (itl.nist.gov)\n\nPower depends on sample size, variability, the magnitude of the departure from the null, and the rejection rule. Larger samples generally make smaller departures detectable. Experimental design and sample-size planning can incorporate power against specified alternatives. Failure to reject may reflect limited information, rather than agreement with the null hypothesis. (stat.berkeley.edu)"
    },
    {
      "heading": "Example: testing a population mean",
      "text": "Suppose \\(X_1,\\ldots,X_n\\) are independent observations from a normal distribution with unknown mean and variance. The one-sample Student’s t-test of \\(H_0:\\mu=\\mu_0\\) uses\n\n\\[T=\\frac{\\bar X-\\mu_0}{S/\\sqrt n},\\]\n\nwhere \\(\\bar X\\) is the sample mean and \\(S\\) the sample standard deviation. The denominator estimates the standard error of the mean. Under \\(H_0\\), \\(T\\) follows Student’s t-distribution with \\(n-1\\) degrees of freedom. (itl.nist.gov)\n\nFor a two-sided level-\\(\\alpha\\) test, rejection occurs when\n\n\\[|T|>t_{1-\\alpha/2,n-1}.\\]\n\nWith \\(\\alpha=0.05\\), hypothetical values \\(n=25\\), \\(\\bar X=102\\), \\(S=5\\), and \\(\\mu_0=100\\) give \\(T=2\\). The critical value is approximately \\(2.064\\), so the null is not rejected. This is not evidence that the population mean is exactly 100. (itl.nist.gov)\n\nTwo-sample tests compare population means. Welch’s formulation accommodates unequal population variances, while paired tests analyze within-pair differences rather than treating all observations as unrelated. Analysis of variance extends mean comparisons to several groups. (itl.nist.gov)"
    },
    {
      "heading": "Confidence intervals and multiple comparisons",
      "text": "Tests and confidence intervals have a mathematical duality. Inverting a family of level-\\(\\alpha\\) tests produces a confidence set containing parameter values not rejected by those tests. For matching two-sided t procedures, a hypothesized mean is rejected exactly when it lies outside the corresponding \\(1-\\alpha\\) confidence interval. (stat.berkeley.edu)\n\nThe multiple comparisons problem arises when many tests are considered together: individual error control does not automatically provide equivalent control for the collection. The Bonferroni correction addresses this by testing each of \\(m\\) hypotheses at level \\(\\alpha/m\\), ensuring that the probability of at least one Type I error is at most \\(\\alpha\\), without requiring independence. (itl.nist.gov)"
    },
    {
      "heading": "Interpretation and reporting",
      "text": "Statistical significance does not measure effect size or practical importance. A small effect can be statistically significant in a large sample, while an important effect may remain undetected in a small sample. Estimates and uncertainty intervals provide information that a rejection decision alone cannot convey. (itl.nist.gov)\n\nInterpretation also depends on model assumptions, study execution, and the analyses actually performed. Selectively reporting only small p-values obscures the testing process and undermines their interpretation. The American Statistical Association’s 2016 statement emphasized transparent reporting and cautioned against basing scientific conclusions solely on whether a p-value crosses a threshold. (doi.org)"
    }
  ],
  "links": [
    "statistics",
    "hypothesis",
    "null-hypothesis",
    "alternative-hypothesis",
    "probability-distribution",
    "random-variable",
    "test-statistic",
    "sampling-distribution",
    "significance-level",
    "probability",
    "p-value",
    "type-i-and-type-ii-errors",
    "statistical-power",
    "experimental-design",
    "normal-distribution",
    "students-t-test",
    "standard-deviation",
    "standard-error",
    "students-t-distribution",
    "degrees-of-freedom",
    "variance",
    "analysis-of-variance",
    "confidence-interval",
    "multiple-comparisons-problem",
    "bonferroni-correction",
    "effect-size"
  ],
  "sources": [
    {
      "title": "itl.nist.gov/div898/handbook/prc/section1/prc13.htm",
      "url": "https://www.itl.nist.gov/div898/handbook/prc/section1/prc13.htm"
    },
    {
      "title": "stats.org.uk/statistical-inference/NeymanPearson1933.pdf",
      "url": "https://stats.org.uk/statistical-inference/NeymanPearson1933.pdf"
    },
    {
      "title": "itl.nist.gov/div898/handbook/eda/section3/eda35.htm",
      "url": "https://www.itl.nist.gov/div898/handbook/eda/section3/eda35.htm"
    },
    {
      "title": "doi.org/10.1080/00031305.2016.1154108",
      "url": "https://doi.org/10.1080/00031305.2016.1154108"
    },
    {
      "title": "stat.berkeley.edu/~ani/s135f06.old/midrev.pdf",
      "url": "https://www.stat.berkeley.edu/~ani/s135f06.old/midrev.pdf"
    },
    {
      "title": "itl.nist.gov/div898/handbook/prc/section2/prc22.htm",
      "url": "https://www.itl.nist.gov/div898/handbook/prc/section2/prc22.htm"
    },
    {
      "title": "itl.nist.gov/div898/software/dataplot/refman1/auxillar/t_test.htm",
      "url": "https://www.itl.nist.gov/div898/software/dataplot/refman1/auxillar/t_test.htm"
    },
    {
      "title": "itl.nist.gov/div898/handbook/eda/section3/eda353.htm",
      "url": "https://www.itl.nist.gov/div898/handbook/eda/section3/eda353.htm"
    },
    {
      "title": "stat.berkeley.edu/~stark/Teach/S240/Notes/lec1.pdf",
      "url": "https://www.stat.berkeley.edu/~stark/Teach/S240/Notes/lec1.pdf"
    },
    {
      "title": "itl.nist.gov/div898/handbook/prc/section4/prc463.htm",
      "url": "https://www.itl.nist.gov/div898/handbook/prc/section4/prc463.htm"
    }
  ],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}