{
  "id": "statistics",
  "title": "Statistics",
  "lang": "en",
  "source_lang": "en",
  "summary": "Statistics is the science of collecting, analyzing, interpreting and presenting data under uncertainty.",
  "category": "mathematics",
  "subcategory": "probability-statistics",
  "type": "concept",
  "aliases": [
    "Statistical science",
    "mathematical statistics"
  ],
  "generated_at": "2026-10-08",
  "model": "pro/claude-haiku-5-5",
  "url": "https://aiwiki.page/en/statistics/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/statistics/",
      "json": "https://aiwiki.page/zh/statistics.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Statistics is the discipline concerned with collecting, organizing, analyzing, interpreting and presenting data, and with drawing conclusions from data when uncertainty is present. It rests on probability and on the mathematics of mathematical modeling, and it supplies the methods by which scientists, governments, businesses and physicians decide what observed patterns mean. Its central problem is inference: using a limited sample to say something about a larger population or about a process that generates data, and stating how reliable that statement is.\n\nThe field has two broad branches. Descriptive statistics summarizes a data set through measures such as the mean, variance and graphical displays. Inferential statistics goes further, using sampling distributions, hypothesis tests, confidence intervals and Bayesian methods to make claims beyond the data at hand. Statistics developed from early state counting and political arithmetic into a mathematical science in the nineteenth and twentieth centuries, with major contributions from figures such as Gauss, Laplace, Fisher and Kolmogorov.\n\nThis entry first treats the foundations: the definition of the field, its historical development, probability theory, and descriptive measures. It then covers the design of data collection, probability distributions, estimation, hypothesis testing and Bayesian inference. Later sections address regression, causal inference, time series, multivariate methods and the link to machine learning, followed by computational methods, applications in epidemiology, economics and official statistics, and the debates over reproducibility and the misuse of significance testing."
    },
    {
      "heading": "Definition and scope of the field",
      "text": "Statistics is the discipline concerned with the collection, analysis, interpretation and presentation of data. Its central task is to say what can be concluded about a larger situation when only part of it has been observed, and how much confidence that conclusion deserves. It is often contrasted with probability theory. Probability starts from a known model, such as a fair die with six equally likely faces, and derives what data that model should produce. Statistics runs in the opposite direction: it starts from observed data and asks which models or parameter values are compatible with them. The two fields are closely linked, because statistical methods rely on probability to describe how data vary, but their starting points differ.\n\nThe field has a core vocabulary. A population is the full collection of units about which a question is asked, such as all adults in a country or all bolts produced by a factory in one month. A sample is the subset actually observed. A variable is a characteristic measured on each unit, such as height, income or a yes-or-no response, and the value recorded for one unit is an observation. A parameter is a numerical summary of the population, such as its mean \\(\\mu\\) or the proportion \\(p\\) holding a given view, and it is usually unknown. A statistic is a numerical summary computed from the sample, such as the sample mean \\(\\bar{x}=\\frac{1}{n}\\sum_{i=1}^{n}x_i\\). Statistics are used to learn about parameters. The common notational convention, with Greek letters for parameters, Latin letters for statistics, and a hat for an estimate, keeps the two apart.\n\nTwo goals structure most work in the field. Descriptive statistics summarizes the data at hand through averages, tables and charts. Inferential statistics uses a sample to draw conclusions about a population or about the process that produced the data, and attaches a measure of uncertainty to those conclusions. A survey of 1,000 voters can describe those 1,000 respondents exactly. Using the same responses to say something about the whole electorate is an inference, and it depends on assumptions about how the respondents were selected.\n\nA further distinction separates the data from the process behind them. The data are the realized outcomes that were recorded. The process is the physical, biological or social mechanism that generated them, together with its parameters. Ten coin tosses yield ten observed outcomes, while the process is a coin with some underlying probability of heads, which the data are used to assess. Many errors in applied work come from treating a pattern in one dataset as a property of the underlying process without asking how the data were collected. A biased sampling scheme can produce a clear pattern that does not exist in the population at all.\n\nThe defining concern of the field is uncertainty arising from variability. Repeating a measurement or drawing a new sample gives different numbers, because units differ naturally, because any sample is only a part of the population, and because measurements contain error. If the world were fully deterministic and every measurement exact, a single observation would settle the question and the field would have little to do. Because this is not the case, inferential statements must carry their uncertainty explicitly, through tools such as the standard error, confidence intervals, credible intervals or p-values. Much of the subject consists of designing data collection and models so that this uncertainty is quantified honestly and kept as small as the situation allows."
    },
    {
      "heading": "Historical origins: from state counting to a mathematical science",
      "text": "Systematic counting long predates the discipline. Ancient states recorded populations, land, grain, labor and taxes, using clay tablets in Mesopotamia and census registers in Egypt, China and Rome. The Roman census, conducted under the authority of censors, assessed citizens for military service and taxation, and the English Domesday Book of 1086 catalogued landholdings after the Norman conquest. These records were administrative instruments that tallied what rulers needed to know, and they did not analyze the numbers. The word reflects this origin. \"Statistics\" derives from the Italian statista and the Latin status, meaning the condition of a state. In the mid-eighteenth century, Gottfried Achenwall at Göttingen popularized the German term Statistik for the systematic description of the political affairs of states. Only later did the word come to denote the general methods for collecting and analyzing numerical data.\n\nIn seventeenth-century England, a group of writers pursued what William Petty called \"political arithmetick,\" the attempt to reason quantitatively about national affairs. John Graunt's Natural and Political Observations ... upon the Bills of Mortality (1662) worked from London's weekly burial lists, which were compiled from parish records. Graunt used these counts to estimate survival by age and to observe that the numbers of births and deaths were remarkably regular from year to year, a finding that suggested stable underlying causes. Edmond Halley's life table of 1693, built from parish registers of the city of Breslau, was more carefully constructed and was soon used to price annuities. Such tables, which later became the basis of the mortality-rate measures and the modern life table, showed that data about individuals could reveal lawlike patterns in populations.\n\nThe nineteenth century transformed these tallies into mathematical tools. Adrien-Marie Legendre published the method of least squares in 1805, and Carl Friedrich Gauss (1809) used it to fit astronomical orbits and linked it to a probability law for measurement errors. Pierre-Simon Laplace supplied a general argument, in 1810, that sums of many small independent errors tend toward a bell-shaped curve, which became the normal distribution and the foundation of the least-squares theory of error. Adolphe Quetelet applied the normal curve to human measurements such as height and chest girth, and in 1835 he introduced the \"average man\" as a social ideal, a step in what he called social physics. Such work fed a broader social-statistics movement. The Statistical Society of London was founded in 1834 and the American Statistical Association in 1839, and governments began to organize regular censuses and vital registration on a systematic basis. Francis Galton's work in the 1880s on heredity introduced correlation and regression as ways of describing relationships between measured variables.\n\nThe early twentieth century saw the discipline formalized in its modern mathematical form. Karl Pearson developed the method of moments, the chi-squared test (1900) and the journal Biometrika (1901). Ronald Fisher introduced maximum likelihood estimation in the early 1920s, developed analysis of variance, and, in The Design of Experiments (1935), argued for randomization in experiments. Jerzy Neyman and Egon Pearson, from 1933 onward, proposed a decision-oriented framework of null and alternative hypotheses with controlled error rates, and Neyman developed the theory of confidence intervals in 1937. Fisher's approach centered on measuring evidence against a null hypothesis through the p-value, whereas the Neyman–Pearson approach emphasized long-run error control and explicit alternatives. The two camps disagreed sharply, and their disputes, along with the older Bayesian tradition revived later in the century, produced the schools of thought that still shape the field's methods and debates."
    },
    {
      "heading": "Probability foundations",
      "text": "Probability began as a mathematical subject in the mid-seventeenth century, when questions about games of chance were treated as problems that could be solved exactly. The exchange of letters between Blaise Pascal and Pierre de Fermat in 1654 is the usual reference point. Among the problems they discussed was the problem of points: how to divide the stakes of an interrupted game between players who each still needed a different number of wins. Their solutions compared the possible completions of the game and weighed them by how likely each was. Christiaan Huygens later gathered such methods in a short treatise on dice games (1657). These early methods worked well for finite sets of equally likely outcomes, but they gave no general rule for cases in which outcomes were unequal or infinite.\n\nA rigorous foundation came only in 1933, when Andrey Kolmogorov set out an axiomatic framework. A probability model is a triple made of a sample-space \\(\\Omega\\), a collection \\(\\mathcal{F}\\) of events, and a function \\(P\\) that assigns numbers to those events. The collection \\(\\mathcal{F}\\) must be a sigma-algebra, closed under complements and countable unions. The function \\(P\\) must be nonnegative, must give \\(P(\\Omega)=1\\), and must be countably additive for disjoint events. The triple is called a probability-space. The framework borrows its machinery from measure-theory, which provides the integrals used to define expectation and the distributions of continuous variables. The restriction to a sigma-algebra is not a technicality. In the standard construction, it is impossible to assign a consistent probability to every subset of the unit interval, so events must be restricted to a well-behaved family. Once the framework is fixed, a random-variable is a measurable function on \\(\\Omega\\), and its distribution is the probability measure it induces.\n\nConditional probability is defined for an event \\(B\\) with \\(P(B)>0\\) by\n\n\\[P(A\\mid B)=\\frac{P(A\\cap B)}{P(B)}.\\]\n\nIt describes how the probability of \\(A\\) changes once \\(B\\) is known. If \\(B_1,\\dots,B_n\\) form a partition of \\(\\Omega\\) into events of positive probability, the law of total probability states that\n\n\\[P(A)=\\sum_{i=1}^{n} P(A\\mid B_i)\\,P(B_i).\\]\n\nThis rule is the basis of much of the probability reasoning used in later sections, including the derivation of Bayes' theorem. Two events are independent when \\(P(A\\cap B)=P(A)P(B)\\), which is equivalent to saying that conditioning on one leaves the probability of the other unchanged. Independence of a larger family requires the product rule for every finite subfamily. Pairwise independence is weaker: there are standard examples of three events that are pairwise independent without being jointly independent. Conditional independence, given a third quantity, is a further refinement that is central to causal and graphical modeling.\n\nExpectation and variance are the basic summaries of a random quantity. The expected value of \\(X\\) is the integral of \\(X\\) with respect to \\(P\\), written \\(E[X]\\). It is linear, so \\(E[aX+bY]=aE[X]+bE[Y]\\) holds whether or not \\(X\\) and \\(Y\\) are independent. Not every random variable has an expectation: the Cauchy distribution has none, which is a reminder that these summaries exist only under integrability conditions. The variance is \\(\\operatorname{Var}(X)=E[(X-\\mu)^2]\\), where \\(\\mu=E[X]\\), and it can also be computed as \\(E[X^2]-\\mu^2\\). Variance scales with the square of a multiplicative constant, so \\(\\operatorname{Var}(aX+b)=a^2\\operatorname{Var}(X)\\). For independent variables it is additive. Its square root, the standard deviation, expresses spread in the units of the original variable. These two quantities recur throughout the rest of the field, in the laws of large numbers, in the central limit theorem, and in the definition of standard error."
    },
    {
      "heading": "Descriptive statistics and exploratory analysis",
      "text": "Descriptive statistics summarizes a set of observations without making claims about a larger population. Its tools fall into three groups: measures of location, measures of spread, and measures of shape, supplemented by graphical displays. Exploratory data analysis, a term popularized by John W. Tukey, treats these summaries as a first step toward understanding data before any formal model is fitted.\n\nMeasures of central tendency\n\nThe mean is the arithmetic average, the sum of the values divided by their count, written \\(\\bar{x} = \\frac{1}{n}\\sum_{i=1}^{n} x_i\\). It uses every observation and underlies much of the theory of sample means and of expected values, but it is pulled strongly toward extreme values. The median is the middle value of the ordered data, or the average of the two middle values when the count is even. It is resistant to outliers, so it is usually preferred for skewed quantities such as household income or house prices. The mode is the most frequent value. It is the only measure that applies to nominal categories, and it helps identify peaks in multimodal data, although a sample may have no mode or several.\n\nThe choice among them depends on the measurement scale. The mean requires interval or ratio data, the median requires only an ordering, and the mode works with categories. When the mean and median differ noticeably, the gap is itself evidence of asymmetry in the distribution.\n\nMeasures of dispersion and shape\n\nThe range, the difference between the maximum and minimum, is easy to compute but depends on only two observations. The sample variance is the average squared deviation from the mean, \\(s^2 = \\frac{1}{n-1}\\sum_{i=1}^{n}(x_i-\\bar{x})^2\\). Dividing by \\(n-1\\) rather than \\(n\\) yields an unbiased estimate of the population variance, a correction discussed under Bessel's correction. The standard deviation, the square root of the variance, returns to the original units and is the usual companion of the mean. Because squaring magnifies large deviations, both measures are sensitive to outliers.\n\nThe interquartile range (IQR), the distance between the 25th and 75th percentiles, is the robust counterpart. It describes the spread of the central half of the data and is unaffected by what happens in the extreme tails.\n\nShape is described by skewness, the standardized third central moment, which is positive when the right tail is longer and negative when the left tail is longer. Kurtosis, the standardized fourth central moment, compares tail heaviness with that of a normal distribution, for which the excess kurtosis is zero. Both are moment-based, so their sample estimates are noisy and heavily influenced by outliers unless the sample is large.\n\nGraphical displays and their interpretation\n\nA histogram partitions the range into bins and draws bars whose heights reflect counts or densities. It shows modality, skew and gaps, but its appearance depends on bin width, so several widths should be compared. A box plot draws a box from the first to the third quartile, marks the median with a line, and extends whiskers to the most extreme points lying within 1.5 IQR of the box. Points beyond the whiskers are plotted individually as candidate outliers. This convention allows several groups to be compared side by side on a common scale.\n\nA scatter plot shows each pair of values as a point. It reveals the form of an association (linear, curved or clustered), its direction and strength, and any unusual points with unusual positions. Interpretation requires caution. A histogram built from few observations can mislead, the apparent pattern in a scatter plot may be driven by a single point, and a correlation coefficient captures only linear association. A strong nonlinear relation can produce a coefficient near zero.\n\nExploratory data analysis as a workflow\n\nExploratory data analysis (EDA) is less a fixed procedure than a sequence of questions put to the data before modeling. A typical workflow begins with structure: the number of observations, the type and units of each variable, and the extent of missing values. Univariate inspection then examines each variable's distribution, and bivariate and multivariate inspection examines how variables move together. Tukey's 1977 book treated this stage as a investigative activity that precedes confirmatory analysis.\n\nOutliers should be investigated rather than deleted automatically. An extreme value may be a data-entry error, a measurement fault, or a genuine and informative observation, and each possibility leads to a different action. Other routine checks include detecting duplicate records, impossible values, inconsistent coding, and patterns of missingness that could bias later results; missing-data imputation is one response to the last of these.\n\nFindings from this stage shape the subsequent model. A strongly skewed variable may call for a logarithmic transformation, a nonlinear pattern for a different functional form, and strong collinearity for dimensionality reduction or regularization. Because the same data that suggested a pattern can also appear to confirm it, EDA findings are best treated as hypotheses to be checked on new data."
    },
    {
      "heading": "Sampling and experimental design",
      "text": "Populations, sampling frames and probability sampling\n\nA statistical study aims at a population, the complete set of units about which a conclusion is wanted. Because a population is rarely observed in full, a sample is drawn from a sampling frame, the list or procedure from which units are actually selected. If the frame leaves out part of the population or contains units outside it, the error enters before any randomness in selection, and no later calculation removes it.\n\nThe central dichotomy is between probability sampling, in which each unit has a known, nonzero chance of selection, and non-probability sampling, such as convenience samples or judgement samples chosen by the researcher. Only under a known selection mechanism can the sampling-distribution of an estimator be derived and its standard-error estimated, which in turn allows a confidence-interval to be interpreted in the usual way. Convenience samples are often cheap and quick to assemble, but the design itself gives no basis for quantifying how far the sample departs from the population.\n\nSimple random, stratified, cluster and systematic designs\n\nSimple random sampling, in which every subset of the target size is equally likely, serves as the benchmark against which other designs are compared. Stratified sampling partitions the population into mutually exclusive strata defined by a variable related to the outcome, then draws a sample within each stratum. When the strata are internally homogeneous and differ from one another, the between-stratum variation is removed from the sampling error, so the estimate of a population mean is typically more precise than under simple random sampling of the same size. Allocation may be proportional to stratum size, which keeps the sample self-weighting, or equal across strata, which is preferable when each subgroup must be estimated with comparable precision.\n\nCluster sampling selects naturally occurring groups such as schools, hospitals or districts, and observes all or a subsample of their members. It reduces travel and listing costs, since a complete list of individuals is unnecessary, but people in the same cluster tend to resemble one another. The information in the sample is therefore smaller than its nominal size. The design effect, defined as \\(\\mathrm{DEFF} = \\operatorname{Var}_{\\text{design}} / \\operatorname{Var}_{\\text{SRS}}\\) for the same number of observations, quantifies this loss, and values above one mean the design is less precise than simple random sampling. Large national surveys often combine the two ideas: they first select clusters such as census areas, then stratify within the chosen clusters.\n\nSystematic sampling takes every \\(k\\)-th unit from an ordered list after a random start. It is simple to administer and spreads the sample evenly along the list, which tends to improve precision when the ordering correlates with the outcome. The risk is that a periodic pattern in the list that matches the sampling interval can produce a biased sample, and the usual variance formulas may not apply cleanly.\n\nSurvey error and sampling bias\n\nSurvey error is commonly divided into several sources. Coverage error arises when the frame omits members of the target population or includes ineligible units. Nonresponse error arises when selected units do not participate at all (unit nonresponse) or leave parts of the questionnaire blank (item nonresponse). Nonresponse bias occurs when respondents differ systematically from nonrespondents on the quantity being measured, and enlarging the sample does not remove it, although it reduces sampling variability. Measurement error arises from questions that respondents misread, recall periods that they misplace events within, ambiguous response categories, or social desirability pressure that leads to underreporting of stigmatized behavior. Frames that carry auxiliary information beyond identifiers, together with weighting based on variables correlated with the outcome, are among the standard responses to these problems.\n\nFisher's principles of experimental design\n\nIn Statistical Methods for Research Workers (1925) and The Design of Experiments (1935), Ronald Fisher set out three principles for designing controlled experiments, developed largely through agricultural field trials at Rothamsted Experimental Station. The first is randomization, the chance assignment of treatments to experimental units. Fisher argued that randomization spreads unrecorded influences across the treatment groups in expectation, and that it supplies the physical basis for the validity of a significance test, so that inference does not rest entirely on untestable distributional assumptions. This principle underlies the modern randomized-controlled-trial.\n\nThe second is replication, applying each treatment to several independent experimental units. Replication allows the experimental error to be estimated, which in turn separates genuine treatment differences from chance variation. The third is blocking, or local control. When units can be recognized in advance as more homogeneous within groups, the groups are formed into blocks, and treatments are compared within each block so that block differences are excluded from the error term. This is the logic behind the analysis-of-variance. Fisher also introduced the Latin square, in which rows and columns serve as two simultaneous blocking factors, each treatment appearing once per row and column. Blocking is especially useful when randomizing a factor is impractical or too costly, and within each block the order of runs is still randomized. Together, the three principles are the foundation of experimental-design and of the causal reasoning discussed under confounding."
    },
    {
      "heading": "Probability distributions and limit theorems",
      "text": "Discrete distributions\n\nA probability distribution assigns probabilities to the values of a random variable. For a discrete variable this assignment is given by a probability mass function. The binomial distribution counts successes in \\(n\\) independent trials, each with success probability \\(p\\), so that \\(P(X=k)=\\binom{n}{k}p^k(1-p)^{n-k}\\). Its mean is \\(np\\) and its variance is \\(np(1-p)\\). The Poisson distribution counts events in a fixed interval when they occur independently at a constant average rate \\(\\lambda\\), with \\(P(X=k)=\\lambda^k e^{-\\lambda}/k!\\). Its mean and variance are both \\(\\lambda\\). Siméon-Denis Poisson described it in 1837. Ladislaus Bortkiewicz's 1898 analysis of horse-kick deaths in Prussian cavalry corps is a well-known application. The Poisson arises as a limit of the binomial when \\(n\\) is large, \\(p\\) is small and \\(np\\) stays near \\(\\lambda\\).\n\nContinuous distributions\n\nA continuous variable is described by a probability density function, and probabilities are integrals of the density over intervals. The normal distribution with mean \\(\\mu\\) and variance \\(\\sigma^2\\) has density \\[f(x)=\\frac{1}{\\sqrt{2\\pi\\sigma^2}}\\exp\\!\\left(-\\frac{(x-\\mu)^2}{2\\sigma^2}\\right).\\] Three further distributions come from normal samples. The chi-squared distribution with \\(k\\) degrees of freedom is the law of a sum of squares of \\(k\\) independent standard normal variables. Friedrich Robert Helmert derived the distribution of the sample variance in 1875, and Karl Pearson used the chi-squared statistic for goodness-of-fit testing in 1900. The Student's t distribution was derived by William Sealy Gosset, who published in 1908 under the pseudonym \"Student\". It describes the standardized sample mean when the unknown population standard deviation is replaced by the sample estimate. For a normal sample of size \\(n\\), it has \\(n-1\\) degrees of freedom and approaches the normal distribution as the degrees of freedom increase.\n\nDistribution functions\n\nThe cumulative distribution function \\(F(x)=P(X\\le x)\\) applies to every real-valued random variable, discrete or continuous. It is non-decreasing, right-continuous, and rises from 0 to 1. For a continuous variable, \\(F\\) is the integral of the density; for a discrete variable, \\(F\\) jumps at each possible value by the probability mass there. This shared definition lets quantiles, tail probabilities and simulated samples be handled in one framework for both kinds of variable.\n\nLimit theorems\n\nThe law of large numbers states that the average of independent observations with a finite expected value converges to that expected value as the number of observations grows. Jakob Bernoulli proved a version in Ars Conjectandi, published posthumously in 1713. Later work produced the weak form, which gives convergence in probability, and the strong form, which gives almost-sure convergence. This explains why relative frequencies stabilize around probabilities.\n\nThe central limit theorem makes a stronger statement about shape. For independent, identically distributed variables with mean \\(\\mu\\) and finite variance \\(\\sigma^2\\), the standardized sample mean \\(\\sqrt{n}(\\bar X_n-\\mu)/\\sigma\\) converges in distribution to a standard normal variable, whatever the original distribution. De Moivre's 1733 normal approximation to binomial probabilities is the earliest special case, and the general theorem was established by Lindeberg and Lévy in the 1920s. Berry–Esseen bounds show that the error of the approximation decreases at the rate \\(1/\\sqrt n\\) when third moments are finite. This justifies normal approximations to sums and averages for moderate \\(n\\) when the summands are not strongly skewed. The approximation fails when variance is infinite. The Cauchy distribution is the standard counterexample: the mean of Cauchy observations has the same distribution as a single observation.\n\nSampling distributions and standard error\n\nA statistic computed from a sample is itself a random variable, and its distribution is its sampling distribution. For the sample mean of \\(n\\) independent observations, the sampling distribution has mean \\(\\mu\\) and variance \\(\\sigma^2/n\\). The standard deviation of that distribution is the standard error, \\(\\mathrm{SE}(\\bar X)=\\sigma/\\sqrt n\\). Since \\(\\sigma\\) is usually unknown, it is estimated by the sample standard deviation \\(s\\), giving \\(s/\\sqrt n\\). Because the standard error falls with the square root of \\(n\\), quadrupling the sample size halves the uncertainty of the mean.\n\nSampling distributions of other statistics are often less tractable. The asymptotic variance of the sample median, for example, depends on the density at the median, and bootstrap resampling is commonly used when no closed form is available. Under normality, \\((n-1)s^2/\\sigma^2\\) follows a chi-squared distribution with \\(n-1\\) degrees of freedom, and the studentized mean \\((\\bar X-\\mu)/(s/\\sqrt n)\\) follows a t distribution with \\(n-1\\) degrees of freedom. These exact results underlie many classical confidence intervals and tests."
    },
    {
      "heading": "Estimation theory",
      "text": "Point estimators and their criteria\n\nAn estimator is a rule that maps a sample \\(X_1,\\dots,X_n\\) to a value \\(\\hat\\theta = T(X_1,\\dots,X_n)\\) intended to approximate an unknown parameter \\(\\theta\\) of a probability distribution. The number it produces for a particular data set is called an estimate. The bias of an estimator is \\(E[\\hat\\theta]-\\theta\\), and an estimator is unbiased when this is zero. The sample mean is unbiased for the population mean, whatever the distribution, as long as the mean exists.\n\nConsistency concerns behaviour as the sample grows: \\(\\hat\\theta_n\\) should converge to \\(\\theta\\) in probability. The law of large numbers establishes this for the sample mean. Consistency says nothing about bias at a fixed sample size. The variance estimator that divides by \\(n\\) is biased for every finite \\(n\\), yet it is consistent.\n\nAmong unbiased estimators, the one with smaller variance is more efficient. The Cramér–Rao bound sets a floor: under regularity conditions, the variance of any unbiased estimator of \\(\\theta\\) based on \\(n\\) independent observations is at least \\(1/(nI(\\theta))\\), where \\(I(\\theta)\\) is the Fisher information carried by one observation. For normal data with known variance, the sample mean attains this bound. The sample median is also unbiased under symmetry but has asymptotic variance \\(\\pi\\sigma^2/(2n)\\), about \\(\\pi/2\\) times that of the mean, so it is less efficient though more robust to outliers.\n\nWhen biased and unbiased estimators must be compared, the mean squared error provides a single scale:\n\n\\[\\operatorname{MSE}(\\hat\\theta)=E[(\\hat\\theta-\\theta)^2]=\\operatorname{Var}(\\hat\\theta)+\\operatorname{Bias}(\\hat\\theta)^2 .\\]\n\nConstructing estimators\n\nThe method of moments, proposed by Karl Pearson in 1894, equates sample moments with the corresponding population moments and solves for the parameters. For a uniform distribution on \\([0,\\theta]\\), the mean is \\(\\theta/2\\), so the estimator is \\(\\hat\\theta = 2\\bar{X}\\). The method is simple and often consistent, but it can produce values inconsistent with the data. In this example, \\(2\\bar{X}\\) can fall below the largest observation, which is impossible under the model.\n\nThe maximum likelihood estimator (MLE) chooses the parameter value that maximizes the likelihood function \\(L(\\theta)=\\prod_{i=1}^{n} f(x_i;\\theta)\\), usually by maximizing its logarithm. Ronald Fisher made this approach central in the 1910s and 1920s. The likelihood is a function of \\(\\theta\\) for fixed data. It is not a probability distribution over \\(\\theta\\), a distinction that is often blurred in practice. Under standard conditions, the MLE is consistent, asymptotically normal, and asymptotically efficient, and it is invariant under reparameterization: if \\(\\hat\\theta\\) is the MLE of \\(\\theta\\), then \\(g(\\hat\\theta)\\) is the MLE of \\(g(\\theta)\\). It is not always unbiased. For normal data, the MLE of \\(\\sigma^2\\) is \\(\\frac{1}{n}\\sum_{i}(X_i-\\bar{X})^2\\), which is biased.\n\nSufficiency and information\n\nA statistic \\(T\\) is sufficient for \\(\\theta\\) when the conditional distribution of the data given \\(T\\) does not depend on \\(\\theta\\), so \\(T\\) retains all information the data carry about the parameter. The Fisher–Neyman factorization theorem gives a practical test: \\(f(x;\\theta)=g(T(x);\\theta)\\,h(x)\\), where \\(h\\) does not involve \\(\\theta\\). For Bernoulli trials, the number of successes is sufficient. For normal data with known variance, the sample mean is sufficient. The Rao–Blackwell theorem states that conditioning an unbiased estimator on a sufficient statistic cannot increase its variance. The Lehmann–Scheffé theorem extends this: when the sufficient statistic is also complete, an unbiased function of it is the minimum-variance unbiased estimator.\n\nFisher information measures how sharply the data constrain the parameter:\n\n\\[I(\\theta)=E\\!\\left[\\left(\\frac{\\partial}{\\partial\\theta}\\log f(X;\\theta)\\right)^{2}\\right]=-E\\!\\left[\\frac{\\partial^{2}}{\\partial\\theta^{2}}\\log f(X;\\theta)\\right],\\]\n\nand \\(n\\) independent observations carry \\(nI(\\theta)\\). Informally, the more strongly the log-likelihood curves at its peak, the more information the data contain. The asymptotic variance of the MLE is \\(1/(nI(\\theta))\\), which is why the MLE attains the Cramér–Rao bound in the limit.\n\nBias correction and the bias–variance trade-off\n\nBessel's correction addresses the bias of the sample variance. Since \\(E\\!\\left[\\frac{1}{n}\\sum_i (X_i-\\bar{X})^2\\right]=\\frac{n-1}{n}\\sigma^2\\), dividing by \\(n-1\\) instead gives an unbiased estimator \\(s^2\\). The intuition is that deviations measured from \\(\\bar{X}\\) are constrained to sum to zero, so estimating the mean uses up one degree of freedom. The correction does not carry over to the standard deviation: \\(s\\) is still biased for \\(\\sigma\\), because the square root is nonlinear.\n\nUnbiasedness is not always the goal. Since \\(\\operatorname{MSE}=\\operatorname{Bias}^2+\\operatorname{Var}\\), an estimator can accept a small bias to achieve a large reduction in variance, and this is the bias–variance trade-off. The James–Stein estimator, published in 1961, showed that when estimating a normal mean vector of dimension three or more, a deliberately biased shrinkage estimator has lower total mean squared error than the sample mean for every parameter value. Penalized methods in regression, such as ridge regression, apply the same logic: they introduce bias to control variance when predictors are many or strongly correlated."
    },
    {
      "heading": "Hypothesis testing and significance",
      "text": "Hypothesis testing is a procedure for deciding whether observed data are compatible with a specified claim about a population. The claim under test, the null hypothesis \\(H_0\\), usually asserts that a parameter takes a particular value or that two groups do not differ. The alternative hypothesis \\(H_1\\) states what is taken to hold if the null is rejected. A test statistic condenses the data into a single number whose distribution is known, or can be approximated, when \\(H_0\\) is true. For a sample mean \\(\\bar{x}\\) from \\(n\\) observations with known standard deviation \\(\\sigma\\), the statistic \\(z = (\\bar{x} - \\mu_0)/(\\sigma/\\sqrt{n})\\) follows a normal distribution under \\(H_0: \\mu = \\mu_0\\). When \\(\\sigma\\) must be estimated from the same data, the t distribution takes its place. Chi-squared statistics serve tests of independence in contingency tables, and F statistics appear in analysis of variance when several group means are compared.\n\nThe p-value is the probability, computed under \\(H_0\\), of obtaining a test statistic at least as extreme as the one observed. It is not the probability that \\(H_0\\) is true. Ronald Fisher treated the p-value as a graded measure of evidence against the null and suggested in 1925 that 5% was a convenient threshold. Jerzy Neyman and Egon Pearson instead framed testing as a decision rule with fixed error rates, which gave rise to the pre-specified significance level \\(\\alpha\\). The routine practice in applied research, which combines Fisher's p-value with a Neyman–Pearson-style cutoff, is a hybrid of the two approaches that the original authors did not fully endorse.\n\nErrors and power\n\nA decision based on a test can go wrong in two ways. A type I error rejects a true null hypothesis; by construction its probability is at most \\(\\alpha\\). A type II error fails to reject a false null, with probability \\(\\beta\\). The statistical power of a test is \\(1-\\beta\\), the probability of detecting an effect that is really present. For a fixed sample size, lowering \\(\\alpha\\) raises \\(\\beta\\), so the two error rates cannot both be reduced without collecting more data. Power grows with the true effect size, the sample size and the precision of measurement. This dependence is why power analysis is carried out before data collection to choose \\(n\\). The Neyman–Pearson lemma shows that the likelihood ratio test is the most powerful test for deciding between two simple hypotheses at a given \\(\\alpha\\).\n\nMultiple comparisons\n\nTesting many hypotheses at level \\(\\alpha\\) each inflates the chance of at least one false rejection. If \\(m\\) independent true nulls are each tested at \\(\\alpha\\), the family-wise error rate is \\(1-(1-\\alpha)^m\\). With \\(m=20\\) and \\(\\alpha=0.05\\), this exceeds 0.6. The Bonferroni correction tests each hypothesis at \\(\\alpha/m\\), which keeps the family-wise error rate at or below \\(\\alpha\\) without assuming anything about dependence among the tests. The cost is conservatism, and power falls quickly as \\(m\\) grows. Step-down refinements such as Holm's procedure control the same error rate while being uniformly at least as powerful.\n\nThe false discovery rate addresses a different target: the expected proportion of rejected hypotheses that are in fact true nulls. The Benjamini–Hochberg procedure, published in 1995, sorts the p-values as \\(p_{(1)} \\le \\dots \\le p_{(m)}\\), finds the largest \\(k\\) with \\(p_{(k)} \\le (k/m)q\\), and rejects the \\(k\\) smallest. Controlling the FDR rather than the family-wise error rate allows many more discoveries when thousands of tests are run together, as in genomics, at the price of tolerating some false positives among them.\n\nEffect sizes and their relation to significance\n\nStatistical significance reflects both the magnitude of an effect and the amount of data. With very large samples, a trivial difference can yield a very small p-value, while with small samples a substantial effect may fail to reach significance. An effect size measures the magnitude of a phenomenon on a scale that does not depend directly on \\(n\\). Common examples include the standardized mean difference \\(d = (\\bar{x}_1 - \\bar{x}_2)/s_p\\), where \\(s_p\\) is the pooled standard deviation, the correlation coefficient, and the odds ratio in logistic regression. Jacob Cohen's benchmarks of roughly 0.2, 0.5 and 0.8 for small, medium and large values of \\(d\\) are widely cited, but they are conventions and their meaning depends on the field. Reporting a confidence interval for the effect size conveys both its estimated size and its uncertainty, which a binary verdict of significant or not does not.\n\nEstimates from results selected for statistical significance tend to overstate the true effect, particularly when power is low. This is one reason why replication attempts often find smaller effects than the original reports."
    },
    {
      "heading": "Bayesian statistics",
      "text": "Bayesian statistics reads probability as a degree of belief about propositions, including statements about unknown quantities, rather than only as the long-run frequency of outcomes. Within this framework an unknown parameter \\(\\theta\\) is treated as a random quantity with its own distribution, and inference consists of revising that distribution as data arrive. The formal tool for the revision is Bayes' theorem, which for data \\(x\\) gives\n\n\\[p(\\theta \\mid x) = \\frac{p(x \\mid \\theta)\\,\\pi(\\theta)}{\\int p(x \\mid \\vartheta)\\,\\pi(\\vartheta)\\,d\\vartheta}.\\]\n\nHere \\(\\pi(\\theta)\\) is the prior distribution, \\(p(x \\mid \\theta)\\) is the likelihood, and \\(p(\\theta \\mid x)\\) is the posterior distribution. The denominator does not depend on \\(\\theta\\), so the rule is often written as posterior proportional to likelihood times prior. Because the posterior of one analysis can serve as the prior of the next, the approach handles sequential data naturally.\n\nThe method is older than the term. Thomas Bayes's essay on inverse probability appeared posthumously in 1763, and Pierre-Simon Laplace developed the approach extensively in the late eighteenth and early nineteenth centuries. Its standing fell during the period when frequentist methods dominated, and it revived strongly from the 1970s, when affordable computers made the calculations feasible. Debate over the interpretation of probability continues, and that debate is discussed in a separate section.\n\nThe prior is the most contested input. Priors can be subjective, encoding expert judgement, or objective and designed to exert little influence on the result; weakly informative priors sit between these poles. Analysts routinely check how sensitive conclusions are to the choice. Where the prior and likelihood belong to particular families, the posterior is available in closed form. A conjugate prior is one whose posterior stays in the same family. A standard example is the beta–binomial pair: if \\(\\theta\\) has a \\(\\mathrm{Beta}(a,b)\\) prior and \\(k\\) successes occur in \\(n\\) trials, the posterior is \\(\\mathrm{Beta}(a+k,\\, b+n-k)\\). Normal–normal and gamma–Poisson pairs have the same property. Once a posterior is fitted, a posterior predictive distribution\n\n\\[p(\\tilde{x} \\mid x) = \\int p(\\tilde{x} \\mid \\theta)\\,p(\\theta \\mid x)\\,d\\theta\\]\n\ndescribes future or replicated data. Posterior predictive checks compare simulated replicates with the observed data to reveal model misfit.\n\nBayesian and frequentist intervals differ mainly in what their probability refers to. A credible interval is a range that contains a stated share of the posterior mass, such as 95%. Its statement is conditional on the data, the model and the prior: given these, the parameter lies in the interval with the stated probability. A confidence interval instead describes the long-run behaviour of the procedure: across repeated samples, intervals built this way would contain the fixed true value at the stated rate. Numerically the two can coincide, for example under a flat prior and a symmetric sampling distribution, but their meanings remain different.\n\nFor comparing models, the Bayes factor is the ratio of marginal likelihoods,\n\n\\[BF_{12} = \\frac{p(x \\mid M_1)}{p(x \\mid M_2)},\\]\n\nand it turns prior odds into posterior odds. Because marginal likelihoods average over the whole parameter space, the Bayes factor penalizes models that spread their prior over implausible values, which gives an automatic preference for parsimony. The same averaging makes it sensitive to the prior in ways that posterior estimation is not, so it requires care.\n\nApart from conjugate cases, posteriors usually have no closed form. Markov chain Monte Carlo methods make the problem tractable by generating a dependent sequence of parameter values whose long-run distribution is the posterior. The Metropolis algorithm, published in 1953, and its extension by Hastings in 1970 provided the basic recipe. Their use in mainstream statistics expanded after Gelfand and Smith's 1990 paper on sampling-based calculation of marginal densities, which drew on Gibbs sampling as a practical scheme, Gibbs sampling, to make hierarchical Bayesian models routinely computable. Modern software built on these ideas, and analysts check convergence and mixing before trusting the output."
    },
    {
      "heading": "Regression and linear models",
      "text": "Regression analysis models how the distribution of a response variable \\(Y\\) depends on one or more predictors \\(X\\). The most common form is the linear model, in which the conditional mean of the response is a weighted sum of the predictors. The simple linear regression model is \\(y_i = \\beta_0 + \\beta_1 x_i + \\varepsilon_i\\). With \\(p\\) predictors, the multiple linear regression model is written as \\(\\mathbf{y} = X\\boldsymbol{\\beta} + \\boldsymbol{\\varepsilon}\\), where the first column of the design matrix \\(X\\) is all ones so that the intercept is included. The error term \\(\\boldsymbol{\\varepsilon}\\) collects whatever the predictors do not explain.\n\nOrdinary least squares and its assumptions\n\nOrdinary least squares (OLS) chooses \\(\\hat{\\boldsymbol{\\beta}}\\) to minimize the sum of squared residuals. When \\(X^\\top X\\) is invertible, this gives the closed form \\(\\hat{\\boldsymbol{\\beta}} = (X^\\top X)^{-1} X^\\top \\mathbf{y}\\). Legendre published the least-squares method in 1805, and Gauss linked it to the probability of errors. Under linearity in parameters, zero conditional mean of the errors given \\(X\\), constant error variance, uncorrelated errors and no perfect multicollinearity, the Gauss–Markov theorem states that OLS is the best linear unbiased estimator. Normally distributed errors are not required for this result. They are required for exact small-sample inference with t and F tests. In large samples, inference instead relies on the central-limit-theorem.\n\nViolations of these assumptions have different consequences. Heteroskedasticity, meaning non-constant error variance, leaves coefficient estimates unbiased but makes the usual standard errors unreliable, which is why robust standard errors are common. Correlation between a predictor and the error term, often caused by omitted variables, biases the estimates. That problem is a matter of causal-inference and confounding rather than of fitting alone.\n\nGeneralized linear models\n\nGeneralized linear models (GLMs), formalized by Nelder and Wedderburn in 1972, extend the framework to responses that are binary, counts or otherwise non-normal. The response is assumed to follow an exponential-family distribution, and a link function \\(g\\) relates its mean \\(\\mu_i\\) to a linear predictor: \\(g(\\mu_i) = \\mathbf{x}_i^\\top \\boldsymbol{\\beta}\\). Parameters are usually estimated by maximum-likelihood-estimation.\n\nFor a binary outcome, logistic-regression uses the logit link, \\(\\log\\frac{p_i}{1-p_i} = \\mathbf{x}_i^\\top \\boldsymbol{\\beta}\\), with a binomial (often Bernoulli) response. Exponentiating a coefficient gives an odds ratio: \\(e^{\\beta_j}\\) is the multiplicative change in the odds of the outcome for a one-unit increase in \\(x_j\\), other predictors held fixed. For counts, log-linear Poisson regression uses \\(\\log \\mu_i = \\mathbf{x}_i^\\top \\boldsymbol{\\beta}\\), so \\(e^{\\beta_j}\\) is a rate ratio. The Poisson model assumes the variance equals the mean. When counts are overdispersed, quasi-Poisson or negative binomial models are used instead. An offset term can adjust for different exposure times.\n\nRegularized regression\n\nWhen predictors are numerous relative to the number of observations, or strongly correlated, OLS estimates can have very high variance, and the matrix \\(X^\\top X\\) may be nearly singular. regularization adds a penalty to the least-squares criterion. Ridge regression penalizes \\(\\lambda\\sum_j \\beta_j^2\\), which shrinks all coefficients smoothly toward zero and spreads weight across correlated predictors. Lasso penalizes \\(\\lambda\\sum_j |\\beta_j|\\), which can set some coefficients exactly to zero and so performs variable selection. Its choice among correlated predictors can be unstable. elastic-net combines both penalties, \\(\\lambda\\left(\\alpha\\sum_j |\\beta_j| + \\frac{1-\\alpha}{2}\\sum_j \\beta_j^2\\right)\\), and tends to keep groups of correlated predictors together. The penalty strength \\(\\lambda\\) is usually chosen by cross-validation. Predictors should be standardized first, because the penalty depends on the scale of each variable. Each method trades some bias for lower variance, in line with the bias-variance-tradeoff.\n\nInterpreting fitted models\n\nIn a linear model, \\(\\beta_j\\) is the expected change in \\(Y\\) for a one-unit change in \\(x_j\\), with the other predictors held fixed. That phrase describes a statistical adjustment and does not by itself establish causation. Standardized coefficients express effects in standard-deviation units, which allows rough comparison across predictors measured on different scales.\n\nThe coefficient of determination, \\(R^2 = 1 - \\frac{SS_{\\text{res}}}{SS_{\\text{tot}}}\\), gives the share of variance in the sample that the model explains. It never decreases when predictors are added, so adjusted \\(R^2\\) or out-of-sample prediction error is often more informative. A high \\(R^2\\) does not show that the model is correctly specified.\n\nResidual diagnostics check the assumptions. A plot of residuals against fitted values reveals nonlinearity and changing variance. A normal Q–Q plot compares residuals with a normal reference. Leverage and Cook's distance identify influential observations. Variance inflation factors measure multicollinearity, and the Durbin–Watson statistic tests for autocorrelation in data ordered in time. Problems revealed by these tools are typically addressed by transforming variables, adding terms or changing the error model."
    },
    {
      "heading": "Causal inference and research design",
      "text": "Statistics separates association from causation. Two variables may move together because one affects the other, because a third factor affects both, or by chance. Correlation measures how closely variables move together but says nothing about direction or mechanism. The main obstacle is confounding: a third variable that influences both the suspected cause and the outcome can produce an association where no effect exists, or hide a real one. In 1854, John Snow compared cholera mortality among households supplied by two London water companies whose intakes differed in their exposure to sewage. The Berkeley graduate admissions data from 1973 show the opposite danger. Pooled figures suggested bias against women, but within most departments the difference reversed or disappeared, because women applied disproportionately to departments with low admission rates. This pattern is known as Simpson's paradox.\n\nPotential outcomes and structural models\n\nThe potential outcomes framework, developed by Jerzy Neyman in 1923 and later extended by Donald Rubin in the 1970s, defines a causal effect as a comparison between the outcome a unit would have under treatment and the outcome it would have without it. Only one of these counterfactual outcomes is ever observed for a given unit, which Paul Holland later called the fundamental problem of causal inference. Researchers therefore target average effects across groups and rely on assumptions that make the unobserved comparison credible. Judea Pearl's structural causal models take a graphical route instead. Causal assumptions are encoded in directed acyclic graphs, and a backdoor criterion identifies which variables must be adjusted for. The same graphs warn against conditioning on colliders, which can create spurious associations. Methods such as propensity scores estimate the probability of treatment given observed covariates so that treated and untreated units can be balanced on those covariates.\n\nRandomized controlled trials and their limits\n\nA randomized controlled trial remains the standard for causal claims because randomization makes treatment assignment independent of both observed and unobserved characteristics, at least in expectation. Ronald Fisher's work on experimental design made this logic central to modern practice. In practice, trials face several limits. They can be expensive, slow to report on long-term outcomes, and ethically impossible for exposures such as smoking. Participants may not comply with assigned treatments, and some drop out, which breaks the simple comparison and forces analysts to choose between intention-to-treat estimates and methods that model compliance. Results obtained in a particular population may not transfer to another, and interference between units can violate the assumption that one person's treatment does not affect another's outcome.\n\nObservational designs\n\nWhen randomization is not feasible, researchers look for natural experiments, situations in which nature or policy assigns treatment in a way that resembles random assignment. Snow's cholera study is an early example. Joshua Angrist's study of the 1969 United States draft lottery used the randomized order of birthdates to estimate the effect of military service on later earnings.\n\nInstrumental variables use a third variable that affects the outcome only through the treatment and is associated with it. Angrist and Alan Krueger used quarter of birth as an instrument for years of schooling. The method requires relevance, which means the instrument must predict treatment, and an exclusion restriction, which means it must affect the outcome only through treatment. With heterogeneous effects, the estimate applies to the subgroup whose treatment status the instrument changes, a quantity known as the local average treatment effect.\n\nDifference-in-differences compares the change over time in a treated group with the change in a control group. Its validity depends on the parallel trends assumption, namely that both groups would have followed similar paths without the intervention. David Card and Krueger's 1994 study of a New Jersey minimum-wage increase, compared with neighboring Pennsylvania, is a well-known application.\n\nRegression discontinuity designs exploit a cutoff on a running variable that determines treatment, such as a test-score threshold for an award. Donald Thistlethwaite and Campbell's 1960 study of National Merit Scholarships is an early example. Units just above and below the threshold are assumed to be comparable, so the jump in outcomes at the cutoff estimates a local effect. Analysts must check that units cannot manipulate their position relative to the cutoff and must choose a bandwidth with care.\n\nRobert LaLonde's 1986 comparison of nonexperimental estimates with the results of a randomized job-training evaluation showed that standard regression adjustments could diverge widely from the experimental benchmark. This finding helped motivate the design-based approaches described above."
    },
    {
      "heading": "Time series and stochastic processes",
      "text": "A time series is a sequence of observations ordered in time, written \\(y_1, y_2, \\dots, y_T\\). Classical inference often assumes independent samples, but successive values of a time series are usually dependent, so the ordering carries information and methods that ignore it can mislead. The formal model behind such data is a stochastic process, a family of random variables indexed by time.\n\nComponents and stationarity\n\nMany observed series are decomposed into a few parts. In the classical additive form, \\(y_t = T_t + S_t + R_t\\), \\(T_t\\) is a slowly varying trend, \\(S_t\\) is a seasonal pattern with a fixed period (for example 12 for monthly data), and \\(R_t\\) is an irregular remainder. A multiplicative form suits series whose seasonal swings grow with the level. Procedures such as X-11 and STL estimate these parts, but they are descriptive tools and do not specify how the data were generated.\n\nMuch of the theory depends on stationarity. A series is weakly stationary when its mean is constant, its variance is finite and constant, and its autocovariance \\(\\gamma(h) = \\operatorname{Cov}(y_t, y_{t+h})\\) depends only on the lag \\(h\\). Trends, changing variance and seasonality all violate this condition. Non-stationary series are often made stationary by differencing, \\(\\nabla y_t = y_t - y_{t-1}\\), a step central to the Box–Jenkins approach. Unit roots can be tested formally. The augmented Dickey–Fuller test takes a unit root as its null hypothesis, while the KPSS test takes stationarity as its null, so the two are often applied together.\n\nAutoregressive, moving average and state-space models\n\nThe basic building block is white noise \\(\\varepsilon_t\\), a sequence of uncorrelated, zero-mean variables with constant variance. An autoregressive model of order \\(p\\), AR(p), predicts the current value from past values: \\(y_t = c + \\phi_1 y_{t-1} + \\dots + \\phi_p y_{t-p} + \\varepsilon_t\\). For AR(1) with \\(|\\phi|<1\\), the stationary variance is \\(\\sigma^2/(1-\\phi^2)\\). A moving average model MA(q) instead writes \\(y_t = \\mu + \\varepsilon_t + \\theta_1 \\varepsilon_{t-1} + \\dots + \\theta_q \\varepsilon_{t-q}\\), so the series responds to past shocks. Combining the two yields ARMA(p,q), and differencing \\(d\\) times yields ARIMA(p,d,q). Order identification uses the autocorrelation and partial autocorrelation functions: the ACF of an MA(q) process cuts off after lag \\(q\\), and the PACF of an AR(p) process cuts off after lag \\(p\\). Estimation is usually by maximum likelihood or least squares, and competing orders are compared with AIC or BIC.\n\nState-space models separate an unobserved state from its noisy measurement: \\(\\alpha_{t+1} = T\\alpha_t + \\eta_t\\) and \\(y_t = Z\\alpha_t + \\varepsilon_t\\). The Kalman filter updates the conditional distribution of the state recursively as data arrive, and it handles missing values and irregular spacing. Forecasts are conditional expectations given past data. For a stationary model, forecast error variance grows with the horizon and levels off at the unconditional variance, so long-range forecasts revert to the mean. Because shuffling observations destroys time order, forecasts are assessed with rolling-origin validation rather than ordinary cross-validation.\n\nMarkov chains and random walks\n\nA Markov chain is a discrete-time process whose next state depends only on the present state, a condition known as the Markov property. Given a transition matrix \\(P\\), the distribution after \\(n\\) steps is governed by \\(P^n\\). An irreducible, aperiodic, positive recurrent chain converges to a unique stationary distribution \\(\\pi\\) satisfying \\(\\pi P = \\pi\\), whatever its starting state. This convergence underlies Markov chain Monte Carlo methods, which build a chain whose stationary distribution is the target posterior.\n\nA random walk is the partial sum \\(S_n = X_1 + \\dots + X_n\\) of independent, identically distributed steps. Its variance is \\(\\operatorname{Var}(S_n) = n\\sigma^2\\), which grows without bound, so the walk is non-stationary. An AR(1) model with \\(\\phi = 1\\) is exactly a random walk. The symmetric simple random walk on the integers returns to its origin with probability one, but in three or more dimensions it does not, a result due to Pólya in 1921. Adding a drift term or a stationary component to a random walk gives many practical models of level-type data.\n\nApplications in finance, climate and signal analysis\n\nIn finance, log prices are often modeled as random walks, an idea traceable to Bachelier's 1900 thesis on speculation, and returns are close to uncorrelated. Squared and absolute returns, however, are strongly autocorrelated, a pattern called volatility clustering. Engle's ARCH model (1982) and Bollerslev's GARCH generalization (1986) capture it by letting conditional variance depend on past shocks. Engle shared the 2003 Nobel Memorial Prize in Economic Sciences with Clive Granger, whose work on cointegration addressed common stochastic trends.\n\nIn climate records, global temperature and sea-level series combine trends, strong serial dependence and multi-year oscillations. Ignoring autocorrelation overstates the effective sample size and produces spuriously narrow uncertainty bands. The choice between a deterministic trend and a random-walk-like accumulation also changes how a warming signal is interpreted. Long-range dependence was documented by Hurst in his 1951 analysis of Nile discharge records, which later motivated fractionally integrated models.\n\nIn signal analysis, the spectral density is the Fourier transform of the autocovariance function (the Wiener–Khinchin theorem), linking time-domain and frequency-domain descriptions. Periodograms reveal hidden periodicities, while the Wiener and Kalman filters extract signals from noise."
    },
    {
      "heading": "Multivariate methods",
      "text": "Multivariate methods analyze several variables at once, so that their joint behavior, and not only each variable taken separately, becomes the object of study. The basic tool is the covariance matrix of a random vector. Its diagonal holds the variance of each variable, and its off-diagonal entries record how each pair of variables moves together. The correlation between two variables is the same information rescaled to lie between −1 and 1, which allows pairs measured in different units to be compared. Because a covariance matrix is symmetric and positive semidefinite, it can be decomposed into orthogonal directions with associated magnitudes. This property underlies most of the techniques below. Much of the classical theory assumes a multivariate-normal-distribution, since its linear combinations remain normal and its conditional distributions have explicit forms.\n\nPrincipal component analysis (PCA) addresses the problem of too many correlated variables. Karl Pearson introduced the idea in 1901 in a geometric form: he sought a line or plane that minimizes the squared perpendicular distances from the data points. Harold Hotelling gave an algebraic formulation in 1933, in which each component is a linear combination of the original variables that has maximal variance among combinations orthogonal to the earlier components. Solving this problem amounts to finding the eigenvectors of the covariance matrix, which can be written as \\[\\Sigma v = \\lambda v\\] where the eigenvalue \\(\\lambda\\) gives the variance captured along direction \\(v\\). The resulting components are uncorrelated and ordered by the share of total variance they explain. Analysts usually retain only the first few, guided by the cumulative proportion of variance explained or by a scree plot. Interpretation relies on the loadings, which show how strongly each original variable contributes to each component. Because the results depend on the scale of the variables, data are often standardized first. PCA is a dimensionality-reduction method that uses no outcome labels, and it is linear; nonlinear and sparse variants extend the same basic idea.\n\nCluster analysis seeks groups among observations when no class labels are available. The k-means algorithm alternates between assigning each observation to its nearest centroid and recomputing each centroid as the mean of its assigned points. It reduces the total within-cluster sum of squares, but it only reaches a local optimum, its outcome depends on the starting values, and it implicitly favors compact, roughly spherical clusters of similar size. A gaussian-mixture-model instead treats the data as a weighted mixture of multivariate normal components, each with its own mean and covariance matrix. Its parameters are usually estimated by the expectation–maximization algorithm, and it yields soft, probabilistic memberships. Because each component has a full covariance matrix, it can represent elliptical clusters that k-means cannot. Choosing the number of clusters is a separate problem without a single correct answer. Common aids include the \"elbow\" in the within-cluster sum of squares, silhouette widths, information criteria such as the Bayesian information criterion for mixture models, and checks of how stable the solution is across resamples.\n\nDiscriminant analysis addresses the reverse situation, in which group membership is known and the goal is to classify new observations. Ronald Fisher's 1936 paper used four measurements of iris flowers from three species to show how a linear combination of variables could separate groups. His criterion chose the direction that maximizes the ratio of between-group separation to within-group spread. Linear discriminant analysis, the standard descendant of this approach, assumes that the classes share a common covariance matrix and that each class has an approximately normal distribution. Under these assumptions the decision boundary is linear, and an observation is assigned to the class with the highest posterior probability, which combines the class densities with prior probabilities. Relaxing the common-covariance assumption produces quadratic discriminant analysis, which yields curved boundaries at the cost of estimating more parameters. Classification performance is usually judged with a confusion matrix computed on data not used for fitting, often through cross-validation, rather than by the proportion of correct classifications on the training data alone.\n\nSeveral practical issues recur across these methods. When the number of variables is large relative to the number of observations, the sample covariance matrix becomes unstable, and its eigenvalues are biased, so shrinkage estimators or dimension reduction are often required. Estimated covariances also depend on the chosen divisor, a point connected to Bessel's correction. Multivariate summaries can conceal outliers that look ordinary in each variable separately, so diagnostic plots and robust estimation are important. Finally, the components of PCA and the clusters found by an algorithm are descriptive constructs whose meaning depends on the substantive domain, and they should be validated rather than treated as established structure."
    },
    {
      "heading": "Statistical learning and machine learning",
      "text": "The boundary between classical statistics and machine learning is largely one of emphasis. Both fields fit models to data, describe error with probability, and share many tools, including linear-regression, logistic-regression, principal-component-analysis and k-means-clustering. The difference lies mainly in what counts as a good model. Classical inference usually asks what a parameter means and how precisely it is estimated. Machine learning usually asks how accurately a rule predicts outcomes for cases it has not seen. Leo Breiman's 2001 essay made this contrast explicit. It separated models that posit a stochastic data-generating model from algorithmic approaches that leave the data mechanism unspecified, and it argued that the statistical community had leaned too heavily on the first kind.\n\nMost learning problems are formulated as either supervised or unsupervised. In supervised-learning, each input comes with a known output, and the task is to learn a function that maps inputs to outputs. The quality of a candidate function is measured by a loss function \\(L\\), and the fitted rule is often chosen to minimize the average loss over the training sample, a principle known as empirical-risk-minimization:\n\n\\[\\hat f = \\arg\\min_{f \\in \\mathcal{F}} \\frac{1}{n}\\sum_{i=1}^{n} L\\big(y_i, f(x_i)\\big)\\]\n\nSquared-error loss leads to estimates of conditional means and to mean-squared-error as the natural yardstick, while absolute loss targets conditional medians. In unsupervised-learning there are no labels, and the aim is to uncover structure through cluster-analysis, dimensionality-reduction or density modeling. Prediction and explanation also pull in different directions. A model with thousands of uninterpretable parameters, such as a deep-learning network, may forecast well without offering any account of mechanism. An explanatory claim, by contrast, needs interpretable parameters and an argument for causation rather than mere association.\n\nAssessing predictive performance requires guarding against two failure modes. overfitting occurs when a model learns the noise in its training sample, so its training error keeps shrinking while its error on new data grows. It learns the patterns in the specific training data but generalizes poorly to unseen data. underfitting is the opposite problem, in which a model is too rigid to capture real structure. Their trade-off is the same tension that appears in the bias-variance-tradeoff. regularization addresses it by penalizing complexity, as in ridge-regression, lasso-regression and elastic-net. To estimate out-of-sample accuracy, practitioners use cross-validation. The data are split into folds, the model is fitted on all but one fold, and its error is scored on the held-out fold, with results averaged over folds. bootstrap-sampling, introduced by Bradley Efron in 1979, resamples the data with replacement to gauge the variability of an estimate or of a model's apparent performance. Learning curves, which plot error against training-set size, show whether more data or a more flexible model is the better remedy.\n\nData leakage is the most common way these estimates become misleading. Leakage happens when information from outside the training dataset is used to create the model. The effect is an optimistic assessment that does not survive deployment. Typical sources include preprocessing that uses statistics computed over the full dataset before splitting, feature selection performed before the cross-validation loop, and repeated measurements from the same individual appearing in both training and validation sets. Early stopping can also leak: if the validation fold is used to choose the stopping iteration and the same fold is then reported as the final score, the evaluation data have shaped the training. Time-dependent data raise a further problem, because a random split can place future observations in the training set. Financial applications therefore use purged cross-validation, which removes overlapping observations and optionally adds a temporal buffer around the test set.\n\nLarge data sets do not eliminate these risks. Very large models can interpolate their training data and still predict well, a pattern that has been studied under the heading of double descent and that has complicated older intuitions about overfitting. Large data sets also make it easier to search many candidate models, which raises the chance of selecting one that merely fits the validation data by luck. Rigorous practice therefore keeps a final test set untouched, places every data-dependent step inside the resampling loop, and judges generalization on data drawn from the conditions in which the model will actually be used. The relevant concept here is generalization, and the methods discussed above are the principal means of checking it."
    },
    {
      "heading": "Computational methods and resampling",
      "text": "Resampling and the bootstrap\n\nResampling methods estimate uncertainty by recomputing a statistic many times on data, instead of relying on closed-form formulas. Their intellectual roots predate the modern versions. Use of the bootstrap idea goes back at least to Simon (1969), who used it as a tool to teach statistics. The jackknife, which recomputes an estimate with one observation left out at a time, dates to the late 1940s, and subsampling to the 1960s (Quenouille (1949) for the jackknife and Hartigan (1969) and McCarthy (1969) for subsampling). Bradley Efron's 1979 paper gave the bootstrap its general form, and its motivation was partly to understand the jackknife's properties the motivation for Efron's paper was to better understand the jackknife and its properties.\n\nThe basic empirical bootstrap treats the observed sample as a stand-in for the unknown population. Given a sample \\(x_1,\\dots,x_n\\) and an estimator \\(\\hat\\theta\\), one draws \\(B\\) new samples of size \\(n\\) with replacement, computes \\(\\hat\\theta^{*b}\\) on each, and estimates the standard error from the spread of these replicates:\n\n\\[\\widehat{\\operatorname{se}}_B=\\sqrt{\\frac{1}{B-1}\\sum_{b=1}^{B}\\bigl(\\hat\\theta^{*b}-\\bar\\theta^{*}\\bigr)^2},\\qquad \\bar\\theta^{*}=\\frac{1}{B}\\sum_{b=1}^{B}\\hat\\theta^{*b}.\\]\n\nPercentile confidence intervals are obtained in the same way, from quantiles of the replicates. The method is valuable precisely where analytic results are unavailable: the bootstrap is a ubiquitous technique to estimate performance characteristics of complex statistical methods which often do not admit simple closed form solutions. It is not universally reliable, however. It can fail for statistics driven by extreme values, such as the sample maximum, and it requires adaptation, such as block resampling, when observations are dependent, as in time series.\n\nPermutation tests\n\nPermutation tests draw their force from a different principle. If the null hypothesis holds that group labels are exchangeable (for instance, that treatment assignment makes no difference to the outcome), then the labels can be shuffled without changing the joint distribution of the data. The test statistic is computed for every rearrangement, or for a large random sample of them, and the p-value is the share of rearrangements that yield a result at least as extreme as the observed one. Ronald Fisher used this logic in The Design of Experiments (1935), notably in the \"lady tasting tea\" example, and Edwin Pitman formalized it for general use in the late 1930s. Because the reference distribution is built from the data themselves, permutation tests make few assumptions about the population shape, and they are exact for small samples when all arrangements can be enumerated. The cost is that they test a sharp null hypothesis about exchangeability and do not, by themselves, supply interval estimates.\n\nMonte Carlo simulation\n\nMonte Carlo methods approximate quantities by random sampling. The approach emerged at Los Alamos in the 1940s. First conceived in 1946 by Stanislaw Ulam at Los Alamos and subsequently developed by John von Neumann, Robert Richtmyer, and Nick Metropolis, the method was first used to calculate neutron diffusion paths for the hydrogen bomb. Ulam's starting point was a card-game calculation: playing a certain number of hands and computing the empirical success probability offered an easier route than exact combinatorics. Nicholas Metropolis coined the name \"Monte Carlo\", and the 1949 paper by Metropolis and Ulam set out the method publicly.\n\nThe core idea is to write an integral as an expectation and estimate it by averaging over random draws. Randomly sample a volume in d-dimensional space to obtain an estimate of an integral at the price of a statistical error. The error shrinks at the rate \\(1/\\sqrt{N}\\) in the number of draws \\(N\\), regardless of dimension, which is why the method outperforms any other integration scheme when the dimension is very large. Practical work depends on generating draws from nonuniform distributions, for which inversion and accept-reject techniques are standard. Simulation is also the workhorse for checking statistical procedures: analysts generate synthetic datasets under known parameters to estimate a test's actual Type I error rate, the power of a design, or the coverage of a confidence interval. Markov chain Monte Carlo, which extends these ideas to high-dimensional posterior distributions, is treated elsewhere in this entry.\n\nStatistical computing and large-scale data\n\nResampling and simulation became routine only after cheap computing. Early bootstrap work required substantial manual effort, and the software that now underpins it was developed afterward. Today the main environments are R, with its large package ecosystem; Python, through libraries such as NumPy, SciPy and statsmodels; and commercial packages such as SAS, SPSS and Stata. Probabilistic programming systems implement Bayesian sampling directly. For very large datasets, computations are distributed across clusters, and approximations such as subsampling or stochastic gradient methods trade some precision for feasibility. These tools also raise reproducibility questions, since results depend on random seeds, software versions and the number of replicates.\n\nHandling missing data\n\nMost real datasets contain gaps, and the appropriate remedy depends on why values are missing. Donald Rubin's classification distinguishes three mechanisms. Missing at random means the missingness is independent of the value of the missing data, though it may depend on other observed variables that explain it. Missing not at random means the missingness depends on both observed and unobserved data. A third category, missing completely at random, requires that missingness be unrelated to any variable, observed or not. Complete-case analysis, which simply discards incomplete records, is unbiased only under the strictest of these assumptions.\n\nMultiple imputation addresses the uncertainty created by missing values rather than hiding it. Multiple copies of the dataset are created with the missing entries replaced by slightly different imputed values in each copy. Multiple imputed datasets are then analyzed independently and the results combined. The combination follows Rubin's rules: if \\(\\hat Q_i\\) and \\(U_i\\) are the estimate and its variance in the \\(i\\)-th of \\(m\\) completed datasets, the pooled estimate is \\(\\bar Q=\\frac{1}{m}\\sum_i \\hat Q_i\\), and its total variance is \\(T=\\bar U+\\left(1+\\frac{1}{m}\\right)B\\), where \\(\\bar U\\) is the mean within-imputation variance and \\(B\\) the between-imputation variance. Multiple imputation yields valid inference when data are missing at random, and when missing data are MNAR, multiple imputations do not lead to valid results. The difficulty is that the two cases cannot be separated from the observed data alone, since the boundary between MAR and MNAR cannot be firmly established because doing so requires observing the unobserved data. Analysts therefore rely on substantive reasoning and sensitivity analyses that test how conclusions change under plausible departures from the missing-at-random assumption."
    },
    {
      "heading": "Applications in science, medicine and public policy",
      "text": "Epidemiology and clinical research\n\nEpidemiology depends on a small set of counting measures, and the choice among them changes what a figure means. Prevalence describes how many people hold a condition at a chosen moment or across a chosen window, and it is typically stated as a proportion of the population studied Prevalence is the proportion of a particular population found to be affected by a medical condition at a specific time. Incidence instead counts newly arising cases, with the denominator limited to people who are still at risk Incidence is the number of new cases of a disease in a specific period of time, divided by the susceptible population. The two answer different questions: prevalence reflects the burden of disease at a given time, while incidence reflects the risk and the speed at which new cases appear Prevalence tells you the total burden of disease at a given moment, while incidence tells you how quickly new cases are appearing.\n\nCumulative incidence, also called the incidence proportion, estimates the probability of developing a condition over a fixed span. In one illustration, 28 new cases among 1,000 people over two years corresponds to 2.8% For example, if a population contains 1,000 persons and 28 develop a condition from the time the disease first occurred until two years later, the cumulative incidence is 28 cases per 1,000 persons, i.e. 2.8%. The incidence rate divides new cases by total person-time at risk, which allows studies with staggered entry and dropout to be analysed without requiring every participant to be followed for the full period The incidence rate can be calculated by dividing the number of subjects developing a disease by the total time at risk from all patients.\n\nClinical trials apply the same statistical machinery at the level of individual patients. Before enrollment, investigators specify the smallest effect worth detecting, the tolerable false-positive rate and the desired power, and these choices fix the sample size. Randomization guards comparisons against confounding, and a pre-specified analysis plan limits selective reporting. Trials that look at accumulating data at interim points need adjusted significance thresholds, so that early stopping does not inflate the chance of a false positive (see multiple-testing). Outcomes such as mortality rates and survival times are analysed with the same probabilistic tools used elsewhere in the field.\n\nOfficial statistics and the consumer price index\n\nNational statistical offices produce official statistics. A census attempts to enumerate an entire population, while sample surveys estimate household and business quantities that cannot be counted directly. Census listings also serve as the sampling frames for later surveys, which ties official data collection closely to the sampling methods described elsewhere in this article.\n\nThe consumer price index is a clear case of statistical construction. Its item list comes from household spending records; in the United States, the Consumer Expenditure Survey covers about 7,000 households, which provides detailed information on spending habits. Prices are gathered from a sample of outlets, and when a sampled product is swapped for a different version, compilers may adjust the recorded price so that quality differences are not mistaken for inflation When a sampled item is replaced by one of a different quality, compilers may adjust its price so that the index reflects price change rather than the quality difference. The index itself is a ratio: the current cost of the fixed basket divided by its base-period cost, with the result scaled by one hundred the price of an item at a given year is divided by that of the initial year, then multiplied by one hundred. Economists still regard the index as an approximation, since it is a useful tool for tracking these economic indicators, though it is not a perfect measure of inflation or the cost of living.\n\nEconomics, national accounts and labour statistics\n\nIn economics, national accounts supply the framework for measuring output. Gross domestic product counts only final goods and services, so that intermediate inputs are excluded and sales at each stage of a supply chain are not counted repeatedly It measures only the value of final goods and services, and does not \"double count\" items as they are bought and sold among manufacturers, distributors and retailers. The same total can be reached from three directions: by measuring expenditure on final goods and services, by adding up the income generated by production, or by adding up the value added in production. On the spending side, the familiar identity adds consumption, investment, government purchases and net exports, usually written as C + I + G + (X − M) GDP = C + I + G + (X – M).\n\nModern national accounting took shape in the twentieth century. In 1934, Kuznets presented a report to the US Congress that laid the foundations for the measurement of national income. The international rules are set out in the 1993 System of National Accounts, which several multilateral bodies compiled jointly The international standard for measuring GDP is contained in the System of National Accounts, 1993, compiled by the International Monetary Fund, the European Commission, the Organization for Economic Cooperation and Development, the United Nations, and the World Bank. Estimates are also revised as more data arrive; large revisions are uncommon, but they do occur revisions of such magnitude are \"relatively rare,\" estimating that only 10 percent of quarterly revisions are that large.\n\nLabour-market indicators such as the unemployment rate are produced from household and establishment surveys, and econometric methods are used to estimate how policies or shocks affect these outcomes (see econometrics and causal-inference).\n\nIndustrial quality control, risk and insurance\n\nIndustrial quality control applies statistics to production processes. Walter Shewhart introduced control charts at Bell Laboratories in the 1920s to distinguish ordinary variation from assignable causes, and acceptance sampling lets a buyer or producer accept or reject a lot after inspecting a random portion of it. Both methods depend on the trade-off between false alarms and missed defects.\n\nRisk assessment combines probability distributions for loads, failure times or exposures and often uses Monte Carlo simulation to estimate the probability that a threshold is exceeded. Insurance pricing rests on the same foundations. Actuaries model claim frequency, often with a Poisson distribution, and model claim sizes with heavier-tailed distributions. Premiums are set so that expected claims, administrative costs and a margin are covered, and pooling many policies reduces the relative variability of total losses, as the law of large numbers predicts. Life insurance tables are built from mortality data in the same way that public health agencies build their life tables."
    },
    {
      "heading": "Controversies, reproducibility and the future of the field",
      "text": "Frequentist and Bayesian foundations\n\nThe disagreement between frequentist and Bayesian approaches concerns what a probability refers to. Frequentists regard probability as the long-run behavior of a repeatable procedure, so an unknown parameter is treated as fixed and an interval is judged by how often the procedure captures it. Bayesians treat probability as a degree of belief that is revised with evidence via bayes-theorem. This requires a prior-distribution, which critics call subjective and defenders call an explicit statement of assumptions. The practical toolkit of twentieth-century statistics blended Ronald Fisher's significance testing with the decision-based framework of Jerzy Neyman and Egon Pearson, and many textbooks presented this mixture as a single method, a practice that philosophers and methodologists have criticized for its internal tensions. These debates connect to the broader philosophy-of-science and to questions about how inductive-reasoning should be justified.\n\nThe p-value and the significance threshold\n\nA 2016 statement from the American Statistical Association (ASA) addressed the widespread misreading of the p-value. It described the quantity as a measure of how incompatible observed data are with a specified model, and it stressed that this is different from the probability that the hypothesis is true or that chance alone produced the result. It is a statement about data in relation to a specified hypothetical explanation, and is not a statement about the explanation itself. The statement also cautioned that conclusions should not rest only on whether a result crosses a conventional cutoff. Its authors observed that treating \"bright-line\" rules (such as \"p < 0.05\") as a basis for scientific claims can lead to erroneous beliefs. Because any effect, no matter how tiny, can produce a small p-value if the sample size or measurement precision is high enough, a small p-value does not by itself establish that an effect is large or important. Related issues appear in statistical-hypothesis-testing, statistical-power and effect-size.\n\nIn 2019 a comment in Nature by Valentin Amrhein, Sander Greenland and Blake McShane went further, arguing that it is time to stop using the term 'statistically significant' entirely. Critics replied that abandoning the threshold could remove a useful obstacle. John Ioannidis wrote that removing it could promote bias, and a later analysis argued that proposals of this kind do not do the work that significance tests were designed to do. The dispute remains unresolved, and it shows how closely multiple-testing practice, false-discovery-rate control and reporting standards are tied to institutional incentives.\n\nThe replication crisis\n\nThe most cited empirical episode is the Open Science Collaboration's 2015 project. Its team conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available. Ninety-seven percent of original studies had significant results, but thirty-six percent of replications had significant results, and replication effect sizes were about half as large as the originals. Interpretations diverge. A 2016 comment in Science argued that the original analysis contained three statistical errors and that the data are consistent with the opposite conclusion, namely, that the reproducibility of psychological science is quite high. The commentary also argued that the benchmark ignored the multiple sources of error in their data. Later reanalyses found that replication rates ranged between 36% and 68%, depending on the definition of a replicated result. The episode made clear that there is no single standard for evaluating replication success.\n\nThe crisis extended beyond psychology into medicine, economics and other empirical fields, where selective publication, flexible analysis choices and small samples were repeatedly identified as contributors. These problems are central to reproducibility as a scientific norm.\n\nRemedies: preregistration, effect sizes and transparent reporting\n\nThe reforms that followed include preregistration, Registered Reports and open-data policies, which together aim to separate confirmatory from exploratory analysis and to make analysis choices visible. the methodological reforms — preregistration, Registered Reports, and open-data policy — that followed directly from it. The 2016 ASA statement similarly argued that proper inference requires full reporting and transparency. Reporting the size of an effect and the importance of a result alongside significance testing is another recommendation, together with confidence or credible intervals, which convey the uncertainty in the estimate itself. Proposals also differ on whether the threshold itself should be made stricter, as argued in some multiplicity-correction debates, or abandoned.\n\nData privacy, ethics and public responsibility\n\nAs data collection has expanded through administrative records, digital traces and big-data platforms, questions of consent, re-identification and secondary use have become central to the discipline. Statistical disclosure methods, including differential privacy, attempt to limit what can be learned about any individual while preserving aggregate accuracy; the 2020 United States census used such an approach, which generated its own debate about accuracy for small areas. Statisticians also influence public decisions through government statistics, medicine and econometrics, so the ASA noted that questions of statistical practice affect not only research, but research funding, journal practices, career advancement, scientific education, public policy, journalism, and law. One commentator frames the social role of statisticians as providing the tools and guidance needed to judge how much to trust data, a role seen as more important as misinformation spreads. The future of the field therefore depends not only on new methods, such as those in machine-learning, but also on transparent practice, honest communication of uncertainty, and accountability for how numbers are used."
    }
  ],
  "links": [
    "probability",
    "mathematics",
    "sampling-distribution",
    "statistical-hypothesis-testing",
    "confidence-interval",
    "bayesian-inference",
    "carl-friedrich-gauss",
    "pierre-simon-laplace",
    "ronald-fisher",
    "andrey-kolmogorov",
    "machine-learning",
    "epidemiology",
    "economics",
    "standard-error",
    "credible-interval",
    "p-value",
    "mortality-rate",
    "life-table",
    "normal-distribution",
    "ordinary-least-squares",
    "correlation",
    "maximum-likelihood-estimation",
    "randomization",
    "null-hypothesis",
    "alternative-hypothesis",
    "blaise-pascal",
    "sample-space",
    "sigma-algebra",
    "probability-space",
    "measure-theory",
    "random-variable",
    "law-of-total-probability",
    "bayes-theorem",
    "statistical-independence",
    "expected-value",
    "variance",
    "standard-deviation",
    "central-limit-theorem",
    "sample-mean",
    "bessels-correction",
    "histogram",
    "box-plot",
    "missing-data-imputation",
    "dimensionality-reduction",
    "stratified-sampling",
    "randomized-controlled-trial",
    "analysis-of-variance",
    "experimental-design",
    "confounding",
    "probability-distribution",
    "probability-mass-function",
    "binomial-distribution",
    "poisson-distribution",
    "probability-density-function",
    "chi-squared-distribution",
    "students-t-distribution",
    "cumulative-distribution-function",
    "law-of-large-numbers",
    "bootstrap-sampling",
    "estimator-bias",
    "consistent-estimator",
    "cramer-rao-bound",
    "fisher-information",
    "mean-squared-error",
    "likelihood-function",
    "sufficient-statistic",
    "bias-variance-tradeoff",
    "ridge-regression",
    "test-statistic",
    "significance-level",
    "type-i-and-type-ii-errors",
    "statistical-power",
    "bonferroni-correction",
    "false-discovery-rate",
    "effect-size",
    "logistic-regression",
    "prior-distribution",
    "posterior-distribution",
    "conjugate-prior",
    "posterior-predictive-distribution",
    "bayes-factor",
    "markov-chain-monte-carlo",
    "gibbs-sampling",
    "gauss-markov-theorem",
    "causal-inference",
    "generalized-linear-model",
    "odds-ratio",
    "regularization",
    "lasso-regression",
    "elastic-net",
    "cross-validation",
    "coefficient-of-determination",
    "counterfactual",
    "structural-causal-model",
    "directed-acyclic-graph",
    "propensity-score",
    "natural-experiment",
    "instrumental-variables",
    "difference-in-differences",
    "regression-discontinuity-design",
    "time-series",
    "stochastic-process",
    "autoregressive-model",
    "kalman-filter",
    "markov-chain",
    "markov-property",
    "stationary-distribution",
    "random-walk",
    "covariance-matrix",
    "multivariate-normal-distribution",
    "k-means-clustering",
    "gaussian-mixture-model",
    "linear-regression",
    "principal-component-analysis",
    "supervised-learning",
    "empirical-risk-minimization",
    "unsupervised-learning",
    "cluster-analysis",
    "deep-learning",
    "causation",
    "overfitting",
    "underfitting",
    "learning-curve",
    "generalization",
    "multiple-testing",
    "consumer-price-index",
    "national-accounts",
    "gross-domestic-product",
    "econometrics",
    "monte-carlo-method",
    "philosophy-of-science",
    "inductive-reasoning",
    "reproducibility",
    "big-data",
    "differential-privacy",
    "government",
    "medicine"
  ],
  "sources": [
    {
      "title": "Difference Between Stratified and Cluster Sampling - GeeksforGeeks",
      "url": "https://www.geeksforgeeks.org/data-science/difference-between-stratified-and-cluster-sampling/"
    },
    {
      "title": "Stratified vs Cluster Sampling: Full Guide",
      "url": "https://www.giroscience.com/stratified-vs-cluster-sampling-data-science-guide"
    },
    {
      "title": "Stratified vs Cluster Sampling — Differences, Uses & Examples",
      "url": "https://statsolvepro.com/stratified-vs-cluster-sampling/"
    },
    {
      "title": "Stratified vs. Cluster Sampling: A Field Researcher's Guide ...",
      "url": "https://projectbist.com/blog/stratified-vs-cluster-sampling-field-guide"
    },
    {
      "title": "A Discussion of Statistical Methods for Design and Analysis of Microarray Experiments for Plant Scientists - PMC",
      "url": "https://pmc.ncbi.nlm.nih.gov/articles/PMC1560901/"
    },
    {
      "title": "Fisher, Statistics, and Randomization Qingyuan Zhao University of Cambridge",
      "url": "http://www.statslab.cam.ac.uk/~qz280/talk/fisher-2022/slides.pdf"
    },
    {
      "title": "R.A. Fisher and his advocacy of randomization",
      "url": "https://www.researchgate.net/publication/5673795_RA_Fisher_and_his_advocacy_of_randomization"
    },
    {
      "title": "The Design of Experiments",
      "url": "https://en.wikipedia.org/wiki/The_Design_of_Experiments"
    },
    {
      "title": "Experimental Design - The Decision Lab",
      "url": "https://thedecisionlab.com/reference-guide/statistics/experimental-design"
    },
    {
      "title": "What Is Design of Experiments (DOE)?",
      "url": "https://asq.org/quality-resources/design-of-experiments"
    },
    {
      "title": "Survey sampling",
      "url": "https://en.wikipedia.org/wiki/Survey_sampling"
    },
    {
      "title": "Total survey error",
      "url": "https://en.wikipedia.org/wiki/Total_survey_error"
    },
    {
      "title": "Sample Representativeness and Nonresponse Bias",
      "url": "https://iseralaska.org/static/akpfs/Sample_Representativeness_Nonresponse_Bias_FAQs.pdf"
    },
    {
      "title": "Part I Survey Data",
      "url": "https://aspe.hhs.gov/sites/default/files/migrated_legacy_files/134521/01.pdf"
    },
    {
      "title": "Best Practices for Nonresponse Bias Reporting",
      "url": "https://statspolicy.gov/assets/fcsm/files/docs/FCSM%20NRBA%20Report%20062623.pdf"
    },
    {
      "title": "Bayesian curve fitting for lattice gauge theorists",
      "url": "https://arxiv.org/pdf/hep-lat/0112023"
    },
    {
      "title": "Bayesian Inference and Prior Distributions",
      "url": "https://www.nature.com/nature-index/topics/l4/bayesian-inference-and-prior-distributions"
    },
    {
      "title": "Prior Probability: Bayesian Statistics Formula, Guide & Examples (2026)",
      "url": "https://statisticsfundamentals.com/statistics-and-probability/prior-probability/"
    },
    {
      "title": "A Short History of Markov Chain Monte Carlo: Subjective Recollections from Incomplete Data",
      "url": "https://arxiv.org/pdf/0808.2902"
    },
    {
      "title": "Markov Chain Monte Carlo for Bayesian Inference - The Metropolis Algorithm",
      "url": "https://www.quantstart.com/articles/Markov-Chain-Monte-Carlo-for-Bayesian-Inference-The-Metropolis-Algorithm/"
    },
    {
      "title": "Markov Chain Monte Carlo Methods for Bayesian Data Analysis in Astronomy",
      "url": "https://arxiv.org/pdf/1706.01629"
    },
    {
      "title": "Understanding Confidence Intervals vs Credible Intervals in Data Analysis",
      "url": "https://codefinity.com/blog/Understanding-Confidence-Intervals-vs-Credible-Intervals-in-Data-Analysis"
    },
    {
      "title": "Credible Intervals: Bayesian Formula, Interpretation & Guide (2026)",
      "url": "https://statisticsfundamentals.com/confidence-intervals/credible-intervals/"
    },
    {
      "title": "Credible Intervals vs. Confidence Intervals",
      "url": "https://www.zubairkhalid.com/knowledge/bioinformatics/credible-intervals-vs-confidence-intervals-what-biologists-need-to-know-for-correct-interpretation"
    },
    {
      "title": "Linear Principal Component Analysis",
      "url": "https://link.springer.com/chapter/10.1007/978-1-4757-2765-4_8"
    },
    {
      "title": "Principal Components",
      "url": "https://www.worldscientific.com/doi/10.1142/9789819810888_0031"
    },
    {
      "title": "Statistical embedding: Beyond principal components",
      "url": "https://arxiv.org/pdf/2106.01858"
    },
    {
      "title": "Iris flower data set",
      "url": "https://en.wikipedia.org/wiki/Iris_flower_data_set"
    },
    {
      "title": "Fisher's Pioneering work on Discriminant Analysis and its Impact on AI",
      "url": "https://arxiv.org/pdf/2309.04774"
    },
    {
      "title": "About Fisher's Iris dataset - angela1c.com",
      "url": "https://www.angela1c.com/projects/iris_project/the-iris-dataset/"
    },
    {
      "title": "Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)",
      "url": "https://projecteuclid.org/journals/statistical-science/volume-16/issue-3/Statistical-Modeling--The-Two-Cultures-with-comments-and-a/10.1214/ss/1009213726.full"
    },
    {
      "title": "Data Leakage in Notebooks: Static Detection and Better Processes",
      "url": "https://arxiv.org/pdf/2209.03345"
    },
    {
      "title": "Data Leakage in Machine Learning - MachineLearningMastery.com",
      "url": "https://machinelearningmastery.com/data-leakage-machine-learning/"
    },
    {
      "title": "Purged cross-validation",
      "url": "https://en.wikipedia.org/wiki/Purged_cross-validation"
    },
    {
      "title": "Machine Learning Plumbery: Data Leakage",
      "url": "https://medium.com/spikelab/machine-learning-plumbery-data-leakage-6537bf287f2f"
    },
    {
      "title": "Bootstrap Methods",
      "url": "https://link.springer.com/rwe/10.1007/978-3-642-04898-2_150"
    },
    {
      "title": "Bootstrapping - an overview",
      "url": "https://www.sciencedirect.com/topics/mathematics/bootstrapping"
    },
    {
      "title": "Hitting the Jackpot: The Birth of the Monte Carlo Method",
      "url": "https://www.lanl.gov/media/publications/actinide-research-quarterly/1123-hitting-the-jackpot-the-birth-of-the-monte-carlo-method"
    },
    {
      "title": "A Survey of Monte Carlo Methods for Parameter Estimation",
      "url": "https://arxiv.org/pdf/2107.11820"
    },
    {
      "title": "Introduction to Monte Carlo Methods",
      "url": "https://arxiv.org/pdf/0905.1629"
    },
    {
      "title": "Missing data and bias in physics education research: A case for using multiple imputation",
      "url": "https://arxiv.org/pdf/1809.00035"
    },
    {
      "title": "MIDA: Multiple Imputation using Denoising Autoencoders",
      "url": "https://arxiv.org/pdf/1705.02737"
    },
    {
      "title": "Handling missing data in clinical research - ScienceDirect",
      "url": "https://www.sciencedirect.com/science/article/pii/S0895435622002189"
    },
    {
      "title": "An Online Bootstrap for Time Series",
      "url": "https://arxiv.org/pdf/2310.19683"
    },
    {
      "title": "Multiple Imputation",
      "url": "https://dept.stat.lsa.umich.edu/~jerrick/courses/stat701/notes/mi.html"
    },
    {
      "title": "Prevalence",
      "url": "https://en.wikipedia.org/wiki/Prevalence"
    },
    {
      "title": "Incidence and prevalence: Video, Causes, Symptoms",
      "url": "https://www.osmosis.org/learn/Incidence_and_prevalence"
    },
    {
      "title": "Prevalence and incidence",
      "url": "https://fiveable.me/introduction-epidemiology/unit-2/prevalence-incidence/study-guide/1QZn9CeqIP8pq2LZ"
    },
    {
      "title": "Incidence (epidemiology)",
      "url": "https://en.wikipedia.org/wiki/Incidence_(epidemiology)"
    },
    {
      "title": "The Consumer Price Index",
      "url": "https://courses.lumenlearning.com/wm-macroeconomics/chapter/changes-in-the-cost-of-living/"
    },
    {
      "title": "Consumer price index",
      "url": "https://en.wikipedia.org/wiki/Consumer_price_index"
    },
    {
      "title": "Gross Domestic Product: Understanding News from Noise",
      "url": "https://www.minneapolisfed.org/article/2002/gross-domestic-product-understanding-news-from-noise"
    },
    {
      "title": "► GDP (Gross Domestic Product) - Simply Explained",
      "url": "https://www.munich-business-school.de/en/l/business-studies-dictionary/financial-knowledge/gdp-gross-domestic-product"
    },
    {
      "title": "What is GDP (Gross Domestic Product) - Worldometer",
      "url": "https://www.worldometers.info/gdp/what-is-gdp/"
    },
    {
      "title": "Gross Domestic Product: An Economy’s All",
      "url": "https://www.imf.org/en/publications/fandd/issues/series/back-to-basics/gross-domestic-product-gdp"
    },
    {
      "title": "Epidemiology Basics: Incidence, Prevalence, R₀, CFR, Attack Rate",
      "url": "https://www.gideononline.com/blogs/epidemiology-metrics-explained-incidence-prevalence-r0-cfr-attack-rate/"
    },
    {
      "title": "5.8 Price Indexes: Consumer Price Index and Inflation Rate",
      "url": "https://ecampusontario.pressbooks.pub/principlesofmacroeconomicscdn/chapter/5-8-price-indexes-consumer-price-index-and-inflation-rate/"
    },
    {
      "title": "Gross domestic product (GDP)",
      "url": "https://www.britannica.com/money/gross-domestic-product"
    },
    {
      "title": "The ASA's Statement on p-Values: Context, Process, and ...",
      "url": "https://www.stat.berkeley.edu/~aldous/Real_World/ASA_statement.pdf"
    },
    {
      "title": "Statistical significance and its critics: practicing damaging science, or damaging scientific practice?",
      "url": "https://link.springer.com/article/10.1007/s11229-022-03692-0"
    },
    {
      "title": "Retiring statistical significance would give bias a free pass",
      "url": "https://www.nature.com/articles/d41586-019-00969-2"
    },
    {
      "title": "OSF",
      "url": "https://osf.io/ezcuj/overview"
    },
    {
      "title": "Comment on “Estimating the reproducibility of psychological science”",
      "url": "https://www.science.org/doi/10.1126/science.aad7243"
    },
    {
      "title": "Changes in methodological study characteristics in psychology between 2010-2021",
      "url": "https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10171647/"
    },
    {
      "title": "RESEARCH ARTICLE SUMMARY ◥ PSYCHOLOGY Estimating the reproducibility of",
      "url": "https://ppw.kuleuven.be/okp/_pdf/Nosek2015ETROP.pdf"
    },
    {
      "title": "Psychology Replication Crisis: What Happened — CASRAI",
      "url": "https://casrai.org/guides/psychology-replication-crisis"
    },
    {
      "title": "AMERICAN STATISTICAL ASSOCIATION RELEASES STATEMENT ON",
      "url": "https://www.amstat.org/asa/files/pdfs/p-valuestatement.pdf"
    },
    {
      "title": "Full article: The ASA Statement on p-Values: Context, Process, and Purpose",
      "url": "https://www.tandfonline.com/doi/full/10.1080/00031305.2016.1154108"
    },
    {
      "title": "P-Values in a Post-Truth World",
      "url": "https://arxiv.org/pdf/2007.03611"
    }
  ],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}