Test Statistic · aiwiki.page

A test statistic is a quantity calculated from sample data for use in statistical hypothesis testing. It summarizes an aspect of the observations relevant to distinguishing a null hypothesis from an alternative hypothesis. Its observed value is compared with a reference distribution describing its behavior under the null hypothesis and associated assumptions. This comparison determines whether the result falls within a rejection region or yields a small p-value. (online.stat.psu.edu)

Mathematical definition

For observations $X_1,\ldots,X_n$ , a test statistic can be written as

T=T(X_1,\ldots,X_n).

It is a specified function of the sample, not a function of unknown parameter values. Hypothesized parameter values and quantities estimated from the observations may nevertheless appear in its formula. Before the observations are collected, $T$ is a random variable; after collection, its realized value is denoted $t_{\mathrm{obs}}$ . Common statistics measure a standardized difference, a discrepancy between observed and expected counts, or a comparison of fitted models. (stat.berkeley.edu)

The sampling distribution of $T$ under the null hypothesis is called its null distribution. A number alone does not establish how unusual a result is: its interpretation depends on this distribution and on which values count as evidence against the null. Large values, small values, or values in both tails may be relevant, depending on the alternative hypothesis. (stat.berkeley.edu)

Rejection regions and p-values

A rejection region $R$ specifies the statistic values that lead to rejection. A test with significance level $\alpha$ controls the probability of rejecting a true null hypothesis at no more than $\alpha$ , under its assumptions. For a simple null hypothesis, this is expressed as

P_{H_0}(T\in R)\leq\alpha.

Boundaries of the region are called critical values. This control concerns the first of the Type I and Type II errors; it does not guarantee a correct decision for any individual sample. (itl.nist.gov)

For an upper-tailed test with a specified null distribution,

p=P_{H_0}(T\geq t_{\mathrm{obs}}).

For a two-sided test based on a symmetric, zero-centered statistic, a common definition is

p=P_{H_0}(|T|\geq|t_{\mathrm{obs}}|).

Thus, the statistic and p-value are distinct: the former summarizes the data, while the latter evaluates its extremeness using a probability distribution. Two-sided extremeness depends on the testing procedure rather than on a universal rule. (itl.nist.gov)

Standardized statistics for means

Many familiar statistics have the structure

\frac{\text{estimate}-\text{hypothesized value}} {\text{standard error}}.

The numerator measures departure from the null value; the denominator expresses sampling variability. For a population mean with known standard deviation $\sigma$ , the statistic is

Z=\frac{\bar X-\mu_0}{\sigma/\sqrt n},

where $\bar X$ is the sample mean and $\mu_0$ is the hypothesized mean. With independent observations from a normal distribution, $Z$ has a standard normal null distribution. (itl.nist.gov)

When the population standard deviation is unknown, the one-sample t-statistic replaces $\sigma$ with the sample standard deviation $S$ :

T=\frac{\bar X-\mu_0}{S/\sqrt n}.

Here, $S/\sqrt n$ estimates the standard error of the mean. For independent, identically distributed normal observations, the null distribution is Student’s t-distribution with $n-1$ degrees of freedom. Estimating the variability therefore changes the appropriate reference distribution. (itl.nist.gov)

As a numerical illustration, let $n=25$ , $\bar x=12$ , $s=5$ , and $\mu_0=10$ . Then $t_{\mathrm{obs}}=2$ : the sample mean lies two estimated standard errors above the hypothesized value. Whether this leads to rejection depends on the chosen significance level, direction of the alternative, and t-distribution with 24 degrees of freedom. The calculation itself is not a decision rule. (itl.nist.gov)

Discrepancy and likelihood statistics

For categorical counts, Pearson’s chi-squared statistic is

X^2=\sum_{i=1}^{k}\frac{(O_i-E_i)^2}{E_i},

where $O_i$ and $E_i$ are observed and null-model expected counts. Larger values indicate greater discrepancy. In a goodness-of-fit test, its reference chi-squared distribution is generally an approximation; the degrees of freedom depend on the categories and parameters estimated from the data. Adequate expected counts are important for this approximation. (itl.nist.gov)

Another approach uses the likelihood function. A likelihood-ratio test compares the best fit permitted by the null hypothesis with the best fit in a larger model:

\Lambda= \frac{\sup_{\theta\in\Theta_0}L(\theta)} {\sup_{\theta\in\Theta}L(\theta)}.

Small values of $\Lambda$ , or large values of $-2\log\Lambda$ , indicate that imposing the null substantially reduces the achievable likelihood. This approach connects testing with maximum likelihood estimation. (online.stat.psu.edu)

Calibration and interpretation

A null distribution need not come from a standard analytic formula. A permutation test recalculates a chosen statistic over rearrangements justified by the null hypothesis. Enumeration or Monte Carlo sampling can provide its reference distribution. The validity of this calibration depends on the permissible rearrangements, not simply on the ability to shuffle observations. (stat.berkeley.edu)

A statistic’s interpretation remains conditional on the model and sampling assumptions. An extreme result may reflect an incorrect null hypothesis or incorrect assumptions. Moreover, a small p-value does not give the probability that the null hypothesis is true and does not measure effect size or practical importance. Failure to reject is likewise not proof of the null. Estimates and confidence intervals address questions about magnitude and uncertainty that a test decision alone does not resolve. (stat.berkeley.edu)