In statistics, a z-score expresses how far a value lies above or below a reference mean, measured in units of the reference standard deviation. It is calculated by subtracting the mean from the value and dividing by the standard deviation. Z-scores are dimensionless: a score of means two standard deviations above the mean, while a score of means one and a half standard deviations below it. Standardization does not require a normal distribution, although interpreting z-scores through standard normal probabilities does.
Definition and interpretation
For a value from a population with mean and finite, positive standard deviation , its z-score is
The sign indicates direction relative to the mean, and the magnitude indicates distance in standard-deviation units:
- : the value equals the mean.
- : the value is above the mean.
- : the value is below the mean.
- : the value is one standard deviation from the mean.
For example, if a reference distribution of examination scores has mean and standard deviation , a score of has
The original value can be recovered from its z-score when the reference mean and standard deviation are known:
A z-score is therefore not an intrinsic property of an observation. The same value can have different z-scores when compared with different populations, time periods, or reference groups.
Mathematical properties
If a random variable has expected value and variance , where , then
satisfies
These properties hold regardless of the shape of the original probability distribution. A standardized variable may remain skewed, discrete, heavy-tailed, or multimodal. Subtracting a mean and dividing by a positive constant changes location and scale, not the underlying distributional shape.
Standardization also removes dependence on measurement units. If , with , then corresponding observations have identical z-scores when each variable is standardized using its own mean and standard deviation. If , their z-scores have opposite signs.
Because the transformation is increasing when the denominator is positive, it preserves ordering. Standardizing two variables separately also preserves their Pearson correlation, provided both have finite, nonzero variances.
Ordinary z-scores are undefined when the reference standard deviation is zero. Population standardization is also unavailable when the mean or variance does not exist.
Z-scores calculated from a sample
When population parameters are unknown, observations are often standardized using a sample mean and sample standard deviation:
where
For a nonconstant sample with , these standardized values satisfy
Their sample mean is consequently zero and their sample variance is one.
Some software instead calculates the standard deviation with divisor , rather than . With that convention, the average squared standardized value is one. Both conventions are used; the distinction matters when reproducing calculations, especially for small samples. The divisor is associated with Bessel’s correction for estimating population variance.
Z-scores calculated using the same sample’s mean and standard deviation are constrained by those shared estimates. They should not be treated as independent draws from a standard normal distribution.
Relationship to the normal distribution
If follows a normal distribution with mean and variance , then
The standard normal cumulative distribution function, conventionally written , converts a z-score into a cumulative probability:
For example, under an exact normal model:
| Z-score | Approximate proportion at or below the value |
|---|---|
| 2.28% | |
| 15.87% | |
| 50.00% | |
| 84.13% | |
| 97.72% |
Approximately 68.27%, 95.45%, and 99.73% of a normal distribution lie within one, two, and three standard deviations of its mean, respectively.
These percentages are not universal properties of z-scores. For a nonnormal distribution, a z-score alone generally does not determine a percentile or tail probability. Even need not correspond to the median.
Distribution-free bounds are possible under weaker assumptions. For example, Chebyshev’s inequality gives
when has mean zero and variance one. Such bounds are usually much less precise than probabilities derived from a specified distribution.
Z-scores and statistical inference
The term “z-score” is also used for a standardized test statistic. In statistical hypothesis testing, the relevant denominator is generally a standard error, not the standard deviation of individual observations.
For example, suppose are independent observations from a normal population with known standard deviation . To test a null hypothesis that the population mean is , the statistic is
Under the null hypothesis, this statistic has an exact standard normal distribution. For suitable nonnormal populations, the central limit theorem can justify an approximate normal distribution for large samples.
The distinction between an individual score and a standardized sample mean is important:
They standardize different quantities and answer different questions.
If the population standard deviation is unknown and replaced by the sample standard deviation, the corresponding mean-test statistic is
For independent normal observations, it follows Student’s t-distribution with degrees of freedom, rather than an exact standard normal distribution. Simply using a sample standard deviation to standardize an individual observation does not, however, make that observation’s standardized score a Student’s t statistic.
Standard normal critical values also enter confidence intervals and p-values. For a two-sided test with an exact standard normal null distribution, an observed statistic gives
This calculation depends on the validity of the null distribution, not merely on the statistic being called a z-score.
Applications
Comparing relative positions
Z-scores allow observations measured on different scales to be compared in terms of their positions within reference distributions. For example, scores of and on different examinations cannot be compared directly, but their standardized distances from their respective means can be compared.
Such a comparison concerns relative standing, not necessarily equal knowledge, difficulty, or practical importance. Equal z-scores also need not represent equal percentiles when the reference distributions have different shapes.
Data preprocessing
In machine learning, feature standardization commonly transforms each numerical variable using its training-set mean and standard deviation. This can prevent variables with large numerical scales from disproportionately influencing distance calculations or penalties.
Standardization is often relevant to principal component analysis, distance-based methods, and models using regularization. Its usefulness depends on the method and the meaning of the variables; making variances equal is a modeling choice rather than a universal requirement.
For evaluation on unseen data, the reference mean and standard deviation are ordinarily estimated from the training data and then applied unchanged to validation or test observations. During cross-validation, these quantities are estimated separately within each training fold. Estimating them from the full dataset allows information from evaluation observations to influence preprocessing.
Identifying unusual observations
Large absolute z-scores can flag observations far from a reference mean. Thresholds such as are common screening conventions, but their interpretation depends on distributional assumptions, sample size, and the number of observations examined.
An extreme z-score does not by itself establish that an observation is erroneous. It may reflect a valid rare event, a poorly chosen reference population, a heavy-tailed distribution, or a change in the underlying process.
Limitations and related transformations
The mean and standard deviation are sensitive to extreme observations. An unusually large value can shift the mean and inflate the standard deviation, potentially making itself or other unusual values appear less extreme after standardization.
Robust alternatives replace the mean with a median and the standard deviation with a robust scale estimate, such as the median absolute deviation. These are related standardization procedures but are not identical to ordinary z-scores.
Z-standardization should also be distinguished from:
- Min–max scaling, which uses observed endpoints to map values to a specified interval.
- Percentile ranking, which describes position in an empirical or theoretical distribution.
- Normal-score transformation, which maps ranks or cumulative probabilities to normal quantiles and can change distributional shape.
A z-score transformation alone does not remove skewness, establish normality, eliminate outliers, or make different reference populations substantively comparable.