Zipf’s law is an empirical regularity in which the frequency or size of an item is approximately inversely proportional to its rank when items are arranged from largest to smallest. Its best-known application is to words in a text: the second most frequent word occurs roughly half as often as the most frequent, the third roughly one-third as often, and so on. Named after the American linguist George Kingsley Zipf, the law also provides a reference model for city populations, the popularity of online resources, and other systems containing a few very common or large items and many rare or small ones.
In its strict form, the relationship is , where denotes rank and denotes frequency or size. More broadly, “Zipfian” often describes a power-law rank relationship , with an exponent near, but not necessarily equal to, one. This distinction matters: an approximately straight line on logarithmic axes does not establish an exact inverse-rank law, and a similar pattern can arise through several different mechanisms.
Zipf’s law connects empirical studies of language and social organization with probability, statistical inference, and information theory. Its mathematical formulation clarifies how rank relates to frequency distributions; its applications reveal both the usefulness and the limitations of describing complex systems through a single scaling exponent.
Historical development
The observation underlying Zipf’s law predates Zipf’s own work. In 1913, the French scholar Jean-Baptiste Estoup discussed regularities in word frequencies in his studies of shorthand and language. Zipf subsequently made rank–frequency relationships central to a broader research program. His books The Psycho-Biology of Language (1935) and Human Behavior and the Principle of Least Effort (1949) brought together evidence from language and other forms of human activity, helping establish the inverse-rank relationship as a widely recognized empirical law.
Zipf interpreted linguistic regularities through a “principle of least effort.” In his account, speakers benefit from reusing a small repertoire of expressions, whereas listeners benefit from distinctions that reduce ambiguity. The observed organization of vocabulary was presented as a balance between these competing pressures. This was a substantive explanatory proposal, not simply another way to state the rank equation. The empirical relationship can be examined independently of whether that particular explanation is accepted, and later models have shown that similar frequency patterns need not arise from communicative optimization.
Related discoveries developed in other fields. Vilfredo Pareto’s late nineteenth-century studies of income distributions established another influential form of power-law regularity. Felix Auerbach discussed the distribution of city sizes in 1913, and Alfred Lotka described an inverse-power relationship for scientific productivity in 1926. These findings concern different units and different statistical representations: an income distribution, an author-productivity distribution, and a rank–size curve are not interchangeable. Their mathematical relationships nevertheless encouraged the broader study of highly unequal distributions.
During the 1950s, Benoît Mandelbrot and Herbert A. Simon developed influential but different approaches to these patterns. Mandelbrot investigated statistical language models and information-theoretic formulations, including a shifted rank law. Simon’s 1955 paper “On a Class of Skew Distribution Functions” showed how cumulative growth with preferential selection can generate skewed distributions. Their work helped move the subject beyond cataloguing empirical straight lines toward specifying processes capable of producing them.
Later research increasingly emphasized large datasets, explicit statistical estimation, and comparisons between competing distributions. Mark E. J. Newman’s 2005 review of power laws and the 2009 methodological work of Aaron Clauset, Cosma Rohilla Shalizi, and Newman became prominent reference points for this approach. In contemporary usage, Zipf’s law is therefore both a specific inverse-rank hypothesis and a historical entry point into the wider analysis of heavy-tailed data. Distinguishing these meanings prevents approximate scaling, exact mathematical distributions, and proposed causal explanations from being treated as the same claim.
Mathematical formulation and normalization
Suppose a collection contains distinct items, ranked by decreasing probability of occurrence. A finite Zipf distribution assigns rank , for , the probability
Here controls the steepness of the distribution, and is the normalizing constant that makes the probabilities sum to one. This is a probability mass function over ranks. The strict inverse-rank law corresponds to ; empirical discussions often retain the name when an estimated exponent differs moderately from one.
Normalization is essential because proportionality alone does not define a probability distribution. For , the constant is the harmonic number . As grows,
where is the Euler–Mascheroni constant and denotes the natural logarithm. Thus the leading item’s probability is , rather than a universal fixed number. For a hypothetical vocabulary of 10,000 items following the strict finite law, is approximately , giving the first-ranked item a probability of about . This is a model calculation, not a general estimate for natural language.
An infinite-support version behaves differently. The series converges only for , in which case
with the Riemann zeta function. At , no positive constant can normalize the inverse-rank sequence over all positive integers. Consequently, an exact infinite Zipf distribution with exponent one is not a probability distribution. Statements about an unbounded inverse-rank vocabulary must specify a cutoff, a changing normalization, or an asymptotic interpretation.
The exponent also determines concentration. A larger places more probability on the leading ranks; a smaller positive makes probabilities more nearly equal across a fixed finite support. If the rank itself is treated as a random variable, its -th moment in the infinite-support model is
provided . In particular, its expected value exists only for , and its variance is finite only for . These are moments of the sampled rank, not of word length, city population, or another quantity associated with the ranked item. Confusing those variables can lead to incorrect interpretations of the model’s mathematical properties.
Rank curves, tail distributions, and exponent conventions
A rank–size relationship describes ordered observations rather than directly specifying how many observations fall within each size interval. Suppose the size at rank is approximately
Inverting the equation gives . Since rank approximately counts the number of observations at least as large as , the fraction exceeding scales as
This exceedance probability is the complementary cumulative distribution function. The relationship is exact only under appropriate idealizations; finite samples, discrete sizes, and ties require additional care.
If sizes can be approximated by a continuous variable with probability density function , integration gives an exceedance exponent , assuming . Hence
A strict Zipf rank exponent therefore corresponds to a density exponent , not to a density proportional to . This distinction is particularly important when comparing linguistic rank plots with studies that report frequency histograms, city-size densities, or income tails.
For a continuous Pareto distribution above a lower cutoff , the survival function is
Its inverse-rank behavior has . Some fields call the Pareto exponent; others report the density exponent , and some use “Zipf exponent” without making clear whether they mean or its reciprocal. Numerical comparisons are meaningful only after identifying the plotted quantity and the exponent convention.
Power laws are also associated with scale invariance. For the ideal rank curve, multiplying rank by multiplies size by , so . On logarithmic axes this becomes
The result is a straight line with slope . Finite cutoffs and normalization still matter, however, and a straight-looking segment does not demonstrate scale invariance over all sizes.
Another consequence follows by dividing ranks into multiplicative intervals. Under the strict law, the total probability between ranks and is approximately , provided the ranks lie well within the support. Successive equal-width intervals on a logarithmic rank axis therefore contain approximately equal total probability. This explains how a large collection of individually rare items can collectively account for a substantial share of observations, even though no single low-frequency item is prominent.
Word frequencies and linguistic structure
The canonical linguistic application begins by dividing a text collection into tokens and grouping those tokens into types. A token is an occurrence; a type is the category to which occurrences are assigned. In an English word-based analysis, repeated occurrences of “the” belong to one type. After types are sorted by their token counts, their rank–frequency curve often contains a broad region close to an inverse-rank relationship. Common grammatical words occupy the highest ranks, while specialized vocabulary, names, and unusual forms populate the lower-frequency tail.
The counting procedure materially affects the result. Converting all letters to lowercase merges forms otherwise counted separately. Lemmatization groups inflected forms under a dictionary form, while a surface-form analysis keeps them distinct. Decisions about punctuation, hyphens, contractions, and numbers change both vocabulary size and frequency counts. Languages with productive inflection or compounding introduce further differences, and languages whose writing systems do not routinely separate words with spaces require explicit segmentation. A Zipf exponent is therefore inseparable from the unit being counted.
Observed curves also change across ranks. The most frequent function words form a small, linguistically constrained group and need not lie on the same line as middle-ranked words. At the opposite end, many types occur once, producing a plateau in the empirical rank plot because their counts are tied. Such words are called hapax legomena. A one-occurrence plateau is partly a consequence of finite observation: a true probability much smaller than the reciprocal of the corpus size cannot be resolved accurately through raw counts.
Vocabulary growth provides a related but distinct regularity, commonly called Heaps’ law. If is the number of distinct types observed after tokens, an empirical approximation is , often with . Under independent sampling from a fixed probability sequence, the expected number of observed types is
For an infinite sequence with asymptotically proportional to , where , this leads to vocabulary growth of order . The strict case requires special treatment because the corresponding infinite probability sequence cannot be normalized.
Real texts depart from independent sampling. Topics make words cluster; authors and genres have different vocabularies; new names and technical terms enter over time. Combining documents can therefore produce a rank distribution different from that of any individual document. Zipf’s law describes aggregate frequency inequality, but does not by itself capture syntax, meaning, narrative order, or the dependence between successive words. Two texts can have almost identical word-frequency curves while differing profoundly in linguistic organization.
Cities, popularity, and other empirical settings
For city populations, Zipf’s law is usually expressed as a rank–size rule: the population of the city at rank is approximately the largest city’s population divided by . This is a statement about a system of cities within a specified territory and period, not about the internal population distribution of a single city. Studies in urbanization and economics have repeatedly examined whether the upper part of a national city-size distribution approaches this form.
The definition of a city is crucial. An administrative municipality, a continuously built-up settlement, and a metropolitan labor-market area can have very different populations. A large urban region divided among many municipalities may appear as several moderate-sized cities under one definition and a single leading city under another. Minimum population thresholds and national boundaries also affect the ranked sample. Consequently, differences between estimated exponents can reflect measurement choices as well as demographic processes.
A dominant capital or commercial center may stand well above an otherwise approximately linear rank plot, a pattern often discussed as urban primacy. At smaller settlement sizes, the distribution may bend away from the upper-tail relationship. A country can therefore exhibit an approximately Zipfian upper tail without its entire settlement system following one power law. Xavier Gabaix’s 1999 work provided an influential growth-based account linking city-size scaling to proportionate growth under suitable conditions, while subsequent research has continued to examine the roles of migration, economic structure, and urban definitions.
Popularity distributions on the Internet offer a different application. Requests for files, views of pages, or visits to sites can be ranked by frequency, often revealing a small head of highly requested objects and a long tail of infrequent ones. Here the measured item might be a page, domain, video, query string, or file, and the observation window can range from minutes to years. Automated traffic, recommendation systems, breaking events, and changing catalogs alter the resulting curve. Similar descriptive patterns occur in sales and cultural consumption, but the exponent is not fixed across products or periods.
Other unequal distributions require particular caution. Lotka-type scientific productivity and species-abundance distributions can be compared mathematically with rank laws, but their empirical units and mechanisms differ. Citation networks also have growth dynamics absent from a fixed vocabulary model, while ecological samples are shaped by detection probabilities and sampling effort. Calling these patterns “Zipfian” is useful only when the measured variable, rank range, and approximation are specified. The resemblance of two rank curves is evidence of a shared statistical shape, not evidence that cities, words, organisms, and online resources obey one common causal law.
Generative mechanisms
One major explanation uses preferential attachment, in which an item already observed frequently is more likely to receive another occurrence. In Simon’s model, each step introduces a new category with probability ; otherwise, an existing category is selected in proportion to its current count. This combines innovation with cumulative advantage. Older or initially fortunate categories tend to become large because their existing size raises their probability of further growth.
For the idealized Simon process, the distribution of category sizes has an asymptotic exponent
Using the rank–density relationship gives a rank exponent . When the innovation probability is small, the rank exponent is close to one. The model demonstrates how a simple stochastic process can generate a Zipf-like pattern without assuming that categories have been explicitly optimized. Its fixed innovation rate and age-dependent advantages are substantive assumptions, however, and observed vocabulary growth or item lifetimes may not match them.
A second family of explanations uses Gibrat’s law of proportionate growth. In a simple form,
where the growth factor does not systematically depend on current size. Taking logarithms converts growth into addition. Repeated independent multiplicative changes ordinarily suggest a lognormal distribution at a fixed age, rather than automatically producing a Pareto tail. Entry, exit, lower boundaries, resetting, and mixing across ages can change the long-run distribution. Suitable combinations yield power-law tails, but their exponent depends on the specific growth and boundary conditions.
Information-theoretic models instead examine trade-offs between diversity and cost. Suppose a distribution over ranks maximizes information entropy subject to a fixed average logarithmic rank cost. With constraints and , a Lagrange-multiplier calculation gives
This derives a power law from a specified optimization problem. It does not establish that logarithmic rank is the actual cost used by speakers or other agents, nor does the calculation alone require . The empirical interpretation of the constraint is the central explanatory issue.
Mixtures and hidden heterogeneity provide another route. Different topics, environments, or subpopulations may each have characteristic frequencies; combining them can yield a broad aggregate distribution even when individual components do not share the same scaling behavior. Random text-generation models can likewise produce approximate rank laws under some assumptions, without reproducing the structure of natural language. These examples establish a key limitation of explanation from distributional shape: Zipf-like scaling is compatible with multiple processes. Discriminating among them requires additional evidence, such as growth trajectories, innovation rates, temporal correlations, or changes induced by external conditions.
Variants, cutoffs, and related distributions
A widely used extension is the Zipf–Mandelbrot law,
where shifts the rank coordinate and the denominator normalizes the finite distribution. For , this reduces to the ordinary finite Zipf form. A positive shift softens the relative differences among the highest ranks. It can represent curvature near the head that a pure power law misses, although adding a parameter also makes fitting more flexible and increases the importance of model comparison.
An exponentially truncated model has the form
where sets a characteristic cutoff scale. At ranks much smaller than , the exponential factor changes slowly and the model can resemble a power law. At much larger ranks, probabilities fall more quickly. Such a form can describe a system with a limited repertoire or additional suppression of rare items, but the fitted cutoff does not by itself identify the underlying constraint.
Broken power laws use different exponents over different rank ranges. For word frequencies, this can accommodate a common-vocabulary region and a steeper rare-word region; for other applications it may reflect different growth regimes or a mixture of populations. Continuity at the breakpoint can be imposed, reducing the number of independent parameters. Yet an apparent break may also arise from incomplete sampling, a change in the definition of an item, or the combination of datasets. A fitted breakpoint is therefore a descriptive feature unless supported by a separate mechanism.
The lognormal distribution is an especially important alternative. It can generate a highly unequal set of sizes and a rank plot that appears nearly linear over a limited range. Stretched exponential distributions can also resemble power laws over restricted intervals. These models differ substantially outside the observed range, so extrapolations to extremely rare events or exceptionally large objects may depend more on model choice than on the visible quality of a plot.
Terminology adds another complication. A “Zipf distribution” often means a discrete probability distribution on positive integers, whereas a “Zipf plot” may simply be an empirical plot of ordered sizes. A “zeta distribution” usually denotes the infinite-support form normalized by . The Yule–Simon distribution, associated with preferential growth, has a power-law tail but is not identical at every integer to a zeta distribution. Distinguishing exact distributions from asymptotic equivalence matters when calculating probabilities for small values, comparing likelihoods, or interpreting deviations at the most frequent ranks.
Estimation and statistical assessment
Assessment begins with the data-generating and measurement process. The analyst must define an item, an observation, the sampling frame, and any threshold below which observations are absent. Word frequencies require a tokenization rule; city sizes require a geographic definition; online requests require decisions about automated traffic and repeated events. The empirical ranks are themselves estimated from these observations. When true frequencies are close, random variation can change their order, especially among rare items.
Fitting a straight line to a log–log plot using ordinary least squares is simple but often unreliable. Count variability is not constant across ranks, zeros cannot be logged, and ordering the observations creates dependencies. Discarding low counts or logarithmically binning observations changes the effective weighting. A high coefficient of determination shows that transformed points lie near a fitted line; it does not establish a particular probability model, the absence of systematic curvature, or the correctness of an exponent estimate.
Maximum likelihood estimation is usually better suited to a specified distribution. If independent observations are generated from a fixed rank-probability model with known support , and is the observed count of rank , then
The estimate maximizes this expression. If ranks are assigned after inspecting the same counts, however, the labels are not externally fixed categories. If unseen types are possible, the support is also uncertain. These features require more care than simply inserting observed ranks into the likelihood.
For a continuous Pareto tail with fixed lower cutoff, a familiar estimator is
This estimates the survival exponent , not the rank exponent directly, and is not an exact formula for a discrete Zipf distribution. The cutoff determines which observations are treated as belonging to the tail; estimating it changes both the sample size and the uncertainty of the exponent. A distribution fitted only above a high threshold may explain a small fraction of all observations.
Goodness-of-fit can be evaluated by comparing empirical and fitted cumulative distributions and by simulating new datasets from the fitted model. When parameters or cutoffs have been estimated, they should generally be re-estimated within the simulated samples used to calibrate a test. Bootstrap sampling can help quantify uncertainty, while likelihood comparisons or predictive evaluation can distinguish power laws from alternatives. Dependence, such as words clustered within documents, requires corresponding adjustments to the sampling model. Failure to reject a Zipf model indicates compatibility with the test and data, not proof of an exact law.
Computational implications and unresolved questions
Zipfian demand has direct consequences for caching and storage. If requests are independent and an ideal cache holds the most popular objects from a fixed catalog of size , its hit probability is
Under the strict inverse-rank model, this is approximately when both quantities are sufficiently large. A relatively small collection of popular objects can therefore serve many requests, but a long tail continues to generate misses. Real caches face additional complications: finite object sizes, changing popularity, replacement policies, and correlations between requests.
In machine learning, unequal token frequencies mean unequal amounts of evidence. A common word or symbol may appear millions of times while a rare one appears only a handful of times. Parameters associated with rare items consequently have fewer direct observations, even when the total training set is very large. Subword representations partly change the unit of analysis by dividing uncommon words into reusable pieces. The resulting frequency curve is a property of both the source material and the segmentation procedure, rather than a tokenization-independent law.
The distribution also matters for compression. An ideal code length for an item of probability is bits. For a Zipf model,
Thus ideal code length grows logarithmically with rank. This connects unequal frequencies with the logic of assigning short codes to common items and longer codes to rare ones. Actual text compression must also exploit sequence dependence and context: a rank distribution alone cannot account for the predictability supplied by grammar, repeated phrases, or document structure.
A continuing theoretical question is why exponents near one occur so often, despite the many mechanisms capable of producing broader power-law families. Preferential growth, constrained optimization, and heterogeneous mixtures can each yield scaling, yet their predictions for the exponent and its stability differ. Important discriminating quantities include the rate at which new categories appear, the persistence of an item’s rank, growth rates conditional on size, and departures from scaling after changes in system organization.
The status of the law also depends on its intended precision. A finite dataset can support an approximately inverse-rank relationship across a stated range, but cannot establish exact scaling over unobserved ranks or future periods. Conversely, systematic departures are often scientifically informative rather than mere imperfections: curvature may reveal separate populations, changing innovation rates, finite resources, or a poorly chosen observational unit. Current research therefore treats the exponent, the scaling range, temporal behavior, and measurement process as distinct objects of investigation, rather than reducing the evidence to whether a single plotted line appears straight.