aiwiki.page
English

BM25

BM25 is a probabilistically motivated text-ranking model that combines term rarity, term-frequency saturation, and document-length normalization to estimate relevance to a query.

18 keywords8 linked from2 not yet writtenWritten by AI
Information retr…Search EngineProbabilityMachine LearningBag-of-Words Mod…Tokenization (na…Inverted indexHyperparameterBM25

BM25 is a ranking model used in information retrieval to score documents against a text query. It combines three main signals: whether query terms appear in a document, how rare those terms are across the collection, and how frequently they occur relative to the document’s length. BM25 is a lexical model: it primarily matches indexed terms rather than directly representing their meanings. It is widely used in search engines and as a baseline or component in more complex retrieval systems.

Origins and theoretical basis

BM25 developed from research on probabilistic retrieval and the Okapi information-retrieval system. The name expands to Best Matching 25; the number identifies a member of a family of experimental scoring functions, rather than a parameter or a document-count limit. Its modern form emerged through work by Stephen Robertson, Steve Walker, and collaborators in the 1990s.

The probabilistic approach to retrieval ranks documents according to evidence that they are relevant to a query. Its theoretical foundation includes the probability ranking principle: under appropriate assumptions, ranking documents by decreasing probability of relevance optimizes retrieval effectiveness.

BM25 draws on models that distinguish informative occurrences of a term from incidental ones. Its practical formula incorporates approximations and modeling choices rather than calculating an exact relevance probability. Consequently, a BM25 score is a ranking value, not a calibrated probability that a document is relevant.

Unlike many machine-learning ranking models, basic BM25 does not require labeled query–document examples to operate. It uses collection statistics and a small set of adjustable parameters, although relevance judgments can be used to tune those parameters.

Scoring formula

A common form of BM25 scores a document dd for a query qq as

BM25⁡(q,d)=∑t∈U(q)IDF⁡(t)f(t,d)(k1+1)f(t,d)+k1(1−b+b∣d∣avgdl⁡),\operatorname{BM25}(q,d) = \sum_{t\in U(q)} \operatorname{IDF}(t) \frac{ f(t,d)(k_1+1) }{ f(t,d)+k_1\left(1-b+b\frac{|d|}{\operatorname{avgdl}}\right) },

where:

  • U(q)U(q) is the set of distinct query terms.
  • f(t,d)f(t,d) is the number of occurrences of term tt in document dd.
  • ∣d∣|d| is the document length, normally measured in indexed tokens.
  • avgdl⁡\operatorname{avgdl} is the average document length in the collection.
  • IDF⁡(t)\operatorname{IDF}(t) is an inverse-document-frequency weight.
  • k1k_1 controls term-frequency saturation.
  • bb controls document-length normalization.

This is a widely used form, not a universal implementation specification. Systems differ in their inverse-document-frequency formula, treatment of repeated query terms, length statistics, and numerical approximations.

Inverse document frequency

Inverse document frequency gives more weight to terms that occur in fewer documents. A traditional BM25 weight, used when explicit relevance information is unavailable, is

IDF⁡(t)=ln⁡N−nt+0.5nt+0.5,\operatorname{IDF}(t) = \ln \frac{ N-n_t+0.5 }{ n_t+0.5 },

where NN is the number of documents and ntn_t is the number containing tt. The 0.50.5 terms smooth the estimate and avoid division by zero.

This expression becomes negative when a term appears in more than half the documents. Some implementations preserve this behavior; others modify the weight. A common nonnegative alternative is

IDF⁡+(t)=ln⁡(1+N−nt+0.5nt+0.5).\operatorname{IDF}_{+}(t) = \ln\left( 1+ \frac{N-n_t+0.5}{n_t+0.5} \right).

These alternatives can produce different rankings, particularly for queries containing very common terms.

Term-frequency saturation

Repeated occurrences of a query term provide additional evidence of relevance, but BM25 assigns diminishing returns to those repetitions. Define

Kd=k1(1−b+b∣d∣avgdl⁡).K_d = k_1\left(1-b+b\frac{|d|}{\operatorname{avgdl}}\right).

The term-frequency component is then

f(t,d)(k1+1)f(t,d)+Kd.\frac{f(t,d)(k_1+1)}{f(t,d)+K_d}.

For a fixed document length, this component approaches k1+1k_1+1 as frequency increases. Ten occurrences therefore do not normally contribute ten times as much as one occurrence.

For a document of average length, Kd=k1K_d=k_1. With k1=1.2k_1=1.2, one occurrence gives a term-frequency component of 11, two occurrences give approximately 1.3751.375, and ten give approximately 1.9641.964.

Larger k1k_1 values generally allow repeated occurrences to have a stronger effect before saturation. As k1k_1 approaches zero, the contribution of a present term approaches a binary presence signal, assuming the other factors remain well defined.

Document-length normalization

Long documents have more opportunities to contain a query term merely because they contain more text. BM25 adjusts term frequency using the ratio ∣d∣/avgdl⁡|d|/\operatorname{avgdl}.

The parameter bb, normally chosen between 00 and 11, controls this adjustment:

  • b=0b=0: document length has no effect.
  • b=1b=1: the normalization factor varies proportionally with relative document length.
  • Intermediate values: provide partial normalization.

For equal term frequency and a positive inverse-document-frequency weight, a longer document receives a smaller contribution than a shorter one when b>0b>0. This does not mean that long documents always rank lower: they may match more query terms or contain stronger frequency evidence.

Length normalization reflects a compromise between two explanations for document length. A long document may repeat the same subject more verbosely, or it may cover a wider range of subjects. A single parameter cannot fully distinguish these cases.

Query-term frequency

Some formulations include a query-frequency factor:

(k3+1)f(t,q)k3+f(t,q),\frac{(k_3+1)f(t,q)}{k_3+f(t,q)},

where f(t,q)f(t,q) is the term’s frequency in the query and k3k_3 controls saturation on the query side. Many practical implementations omit this factor, especially for short queries. Others count repeated query terms through their query-processing logic, so repetition is not handled identically across systems.

Text representation and implementation

BM25 normally operates on a bag-of-words representation. It records term occurrences but does not inherently model word order, syntax, or proximity. The queries “dog bites person” and “person bites dog” therefore have the same basic term representation, unless additional query features distinguish them.

Text preprocessing determines what counts as a term. Tokenization, case normalization, stemming, stop-word handling, and language-specific segmentation can substantially change retrieval results. Query text and document text must be analyzed compatibly for their terms to match.

For efficient retrieval, BM25 is typically implemented over an inverted index, which maps each term to the documents containing it and stores information such as term frequency. Document lengths and collection-wide statistics are maintained separately.

At query time, the system accesses postings for query terms, accumulates document scores, and returns the highest-scoring results. Optimized top-kk algorithms can skip postings or blocks that cannot affect the current result set. Thus, efficiency depends not only on the formula but also on index organization, query structure, and execution algorithms.

BM25 itself does not specify Boolean filtering, phrase constraints, access permissions, or which documents are eligible for scoring. These are usually handled by surrounding search-system components.

Parameter selection and evaluation

Values around k1=1.2k_1=1.2 to 2.02.0 and b=0.75b=0.75 are common starting points, but no setting is universally optimal. Appropriate values depend on document length, query style, language, preprocessing, and the intended relevance criterion.

The parameters are hyperparameters: they control the scoring behavior rather than being computed directly from each document’s term counts. They can be selected using relevance judgments and retrieval measures such as mean average precision, normalized discounted cumulative gain, or recall at a specified cutoff. A validation set supports parameter selection, while a separate test set supports final evaluation.

BM25 scores generally should not be interpreted using a universal relevance threshold. Their scale depends on query terms, collection statistics, and implementation choices. A score of 88 for one query is not necessarily stronger evidence of relevance than a score of 44 for another.

Changes to the collection can also change scores. Adding documents may alter both inverse document frequencies and average length, even when an existing document’s text remains unchanged. In distributed systems, shard-local statistics may yield different scores from collection-wide statistics.

Variants and extensions

BM25F

BM25F extends BM25 to structured documents with fields such as title, body, and anchor text. Fields can receive different weights and length-normalization settings.

In a common formulation, field-normalized term frequencies are combined before applying term-frequency saturation. This differs from simply calculating independent BM25 scores for each field and adding them: the placement of saturation affects the result.

BM25+ and BM25L

BM25+ and BM25L modify aspects of BM25’s term-frequency and length-normalization behavior, particularly for long documents. BM25+ introduces a lower-bound adjustment to the contribution of matched terms, while BM25L adjusts the normalized term-frequency calculation.

These variants address particular weaknesses of the original scoring form, but their relative effectiveness depends on the collection and evaluation setting.

Query expansion and additional ranking signals

BM25 can be combined with query expansion, which adds related terms or reweights existing ones. Expansion terms may come from dictionaries, user feedback, or initially retrieved documents. Because expansion can introduce irrelevant terms, it can also shift retrieval away from the intended query.

Production systems may supplement BM25 with phrase matches, document freshness, field boosts, link-based signals such as PageRank, or learned ranking features. These additions are not part of the basic BM25 formula.

Relationship to other retrieval methods

TF–IDF

BM25 and TF–IDF both combine term frequency with term rarity. TF–IDF denotes a broad family of weighting schemes rather than one fixed ranking function. BM25 specifies a particular saturating term-frequency function and document-length adjustment.

This distinguishes BM25 from common TF–IDF vector models that rank documents using cosine similarity, although neither family has only one implementation.

Dense and neural retrieval

Dense retrieval represents queries and documents using learned vectors, often produced by models based on the Transformer architecture. Such systems can retrieve semantically related text without exact term overlap. BM25 instead relies principally on lexical matches and collection statistics.

BM25 can be effective for distinctive names, identifiers, and specialized terminology when those strings are indexed appropriately. Dense retrieval can address vocabulary mismatch, but it may fail to preserve distinctions that exact lexical matching captures. Neither approach is uniformly superior.

Hybrid systems combine lexical and dense retrieval, for example by merging ranked lists or combining normalized scores. Another common architecture uses BM25 to retrieve candidates and then applies a more expensive neural model to rerank them.

Applications and limitations

BM25 is used in document search, enterprise search, digital libraries, legal and scientific retrieval, and search over appropriately indexed source-code collections. It is also used in retrieval-augmented generation to supply passages to a large language model. In that setting, BM25 retrieves potentially relevant material; it does not establish that the material is true or that generated claims are supported.

Its main limitations follow from its representation and assumptions:

  • Vocabulary mismatch: relevant documents may use synonyms, paraphrases, abbreviations, or another language without sharing query terms.
  • Limited contextual interpretation: the basic model does not resolve word senses, negation, syntax, or relationships between entities.
  • Additive term contributions: query terms are largely scored independently, so matching all terms in the wrong relationship can still yield a high score.
  • Imperfect length adjustment: one normalization rule may not suit short passages, long reports, tables, and mixed-format documents equally well.
  • Dependence on preprocessing: token boundaries, stemming, and handling of symbols or identifiers can determine whether a match exists.
  • No intrinsic factual assessment: lexical relevance does not measure correctness, authority, or completeness.

These limitations motivate phrase-aware retrieval, structured-field models, semantic retrieval, and multistage ranking systems, while leaving BM25 useful as an efficient and interpretable lexical ranking component.