{
  "id": "beam-search",
  "title": "Beam Search",
  "lang": "en",
  "source_lang": "en",
  "summary": "Beam search is a heuristic search algorithm that retains a limited number of promising partial solutions at each step, trading search coverage for lower computational cost.",
  "category": "computer-science",
  "subcategory": "algorithms-theory",
  "type": "concept",
  "aliases": [
    "Beam-search decoding"
  ],
  "generated_at": "2026-11-03",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/beam-search/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/beam-search/",
      "json": "https://aiwiki.page/zh/beam-search.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Beam search is a heuristic search algorithm that explores a space of possible solutions while retaining only a limited number of promising candidates at each stage. This limit, called the beam width, controls how many partial solutions remain available for further expansion. By discarding other candidates, beam search reduces memory use and computation relative to exhaustive search, but generally sacrifices guarantees of finding a solution or the globally best solution. It is widely used in artificial intelligence, particularly for decoding sequences in machine translation, speech recognition, and other structured prediction tasks."
    },
    {
      "heading": "Basic procedure",
      "text": "In its common level-synchronous form, beam search resembles breadth-first search, except that it retains at most \\(k\\) candidates at each depth rather than the entire frontier. The positive integer \\(k\\) is the beam width.\n\nA candidate represents a partial solution, such as a path through a graph or a prefix of an output sentence. A scoring function ranks candidates according to their estimated usefulness. Depending on the problem, higher scores may indicate greater probability or reward, while lower scores may indicate lower cost.\n\nA typical iteration proceeds as follows:\n\n- Expand: Generate possible successors of each candidate in the current beam.\n- Score: Evaluate the resulting candidates.\n- Select: Retain only the best \\(k\\) eligible candidates across the combined successor set.\n- Terminate or repeat: Record completed solutions and continue expanding unfinished candidates until a stopping condition is satisfied.\n\nSelection is normally global across all successors, not a separate selection of \\(k\\) successors for each parent. Several surviving candidates may therefore descend from the same parent.\n\nLet \\(B_t\\) denote the beam at depth \\(t\\), let \\(\\operatorname{Succ}(h)\\) be the successors of candidate \\(h\\), and let \\(s(h)\\) be a score to maximize. The basic update is\n\n\\[C_{t+1}=\\bigcup_{h\\in B_t}\\operatorname{Succ}(h),\\]\n\n\\[B_{t+1}=\\operatorname{TopK}_{h\\in C_{t+1}}s(h).\\]\n\nThese expressions omit application-specific handling of completed solutions, duplicate states, constraints, and ties. Such choices can materially change the algorithm’s behavior.\n\nFor graph search, implementations may also detect repeated states or cycles. Without such handling, several candidates can represent redundant paths or revisit the same states."
    },
    {
      "heading": "Beam width and pruning",
      "text": "Beam width is a hyperparameter governing the trade-off between search effort and the number of alternatives retained.\n\nWith \\(k=1\\), the level-synchronous procedure reduces to greedy search: only the highest-scoring successor of the current candidate survives. Larger widths preserve alternatives that may initially appear less promising but lead to better complete solutions.\n\nFor example, suppose two first-step choices have probabilities \\(0.6\\) and \\(0.4\\). If their most probable continuations have conditional probabilities \\(0.5\\) and \\(0.95\\), respectively, the best complete paths through them have probabilities\n\n\\[0.6\\times0.5=0.30 \\quad\\text{and}\\quad 0.4\\times0.95=0.38.\\]\n\nGreedy search discards the second choice immediately, whereas a beam of width two can preserve it and discover the better complete path.\n\nThe defining limitation is irreversible pruning: ordinary beam search does not return to discarded candidates. A promising solution can therefore be lost because one of its prefixes ranked poorly.\n\nFor a finite search tree, a beam wide enough to retain every candidate at every level eliminates beam pruning and reproduces exhaustive level-wise exploration. At practical widths, however, beam search is generally neither complete nor optimal. Moreover, beams produced with different widths need not remain nested across successive steps, so increasing width does not provide a universal guarantee of a better returned score."
    },
    {
      "heading": "Sequence decoding",
      "text": "Beam search is especially prominent in natural language processing. An autoregressive language model assigns a probability to an output sequence \\(y=(y_1,\\ldots,y_T)\\), optionally conditioned on an input \\(x\\):\n\n\\[P(y\\mid x)=\\prod_{t=1}^{T}P(y_t\\mid y_{<t},x).\\]\n\nHere, \\(y_{<t}\\) denotes the preceding output symbols. In text systems, these symbols are typically tokens defined by a tokenization scheme.\n\nDecoding commonly seeks a high-probability complete sequence:\n\n\\[\\hat y=\\arg\\max_{y\\in\\mathcal Y}P(y\\mid x),\\]\n\nwhere \\(\\mathcal Y\\) is the set of allowed outputs. Exhaustively enumerating that set is usually impractical because the number of possible sequences grows exponentially with length.\n\nA beam decoder retains several output prefixes. Each prefix is extended with possible next tokens, and the highest-scoring extensions survive. Scores are usually accumulated as log probabilities:\n\n\\[s(y_{1:t})=\\sum_{i=1}^{t}\\log P(y_i\\mid y_{<i},x).\\]\n\nLogarithms preserve the ordering of positive probabilities and avoid the numerical underflow associated with multiplying many small values.\n\nAn end-of-sequence token marks completion. Its probability is part of the probability of the complete sequence. Implementations differ in whether finished candidates occupy beam slots or are stored separately while unfinished candidates continue expanding.\n\nBeam search is a decoding procedure, not a model architecture or training method. It can be used with recurrent neural networks, Transformers, and other models, including systems built around an encoder–decoder architecture.\n\nLength effects and adjusted scores\n\nRaw sequence probability creates a preference for shorter outputs in the sense that extending a fixed prefix cannot increase its probability. This mathematical property does not by itself establish the correct output length; useful decoding also depends on the model’s learned termination behavior.\n\nDecoders often modify the ranking objective. One possible length-normalized score is\n\n\\[s_{\\mathrm{norm}}(y)= \\frac{\\log P(y\\mid x)}{|y|^\\alpha},\\]\n\nwhere \\(|y|\\) is the output length and \\(\\alpha\\) controls the adjustment. Other formulas use an offset length penalty.\n\nFurther adjustments can include rewards for covering the input, penalties for repetition, or combinations of several model scores. These modifications change the objective being optimized: a better adjusted score does not necessarily imply a higher model probability.\n\nStopping conditions\n\nFinding one completed candidate is generally insufficient reason to stop, because an unfinished candidate may later produce a better completion.\n\nFor unnormalized cumulative log probability, an unfinished prefix’s score is an upper bound on the score of any extension: each additional log probability is nonpositive. If the best completed sequence scores at least as highly as every surviving unfinished prefix, no descendant of those prefixes can beat it.\n\nThis stopping guarantee applies only to candidates that survived pruning. It does not restore global optimality. Length normalization, future rewards, and other score adjustments may invalidate the simple bound and require different stopping rules."
    },
    {
      "heading": "Computational requirements",
      "text": "Let the maximum search depth be \\(T\\), the beam width be \\(k\\), and the maximum branching factor be \\(b\\). A straightforward implementation generates at most approximately \\(kb\\) successors per step, giving \\(O(Tkb)\\) candidate generation and scoring operations when each operation has constant cost.\n\nThe actual time complexity also depends on selecting the best candidates. Sorting all \\(kb\\) successors costs \\(O(kb\\log(kb))\\) per step. Partial-selection methods or a bounded priority queue can avoid a full sort.\n\nMemory requirements depend on what is stored:\n\n- The live beam contains at most \\(k\\) candidates.\n- Materializing all successors can require \\(O(kb)\\) temporary candidate records.\n- Storing complete prefixes directly can require \\(O(kT)\\) space.\n- Parent pointers can share prefix structure, although recovering outputs still requires retaining relevant history.\n\nFor neural decoding, candidate bookkeeping is only part of the cost. Each prefix also has model state. In Transformer inference, a key–value cache can substantially increase memory consumption as beam width and sequence length grow.\n\nCandidate computations can often be batched on a graphics processing unit, but batching does not eliminate the additional arithmetic and memory associated with multiple hypotheses."
    },
    {
      "heading": "Related algorithms and variants",
      "text": "Best-first and A* search\n\nGeneral best-first search maintains a frontier ordered by a heuristic evaluation function, potentially containing nodes at different depths. Level-synchronous beam search instead compares candidates at a common expansion step and imposes a hard width limit.\n\nA* search can provide optimality guarantees under appropriate assumptions about costs and heuristics. Ordinary beam pruning can discard the path to an optimal solution, even when its scoring function uses an admissible heuristic.\n\nDynamic programming and Viterbi decoding\n\nDynamic programming exploits reusable subproblems or states with equivalent futures. Viterbi decoding, for example, retains the best path to each relevant state in suitable finite-state models.\n\nBeam search instead limits the total number of retained hypotheses. It may discard a state that remains necessary for the globally best result.\n\nThe approaches can also be combined: a decoder may merge equivalent states using dynamic programming and then apply beam pruning. For general autoregressive neural models, distinct prefixes usually induce different future predictions and cannot simply be merged into one exact state.\n\nDiversity-oriented and stochastic variants\n\nOrdinary beam search often retains near-duplicate candidates. Diverse beam search modifies selection, for example by dividing candidates into groups and penalizing overlap, to encourage distinct alternatives. Diversity then becomes part of the search objective rather than a consequence of retaining more hypotheses.\n\nStochastic beam methods introduce randomness into candidate selection. Some are designed around particular sampling distributions, while others are heuristic combinations of search and sampling. A deterministic top-\\(k\\) beam should not be interpreted as a set of independent samples from the model.\n\nConstraint-aware and backtracking variants\n\nConstrained beam search tracks requirements such as mandatory phrases, valid syntactic structures, or legal action sequences. Candidates may be partitioned according to constraint progress so that an important but temporarily low-scoring path is not immediately eliminated.\n\nOther variants preserve discarded alternatives for later reconsideration, as in beam-stack search or limited-discrepancy methods. Their guarantees depend on the specific backtracking and pruning rules; they are not properties of ordinary beam search."
    },
    {
      "heading": "History and applications",
      "text": "Beam search became established in early artificial-intelligence and speech-understanding research. The Harpy speech-understanding system of the 1970s is a notable early example of beam-style pruning. In speech systems, “beam” can also refer to a score threshold relative to the current best hypothesis, rather than a fixed number of candidates. Threshold-based and fixed-width pruning are related but distinct.\n\nLater applications included statistical machine translation, parsing, and neural sequence generation. Beam search has also been used in automatic text summarization, image captioning, planning, and program generation.\n\nIts role is to approximate the search over outputs or action sequences. It does not determine the underlying model’s accuracy, and a wider beam cannot compensate for a model that assigns high scores to unsuitable solutions."
    },
    {
      "heading": "Limitations and evaluation",
      "text": "Beam search has several separate sources of failure:\n\n- Search error: Pruning removes the highest-scoring complete solution.\n- Scoring error: The scoring function favors an incorrect or unsuitable result.\n- Limited diversity: Many beam positions are occupied by almost identical candidates.\n- Constraint failure: Necessary requirements are omitted or inadequately represented during search.\n- Resource cost: Additional hypotheses increase computation and memory use.\n\nA larger beam may reduce some search errors while exposing deficiencies in the scoring model. For example, it may find an output with higher model probability but worse translation quality. This distinction is particularly important for large language model generation, where the most probable sequence is not necessarily the most informative, varied, or appropriate response.\n\nEvaluation therefore distinguishes the decoder’s objective score from external task quality. Beam width, score adjustments, and stopping rules are commonly selected using a validation set, while final comparisons use a separate test set. Reproducible comparisons also specify how completed candidates, ties, constraints, duplicate states, and maximum output lengths are handled."
    }
  ],
  "links": [
    "heuristic",
    "algorithm",
    "artificial-intelligence",
    "machine-translation",
    "speech-recognition",
    "breadth-first-search",
    "hyperparameter",
    "natural-language-processing",
    "language-model",
    "tokenization",
    "recurrent-neural-network",
    "transformer-architecture",
    "encoder-decoder-architecture",
    "time-complexity",
    "priority-queue",
    "kv-cache",
    "graphics-processing-unit",
    "heuristic-evaluation-function",
    "a-star-search",
    "dynamic-programming",
    "automatic-text-summarization",
    "large-language-model",
    "validation-set",
    "test-set"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}