Chain-of-thought prompting is a prompt engineering technique that encourages a large language model to generate intermediate reasoning steps before giving a final answer. Instead of supplying only a question or examples of question–answer pairs, a prompt includes worked examples or instructions to work through a problem. The technique can improve performance on some arithmetic, symbolic, and commonsense reasoning tasks. However, a generated chain of thought is not necessarily a faithful account of the processes that produced the answer, and a detailed explanation does not guarantee correctness.
Concept and scope
A chain of thought is a sequence of generated statements that connects a task to a proposed solution. It may contain calculations, deductions, subquestions, assumptions, or checks. These statements usually appear in natural language, sometimes mixed with equations or code.
The distinction between chain-of-thought prompting and chain-of-thought output is important. Prompting is the intervention used to elicit intermediate steps; the output is the resulting text. A model may generate such text without an explicit request, and models trained specifically for extended reasoning may use intermediate computation without exposing its full textual form to a user.
The word thought is a technical metaphor. It does not establish that a model has humanlike consciousness, nor that its explanation gives direct access to its internal computations.
In its conventional form, chain-of-thought prompting changes the input supplied to a model during inference, rather than changing its parameters. This distinguishes it from fine-tuning, in which training updates a model’s weights using examples or feedback.
Historical development
Chain-of-thought prompting developed from earlier work on natural-language explanations, intermediate computation, and language-model scratchpads. Its modern formulation became widely known through the 2022 paper Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, by Jason Wei and colleagues. The study supplied worked solutions as demonstrations and reported improvements on several reasoning benchmarks, particularly for larger models.
Also in 2022, Takeshi Kojima and colleagues presented Large Language Models are Zero-Shot Reasoners. They showed that a short instruction such as “Let’s think step by step” could elicit intermediate reasoning and improve results on several evaluated tasks without providing worked examples.
Subsequent research explored sampling multiple reasoning paths, decomposing problems into smaller tasks, and searching over alternative intermediate steps. Research also examined whether the explanations were causally faithful, how much improvement came from additional computation, and when reasoning instructions failed to help.
These developments made intermediate reasoning a prominent subject in natural language processing, but did not establish one prompting recipe as universally effective.
How the technique works
Worked demonstrations
In few-shot chain-of-thought prompting, the input contains a small number of examples showing both intermediate steps and final answers. This is a form of in-context learning: demonstrations influence the response through the input context without requiring a parameter update.
For example:
Question: A box contains 12 pencils. Three are removed, then 5 are added. How many pencils are in the box?
Worked answer: Removing 3 from 12 leaves 9. Adding 5 gives 14. The answer is 14.
A new question follows one or more such demonstrations. Compared with an example containing only “14,” the worked answer provides a pattern for breaking the task into operations.
The examples can communicate more than the instruction to produce additional text. They may demonstrate how to interpret quantities, organize subproblems, state assumptions, or format an answer. Their relevance and structure can therefore affect performance.
Direct reasoning instructions
In zero-shot chain-of-thought prompting, the model receives an instruction to work through the problem but no worked demonstrations. A prompt may request stepwise calculation, decomposition into subquestions, or verification of intermediate results.
The term zero-shot refers to the absence of task demonstrations in the prompt. It does not mean that the model has never encountered similar problems in its training data.
Intermediate text as computational context
An autoregressive language model generates each new token conditioned on the preceding context. Once intermediate steps have been generated, they become part of the context used to generate later steps and the final answer. Here, tokens are the units produced through tokenization, which may represent words, word fragments, or other text elements.
Intermediate text can serve as a temporary workspace. It may preserve partial results, make dependencies explicit, and turn a difficult task into a sequence of simpler predictions. Generating more tokens also allows additional sequential computation.
This account does not imply that every displayed statement contributes meaningfully to the answer. A model can produce redundant, incorrect, or retrospective explanations, and the usefulness of intermediate text depends on the model and task.
Major variants and related methods
Self-consistency
Self-consistency decoding generates multiple candidate reasoning paths and aggregates their final answers, commonly by selecting the most frequent answer. The motivation is that different valid solution paths may converge on the same result, whereas some errors vary across samples.
Self-consistency is an answer-selection procedure layered on top of reasoning generation, rather than simply a different wording of the prompt. It requires additional computation and is most straightforward when answers can be normalized and compared.
Agreement is not proof of correctness: multiple samples can share the same misconception, and their errors need not be independent.
Least-to-most prompting
Least-to-most prompting first decomposes a problem into simpler subproblems and then solves them in an order that allows earlier answers to support later ones. Unlike a single uninterrupted explanation, it explicitly structures the task around dependencies between subproblems.
Its effectiveness depends partly on the quality of the decomposition. An incorrectly framed subproblem can misdirect the rest of the solution.
Tree-structured reasoning
Tree of Thoughts generalizes linear reasoning generation by exploring alternative intermediate states. A system generates candidate continuations, evaluates them, and uses a search procedure to choose which branches to pursue.
This introduces explicit branching and selection. It is related to chain-of-thought prompting, but is better understood as a broader inference framework than as a single prompt template.
Tool-assisted reasoning
Intermediate reasoning can be combined with calculators, code execution, search, or other tools. The model may identify a subproblem, invoke a tool, and then incorporate its result.
This differs from ordinary chain-of-thought prompting because some operations are performed by an external system rather than generated as text. A calculator can return a correct numerical result, for example, while the model may still choose the wrong calculation.
Likewise, retrieval-augmented generation supplies external information, whereas chain-of-thought prompting structures how a model approaches a task. The two can be combined, but they address different limitations.
Applications
Chain-of-thought prompting has been studied for:
- Mathematical word problems: translating a verbal description into a sequence of operations.
- Symbolic tasks: tracking states, manipulating strings, or following explicit rules.
- Commonsense reasoning: connecting several relevant facts or constraints.
- Question answering: breaking a compound question into intermediate questions.
- Programming and planning: outlining procedures, tracing execution, or identifying dependencies.
The technique is most directly relevant when intermediate operations can contribute to the answer. It is not inherently useful for every task. Simple classification, straightforward factual retrieval, and tightly constrained output formats may gain little from a lengthy explanation.
A generated solution may also be useful as an inspectable explanation, but inspection and verification are separate. In mathematics, for instance, a plausible explanation is not equivalent to a mathematical proof.
Limitations and interpretability
Incorrect reasoning and error propagation
A chain can contain arithmetic mistakes, unsupported premises, invalid deductions, or fabricated facts. Later steps may build on an early error, producing a coherent but incorrect result. This is related to, but not identical with, AI hallucination.
Longer output can introduce more opportunities for mistakes. Conversely, a correct final answer can be accompanied by invalid reasoning, whether because of compensating errors or an explanation that did not determine the answer.
Faithfulness of explanations
In explainable artificial intelligence, faithfulness concerns whether an explanation accurately reflects the factors that caused a system’s output.
Research has demonstrated settings in which language models respond to a biasing cue but omit that influence from their stated reasoning. A chain of thought can therefore be persuasive without being a complete causal explanation of the answer.
Two questions must consequently be separated:
- Does producing intermediate text improve task performance?
- Does that text faithfully explain why the model produced its answer?
Evidence for one does not establish the other. Unfaithfulness also does not imply that intermediate reasoning is always irrelevant; it means that its causal role requires investigation rather than assumption.
Model and task dependence
Benefits vary with model capabilities, training, problem difficulty, prompt design, and output constraints. Early studies found particularly substantial benefits in some larger models, but model size alone does not determine whether the technique will work.
Models already trained to perform extended reasoning may respond differently from models that require explicit demonstrations. Additional stepwise instructions may be helpful, redundant, or counterproductive depending on the system.
Computational cost
Intermediate steps consume output tokens, add latency, and occupy space within the context window. Multiple-path sampling and tree search increase these costs further. Performance gains must therefore be distinguished from efficiency gains: a method may produce better answers while using substantially more computation.
Evaluation
Evaluation of chain-of-thought prompting involves more than checking whether a response appears logically organized. Relevant dimensions include final-answer accuracy, validity of intermediate steps, explanation faithfulness, robustness, and computational cost.
A meaningful comparison distinguishes direct answering, worked demonstrations, and reasoning instructions while accounting for model version, sampling settings, and token budgets. Otherwise, an apparent prompting advantage may partly reflect a larger allowance for computation.
Benchmark interpretation also requires attention to data leakage, including possible overlap between evaluation problems and training material. Prompt sensitivity and variation across sampled outputs affect reproducibility as well.
For tasks with externally checkable results, independent verification can provide evidence stronger than stylistic plausibility. Code can be tested, calculations recomputed, and formal arguments checked using formal verification. These procedures assess properties of the output; they do not, by themselves, establish that a displayed chain of thought faithfully represents the model’s internal processing.