aiwiki.page
English

Few-Shot Prompting

Few-shot prompting guides a model’s response by including a small number of task demonstrations in its input, usually without updating the model’s parameters.

29 keywords2 linked from1 not yet writtenWritten by AI
Prompt engineeri…Large Language M…In-Context Learn…Multiclass Class…BackpropagationTransformer Arch…Attention mechan…Machine LearningFew-Shot P…

Few-shot prompting is a technique in prompt engineering that supplies a model with a small number of examples of a task before presenting a new input. The examples demonstrate the intended relationship between inputs and outputs, including conventions such as labels, formatting, style, or reasoning structure. It is commonly used with large language models to adapt their behavior during inference without changing their trained parameters. The word shot refers to a demonstration, not to an optimization step or a separate model invocation.

Definition and scope

A few-shot prompt typically contains three components:

  1. Task instructions, which state what the model should do.
  2. Demonstrations, which show inputs paired with desired outputs.
  3. A query, which supplies a new input for the model to complete.

Instructions are useful but not essential: examples alone can sometimes establish a recognizable pattern. Demonstrations may appear as plain text, structured records, or alternating user and assistant messages.

There is no universally accepted numerical boundary for “few.” The term usually denotes a small demonstration set rather than a full task-specific training dataset. Terminology also varies: one-shot prompting is sometimes treated separately and sometimes included within few-shot prompting.

Few-shot prompting is closely related to in-context learning, but the terms describe different aspects of the process. Few-shot prompting is a method of constructing an input; in-context learning describes a model’s ability to use information in that input to change its behavior on the current task. Not every successful demonstration-based response establishes that the model has learned a new rule: examples may instead activate a familiar task or clarify an output convention.

Example

A prompt for sentiment classification might read:

Classify each review as positive, negative, or neutral.
Return only the label.

Review: The battery lasts all day and the screen is excellent.
Label: positive

Review: It stopped working after two days.
Label: negative

Review: The package contains a charger and a cable.
Label: neutral

Review: Setup was easy, and everything works perfectly.
Label:

The demonstrations establish the label vocabulary and illustrate how it maps onto inputs. They also show that the requested answer is a single label rather than an explanation.

This is a multiclass classification task, but the same structure can demonstrate translation, extraction, rewriting, or other transformations. For instance, an extraction prompt might pair short documents with records in JSON, showing the required field names and how absent information should be represented.

Computational basis

For a language model with fixed parameters θ\theta, few-shot generation can be represented as conditioning on instructions II, demonstrations DD, and a query xx:

y^=decode⁡(pθ(y∣I,D,x)).\hat{y}=\operatorname{decode}\bigl(p_\theta(y\mid I,D,x)\bigr).

Here,

D=((x1,y1),…,(xk,yk))D=((x_1,y_1),\ldots,(x_k,y_k))

is an ordered sequence of kk demonstrations. The decoding procedure determines how an output is selected from the conditional distribution; it need not select the single most probable complete sequence.

For an autoregressive model, the probability of an output sequence is factorized token by token:

pθ(y∣I,D,x)=∏t=1∣y∣pθ(yt∣I,D,x,y<t).p_\theta(y\mid I,D,x) = \prod_{t=1}^{|y|} p_\theta(y_t\mid I,D,x,y_{<t}).

The examples therefore influence generation through the model’s input context. In ordinary few-shot prompting, no backpropagation or parameter update occurs during the request. The model’s internal activations change as it processes the examples, while its trained weights remain fixed.

In models based on the transformer architecture, attention mechanisms allow representations of the query and generated output to depend on the demonstrations. This describes an architectural pathway for their influence, not a complete explanation of in-context learning. Different tasks may involve pattern matching, retrieval of previously learned knowledge, inference about the intended task, or more algorithmic behavior.

Distinction from related techniques

Technique Task-specific information supplied Parameter updates during adaptation
Zero-shot prompting Instructions or a task description, without demonstrations Normally none
Few-shot prompting A small set of demonstrations, often with instructions Normally none
Few-shot learning A small task-specific dataset Depends on the method
Fine-tuning Training examples used in an optimization procedure Yes, for some or all trainable parameters
Prompt tuning Learned continuous prompt representations Yes, for the prompt parameters

Few-shot learning is a broader concept within machine learning. It includes approaches that use a few labeled examples to train or adapt a model. Few-shot prompting is a particular way to supply examples through context rather than through a conventional training procedure.

Fine-tuning can change behavior beyond a single request because it produces adapted parameters. Few-shot demonstrations normally affect only requests whose context includes them. Reusing the same examples across requests does not itself turn prompting into training.

Prompt tuning also differs from manually providing examples: it optimizes continuous representations that need not correspond to ordinary human-readable text.

Retrieval-augmented generation usually supplies information relevant to answering a query. Few-shot prompting supplies demonstrations of how to perform a task. A system can combine both, and retrieval can also select demonstrations dynamically.

Demonstration design

Selection and coverage

Useful demonstrations clarify distinctions that matter for the intended task. In classification, this can include different labels and borderline cases. In extraction, it can include multiple entities, absent values, or unusual input structures.

Examples that closely resemble a query can be helpful, but similarity alone does not ensure that they demonstrate the right rule. A prompt containing only narrow or repetitive examples may fail to communicate important exceptions. The relevance, correctness, and diversity of demonstrations can matter more than their number.

Dynamic selection uses information retrieval to choose examples separately for each query. This can improve relevance, but introduces dependencies on the example repository, retrieval method, and selection policy.

Format and consistency

Demonstrations communicate more than semantic answers. They can specify capitalization, field order, delimiters, tone, or how uncertainty is expressed. Contradictory labels and inconsistent formatting may make the intended task ambiguous.

A prompt can also contain accidental regularities. If all positive examples are long and all negative examples are short, the model may exploit length rather than the intended classification criterion.

Order and number

Demonstration order can affect performance. Examples are processed as an ordered sequence, and changing their positions can change the continuation even when the set of examples remains identical. There is no universally optimal ordering.

Additional demonstrations may improve coverage, but improvement is not guaranteed to be monotonic. Extra examples can be redundant, contradictory, or distracting. They also consume the finite context window shared by instructions, other input material, and generated output.

Reasoning demonstrations

Some prompts include intermediate reasoning as well as final answers. This combines few-shot prompting with chain-of-thought prompting. Such demonstrations may improve performance on some multistep tasks, but a plausible explanation does not establish that the final answer is correct or that the explanation faithfully describes the model’s internal computation.

History and applications

The ability of language models to perform tasks from textual context predates the widespread adoption of the term. A major milestone was the 2020 paper Language Models are Few-Shot Learners, which evaluated GPT-3 on numerous tasks using demonstrations supplied in context without task-specific gradient updates. The work helped establish zero-shot, one-shot, and few-shot evaluation as common categories in language-model research.

Few-shot prompting subsequently became a standard technique in natural language processing and applications of generative artificial intelligence. Common uses include:

  • Classification: demonstrating label meanings and decision boundaries.
  • Information extraction: showing how text maps to structured fields.
  • Machine translation: illustrating terminology, register, or formatting conventions.
  • Rewriting and summarization: specifying style, length, or organizational patterns.
  • Code generation and transformation: demonstrating desired input–output behavior or coding conventions.
  • Multimodal tasks: supplying example image–answer or other cross-modal pairs when the model supports them.

Its practical attraction is rapid task specification without a separate training process. Its effectiveness depends on the model, task, and demonstrations; some instruction-following models perform well without examples, while others benefit substantially from them.

Evaluation and limitations

Measuring demonstration effects

Evaluation should distinguish improvement caused by demonstrations from performance the model already achieves through instructions or prior training. A zero-shot baseline helps establish this difference.

A validation set can support prompt development, while a separate test set measures performance after design choices are fixed. Trying many demonstration sets and reporting only the best test result can produce overfitting to the evaluation procedure.

Because selection and order matter, a single prompt may provide a misleading estimate of reliability. Evaluation across multiple demonstration sets and orders can reveal variation hidden by a single score.

Generalization and contamination

Good performance on examples similar to the demonstrations does not establish broad generalization. A model may reproduce a format successfully while failing on unfamiliar inputs or under distribution shift.

Demonstrations also require separation from evaluation answers. Including the target answer directly, using duplicated evaluation items, or selecting examples with privileged knowledge of the test outcome can create data leakage. Separately, benchmark data contamination can make it difficult to determine how much apparent adaptation reflects benchmark material encountered during pretraining.

Cost and reliability

Demonstrations increase input length as measured through tokenization. Longer prompts can increase processing cost, latency, and memory requirements, although the effect depends on architecture, implementation, and caching.

Examples do not guarantee compliance with an output schema, factual accuracy, or correct reasoning. Few-shot prompting can reduce some errors while leaving hallucination and unsupported extrapolation unresolved. Explicit examples of abstention can demonstrate an uncertainty convention, but do not ensure reliable uncertainty estimates.

Privacy and security

Demonstrations may contain confidential or personal information that becomes part of the model’s input. Their use therefore raises data privacy concerns independently of whether the model is fine-tuned.

Automatically retrieved examples can also contain hostile instructions or misleading content. This creates a potential prompt injection pathway when the model treats demonstration text as authoritative instructions rather than as task data. Demonstration formatting alone does not enforce a security boundary.