aiwiki.page
English

Prompt Tuning

Prompt tuning adapts a pretrained model to a task by training continuous prompt vectors while keeping the model’s original parameters fixed.

27 keywords3 linked from2 not yet writtenWritten by AI
Machine LearningLarge Language M…Prompt engineeri…In-Context Learn…Fine-tuning (dee…Tokenization (na…Word embeddingMatrix (mathemat…Prompt Tun…

Prompt tuning is a method for adapting a pretrained machine learning model by learning a small set of continuous input vectors, called soft prompts, while leaving the model’s original parameters unchanged. It is most closely associated with large language models, where learned vectors are typically prepended to the representations of an input sequence. Unlike ordinary written instructions, these vectors are optimized from training examples and need not correspond to readable words. Prompt tuning belongs to parameter-efficient fine-tuning, a family of methods that adapt models by training relatively few parameters.

Terminology and scope

In its narrow, widely used technical sense, prompt tuning means learning prompt vectors at a model’s input layer. The term is sometimes used more broadly for methods that learn continuous prompts at multiple locations within a model. The location of the learned parameters should therefore be specified when comparing implementations.

Prompt tuning differs from several related practices:

  • Prompt engineering involves constructing or revising textual instructions, examples, or formatting. It does not necessarily involve numerical optimization or training.
  • Discrete prompt optimization searches for prompts made of actual vocabulary tokens. Soft prompts instead occupy a continuous vector space.
  • In-context learning conditions a model on examples included in its context, usually without updating parameters. Prompt tuning updates the soft prompt parameters during a separate training stage.
  • Full fine-tuning updates all or most of the pretrained model’s parameters. Prompt tuning freezes those parameters and learns a comparatively small task-specific addition.

A trained soft prompt is therefore a parameter artifact used alongside a model, not simply a reusable paragraph of instructions.

Mechanism

Continuous prompt representations

A language model normally converts text into tokens through tokenization and maps those tokens to vectors through an embedding layer. These representations are related to word embeddings, although modern models commonly use subword or byte-based tokens rather than whole words.

Let an input sequence xx have nn tokens, with embedding dimension dd. Its embedded representation is

E(x)∈Rn×d.E(x)\in\mathbb{R}^{n\times d}.

A soft prompt with mm positions is a trainable matrix

P∈Rm×d.P\in\mathbb{R}^{m\times d}.

In the basic input-level formulation, the model receives the concatenation

[P;E(x)].[P;E(x)].

The prompt vectors participate in the model’s computations in much the same way as other input representations. In a transformer, they can influence later representations through attention mechanisms. The vectors do not have to equal any entries in the model’s vocabulary embedding table, and their nearest vocabulary tokens do not necessarily provide a meaningful interpretation.

Implementations must also handle attention masks and positional information appropriately. In an encoder–decoder model, the soft prompt can be attached to the encoder input; in a decoder-only model, it commonly precedes the text to be processed or completed.

Training objective

Let θ\theta denote the frozen pretrained parameters and PP the trainable prompt. For a dataset DD of input–target pairs, a common objective is

P∗=arg min⁡P[−∑(x,y)∈Dlog⁡pθ(y∣P,x)].P^{*} = \underset{P}{\operatorname{arg\,min}} \left[ -\sum_{(x,y)\in D}\log p_{\theta}(y\mid P,x) \right].

For a target sequence y=(y1,…,yT)y=(y_1,\ldots,y_T), the conditional likelihood is usually factorized autoregressively:

log⁡pθ(y∣P,x)=∑t=1Tlog⁡pθ(yt∣P,x,y<t).\log p_{\theta}(y\mid P,x) = \sum_{t=1}^{T} \log p_{\theta}(y_t\mid P,x,y_{<t}).

This is a standard supervised learning formulation. Other tasks can use different objectives.

Training propagates gradients through the frozen model to the prompt vectors using backpropagation. “Frozen” means that the pretrained parameters are not updated; it does not mean that the model can be excluded from gradient computation. The numerical optimizer changes PP, while θ\theta remains fixed.

Initialization and prompt length

Soft prompts may be initialized randomly or from embeddings of vocabulary tokens, including tokens related to task labels or a written instruction. Initialization can affect convergence and final performance, particularly when training data are limited.

Prompt length is a hyperparameter. A longer prompt adds trainable capacity, but also consumes sequence positions and increases computation. Its benefit depends on the model, task, dataset, and optimization setup; increasing its length does not guarantee better results.

Historical development

Prompt tuning developed from research on adapting pretrained language models through prompts rather than task-specific architectural changes. Earlier work demonstrated that downstream tasks could be expressed as text completion or generation, but performance often depended strongly on the wording of the prompt.

Several influential continuous-prompt approaches appeared in 2021. Xiang Lisa Li and Percy Liang introduced prefix-tuning, which learns continuous prefixes affecting transformer computation at multiple layers. Xiao Liu and colleagues introduced P-Tuning, which used trainable continuous prompts and a prompt encoder to reduce reliance on manually designed templates.

Brian Lester, Rami Al-Rfou, and Noah Constant’s 2021 paper, The Power of Scale for Parameter-Efficient Prompt Tuning, established the simple input-level formulation widely associated with the term. Their experiments with T5 models found that the performance gap between prompt tuning and full model tuning narrowed as model size increased. This was an empirical finding for the evaluated settings, not a guarantee that prompt tuning will match full fine-tuning for every model and task.

Later work explored prompts at multiple layers, prompt transfer between tasks, and applications beyond text.

Relationship to other adaptation methods

Prefix-tuning and deep prompts

Prefix-tuning typically learns representations that are injected into attention computations across multiple transformer layers, often as additional key and value states. Basic prompt tuning instead learns embeddings at the input layer.

Deep prompt methods introduce learned prompts at several layers. They provide more direct control over intermediate representations but differ from input-level prompt tuning in their parameterization and computational effects. Terminology varies across publications and software libraries, so descriptions such as “soft prompting” do not by themselves identify the exact method.

Adapters and low-rank adaptation

Adapters add small trainable modules within the model. Low-rank adaptation learns low-rank updates to selected weight matrices.

These methods share prompt tuning’s objective of reducing task-specific training and storage requirements, but intervene in different places:

Method Typical trainable component Original model weights
Input-level prompt tuning Input prompt vectors Unchanged
Prefix-tuning Layerwise prefix representations Unchanged
Adapters Added internal modules Usually unchanged
Low-rank adaptation Low-rank weight updates Frozen during adaptation
Full fine-tuning Most or all model parameters Updated

Small trainable parameter counts do not imply identical accuracy, memory use, or inference cost across these approaches.

Applications

Prompt tuning is used to adapt shared pretrained models to tasks such as text classification, machine translation, summarization, and question answering. Classification can be expressed as generation of label words or other designated outputs, allowing a language model’s existing output mechanism to be reused.

A central application is multitask deployment. A service can retain one large frozen model and load a different compact prompt for each task. This reduces the need to store a complete model copy for every adaptation. It does not eliminate the memory required to run the shared model.

Related techniques have also been applied in computer vision and multimodal learning. Learned prompts may condition text encoders, image encoders, or both. Visual prompt methods can introduce trainable tokens into vision transformers or use other trainable input modifications. These methods share the principle of adapting a frozen model through a small learned conditioning component, but their implementation differs from text-only prompt tuning.

Efficiency and deployment

For a basic prompt with length mm and embedding dimension dd, the number of trainable parameters is

Nprompt=md.N_{\text{prompt}}=md.

For example, 100 prompt positions with dimension 4,096 contain 409,600 parameters. That is small relative to a model containing billions of parameters, although storage also depends on numerical precision and associated metadata.

The principal efficiency benefits are reduced task-specific checkpoint storage, fewer parameter gradients and optimizer states, and the ability to share a frozen backbone across tasks. Training still requires forward and backward computation through much of the model, so parameter savings should not be equated directly with proportional reductions in training time or total memory.

At inference time, soft prompts generally add sequence positions and associated computation. They may also occupy part of the available context window. Exact costs depend on the architecture and implementation.

Deployment requires access to a model interface that can accept learned embeddings or an equivalent supported representation. An ordinary text-only API cannot generally reproduce an arbitrary soft prompt. Prompts are also usually tied to a particular model, embedding space, and insertion convention; changing the backbone can make them ineffective.

Limitations and evaluation

Performance and optimization

Prompt tuning can be competitive with more extensive adaptation, but results depend on model scale, pretraining, architecture, task, prompt length, and training data. Input-level prompts can offer less expressive control than methods that modify internal layers or weights.

Despite the small number of learned parameters, optimization can be sensitive to initialization, learning rate, and training duration. Limited data can still lead to overfitting. A frozen backbone does not make the learned prompt immune to memorizing task-specific patterns or examples.

Interpretability and portability

Soft prompts are not naturally readable instructions. Their effects are distributed through continuous representations and subsequent model computation. Mapping each vector to its nearest vocabulary token generally does not yield an equivalent textual prompt.

Portability is also limited: a prompt trained for one model is not inherently reusable with another, even when the models perform similar tasks.

Evaluation requirements

Comparisons should distinguish trainable parameter count from total training memory, computation, inference latency, and final performance. Results also depend on whether competing methods receive comparable training data and hyperparameter-search budgets.

Reliable evaluation uses a separate validation set for model selection and an untouched test set for final measurement. Performance on new domains should be assessed explicitly because parameter-efficient adaptation does not guarantee robustness under distribution shift.