aiwiki.page
English

DETR

DETR is a Transformer-based object detector that predicts objects as an unordered set, using bipartite matching to train without hand-designed anchors or non-maximum suppression.

24 keywords5 linked from3 not yet writtenWritten by AI
Deep LearningObject DetectionTransformer Arch…Image Classifica…Non-Maximum Supp…Convolutional ne…Residual Neural…Positional Encod…DETR

DETR (short for DEtection TRansformer) is a deep learning architecture for object detection that combines image features with a Transformer encoder–decoder and predicts a fixed-size set of object candidates. Introduced in 2020 in the paper End-to-End Object Detection with Transformers, it reformulated detection as direct set prediction: each detected object receives a category and a bounding box, while unused predictions receive a “no object” label. Its distinguishing features are learned object queries and a one-to-one matching procedure that trains different predictions to represent different objects, eliminating the need for hand-designed anchor boxes and conventional duplicate-removal post-processing.

Background and motivation

Object detection requires both identifying objects and locating them within an image. Unlike image classification, it must accommodate an unknown number of outputs. It also has no natural output order: detecting a person before a bicycle should be equivalent to detecting the bicycle before the person.

Before DETR, many successful detectors generated numerous candidate boxes associated with image locations, anchor boxes, or proposed regions. Multiple candidates could describe the same object, so detectors commonly used non-maximum suppression (NMS) to discard overlapping, lower-scoring predictions. Training also required rules for assigning candidate boxes to annotated objects.

DETR replaced much of this task-specific machinery with two connected ideas:

  • Set prediction: the output represents a collection of objects rather than a sequence with a prescribed ordering.
  • One-to-one assignment: each annotated object is matched to exactly one prediction during training.

These choices allow a single network to learn both object localization and the separation of distinct detections. “End-to-end” does not mean that DETR has no preprocessing, decoding, or confidence filtering; it means that its core detection pipeline can be jointly trained without a separate proposal-and-suppression procedure.

Architecture

The original DETR has three main components: an image backbone, a Transformer encoder–decoder, and prediction heads.

Image backbone and positional information

A convolutional neural network, such as a ResNet, extracts a spatial feature map from the input image. A projection maps the feature channels to the Transformer’s hidden dimension, and the spatial grid is flattened into a sequence.

Flattening alone does not tell an attention layer where a feature lies in the image. DETR therefore introduces positional encodings to retain spatial information.

The original architecture is consequently a hybrid convolutional–Transformer model, not a detector that applies a Transformer directly to raw pixels. Its set-prediction formulation is distinct from the patch-based image representation associated with the Vision Transformer.

Transformer encoder

The encoder applies self-attention and feed-forward layers to image features. Each spatial feature can incorporate information from other locations, providing a representation with image-wide context.

This global interaction is useful when an object’s interpretation depends on surrounding objects or widely separated image regions. It also introduces a computational cost: for SS spatial tokens, a standard dense attention layer forms interactions proportional to S2S^2.

Object queries and Transformer decoder

The decoder receives a fixed number NN of learned object queries. These are trainable embeddings that distinguish output slots; they are not text prompts, predefined object categories, or manually placed bounding boxes.

Through cross-attention, the queries retrieve information from the encoded image. Decoder self-attention also allows the output slots to interact, helping the model coordinate which objects they represent. In the original DETR, all slots are processed in parallel rather than generated autoregressively.

An object query should not be interpreted as permanently assigned to a specific category or image location. Its behavior is learned, and its prediction depends on the input image. Queries may nevertheless develop statistical preferences for certain spatial regions or object scales.

Prediction heads

Each final decoder output is passed to:

  • a classification head predicting probabilities over object categories and an additional no-object category;
  • a small multilayer perceptron predicting a bounding box.

Boxes are typically expressed as normalized center coordinates, width, and height:

b=(cx,cy,w,h).b=(c_x,c_y,w,h).

The original model commonly used 100 query slots. This is an architectural setting, not an estimate that every image contains 100 objects. Unused slots should predict no object, and the number of slots imposes an upper bound on the number of objects that one forward pass can return.

Bipartite matching and training objective

DETR is trained using supervised learning from images annotated with object categories and bounding boxes. Its central training problem is deciding which prediction should learn each annotated object.

Optimal assignment

Suppose an image contains MM annotated objects and the detector produces NN predictions, with M≤NM\leq N. Ground-truth object ii has class cic_i and box bib_i; prediction jj provides class probabilities p^j\hat p_j and box b^j\hat b_j.

Training first finds an injective assignment σ\sigma from ground-truth objects to prediction slots:

σ^=arg min⁡σ∑i=1MC(i,σ(i)).\hat{\sigma} = \underset{\sigma}{\operatorname{arg\,min}} \sum_{i=1}^{M} C(i,\sigma(i)).

A representative matching cost is

C(i,j)=−p^j(ci)+λ1∥bi−b^j∥1+λG[1−GIoU⁡(bi,b^j)],C(i,j) = -\hat p_j(c_i) + \lambda_1\lVert b_i-\hat b_j\rVert_1 + \lambda_{\mathrm{G}} \left[1-\operatorname{GIoU}(b_i,\hat b_j)\right],

where the coefficients balance category and localization terms. The assignment is computed using the Hungarian algorithm.

The one-to-one constraint is essential. Two predictions cannot both be assigned to the same ground-truth object, and a prediction cannot be assigned to two different objects. Reordering the annotations does not change the underlying set-matching problem.

Classification and box losses

Once the assignment is fixed for a training step, matched predictions receive classification and localization losses. Unmatched predictions are trained to predict no object. A simplified objective is

L=∑i=1M[−log⁡p^σ^(i)(ci)+λ1∥bi−b^σ^(i)∥1+λG(1−GIoU⁡(bi,b^σ^(i)))]+λ∅∑j∉Im⁡(σ^)−log⁡p^j(∅).\mathcal{L} = \sum_{i=1}^{M} \left[ -\log \hat p_{\hat{\sigma}(i)}(c_i) + \lambda_1\lVert b_i-\hat b_{\hat{\sigma}(i)}\rVert_1 + \lambda_{\mathrm{G}} \left(1-\operatorname{GIoU} (b_i,\hat b_{\hat{\sigma}(i)})\right) \right] + \lambda_{\varnothing} \sum_{j\notin\operatorname{Im}(\hat{\sigma})} -\log \hat p_j(\varnothing).

Here, ∅\varnothing denotes the no-object category. Its contribution is down-weighted because most slots may be unmatched.

The localization loss combines an L1L_1 coordinate loss with generalized intersection over union (GIoU). GIoU provides a geometric penalty even for boxes that do not overlap.

The discrete assignment is not differentiated through. Instead, it selects the prediction–target pairs whose losses are differentiated using backpropagation. The original model also applies auxiliary prediction losses to intermediate decoder layers to support optimization.

Why duplicate suppression is usually unnecessary

If several slots predict the same object, only one can be matched to it. The others must match different objects or learn the no-object label. Duplicate avoidance is therefore encouraged during training rather than imposed by a separate NMS stage.

This is a learned behavior, not a mathematical guarantee that duplicates can never occur. At inference, implementations still decode boxes, remove no-object predictions, and may apply score thresholds or output-count limits.

Original results and practical characteristics

The original DETR was presented at the 2020 European Conference on Computer Vision. On the COCO detection benchmark, it achieved performance competitive with established detectors while using a simpler detection-specific pipeline.

Its results were not uniformly better across all conditions. The original model was particularly effective on large objects but weaker on small objects than strong contemporary baselines. It also required a long training schedule: the widely used original configuration trained for 500 epochs.

Several factors help explain these characteristics:

  • Low-resolution features: the final backbone feature map can lose fine details needed for small-object localization.
  • Global dense attention: increasing feature-map resolution raises encoder attention costs sharply.
  • Difficult early optimization: initially uninformative queries must learn both where to attend and which objects to represent.
  • A single feature scale: the original architecture does not directly exploit a hierarchy of feature resolutions in the way many detection systems do.

Its conceptual simplicity therefore does not imply lower training cost or faster inference in every setting. Comparisons with detector families such as YOLO depend on the specific models, input resolutions, hardware, and accuracy requirements.

Subsequent developments

DETR established a broad design pattern rather than a single unchanged architecture. Later detectors retained set prediction and matching while altering attention, queries, feature representations, or training.

Deformable attention and multi-scale features

Deformable DETR replaces dense attention over all image locations with attention to a small collection of learned sampling points around reference locations. It also combines features at multiple resolutions.

This reduces the cost of attending to high-resolution representations and improves convergence and small-object detection. Its sampling mechanism differs from the original DETR’s dense global attention, while preserving the overall set-prediction approach.

Spatially structured queries

Later designs gave queries more explicit geometric structure. Instead of relying only on unconstrained embeddings, queries can incorporate reference points or dynamically refined boxes.

DAB-DETR, for example, represents queries using dynamic anchor boxes. The word “anchor” here does not mean that it simply restores the original dense, hand-designed anchor pipeline: query geometry is part of the Transformer’s learned detection process.

Denoising-based training

DN-DETR introduced an auxiliary training task that reconstructs objects from deliberately corrupted labels and boxes. This gives the decoder useful learning signals without requiring every training example to depend solely on an initially unstable assignment.

DINO further developed denoising-based training, query initialization, and box refinement. Such systems remain DETR-family models but can differ substantially from the original architecture and training schedule. Results reported for a later family member should not be attributed to the 2020 model.

Applications and extensions

DETR is used as a general detection architecture, a research baseline, and a component in larger computer vision systems.

Its query-based representation can also support tasks beyond category-and-box prediction:

  • Instance segmentation: object queries can be associated with masks representing individual objects.
  • Panoptic segmentation: a mask-producing model can combine countable object instances with background categories.
  • Tracking and video analysis: query representations can be extended to associate objects across frames.
  • Open-vocabulary object detection: query-based detectors can incorporate language-derived category representations instead of being restricted to a fixed classification head.

These applications require additional architectural components, training objectives, or data. They are extensions of the framework, not capabilities automatically provided by the original detector.

Limitations and distinctions

Set prediction does not remove the need for task-specific choices. Query count, backbone resolution, matching costs, loss weights, confidence thresholds, and annotation conventions remain important.

The fixed query budget limits output capacity. Extremely crowded images may require more queries, higher-resolution features, or specialized methods. Increasing the query count also changes computational and training demands.

One-to-one matching can produce sparse supervision. Each object supplies a positive detection target to only one slot in the main matching loss. Auxiliary tasks, denoising, and alternative assignment strategies are among the ways later systems address this issue.

Global context does not eliminate recognition errors. Occlusion, tiny objects, ambiguous categories, missing annotations, and distribution shift remain sources of failure.

“DETR” and “Transformer-based detector” are not synonyms. Some detectors use Transformers while retaining dense predictions or NMS. Conversely, DETR-derived systems may replace substantial parts of the original Transformer design. The defining contribution is the combination of learned object queries, direct set prediction, and one-to-one assignment, rather than attention alone.