{
  "id": "deeplab",
  "title": "DeepLab",
  "lang": "en",
  "source_lang": "en",
  "summary": "DeepLab is a family of convolutional neural network models for semantic image segmentation, known for atrous convolution, multiscale context aggregation, and encoder–decoder refinement.",
  "category": "computer-science",
  "subcategory": "artificial-intelligence",
  "type": "product",
  "aliases": [
    "DeepLab segmentation models",
    "DeepLab model family"
  ],
  "generated_at": "2026-11-04",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/deeplab/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/deeplab/",
      "json": "https://aiwiki.page/zh/deeplab.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "DeepLab is a family of deep learning models for semantic image segmentation: assigning a semantic category, such as road, person, or vegetation, to each pixel in an image. Developed by researchers at Google and collaborating institutions, the family helped establish atrous convolution and multiscale context aggregation as central techniques for dense visual prediction. Its principal versions—DeepLabv1, DeepLabv2, DeepLabv3, and DeepLabv3+—address the competing requirements of recognizing objects through broad contextual information and locating their boundaries precisely.\n\nUnlike instance segmentation, standard DeepLab models do not distinguish separate objects belonging to the same category. Two adjacent people, for example, may receive the same semantic label without being identified as separate instances."
    },
    {
      "heading": "Task and architectural foundations",
      "text": "A DeepLab model takes an image as input and produces a spatial map of class scores. After conversion to class probabilities, each pixel is usually assigned the class with the highest probability. The resulting segmentation mask can have the same spatial dimensions as the input, although intermediate predictions are commonly computed at lower resolution and then resized.\n\nDeepLab builds on convolutional neural networks and the fully convolutional approach to dense prediction. A feature-extraction backbone transforms the image into increasingly abstract feature maps, and a segmentation head converts those features into spatial class predictions. Backbones adapted from image classification networks can be initialized with weights pretrained on ImageNet, providing a form of transfer learning.\n\nThe central architectural problem is that classification networks typically reduce spatial resolution through pooling and strided convolution. This makes computation more manageable and supports recognition over large image regions, but discards information needed for accurate boundaries. DeepLab addresses this problem through several complementary mechanisms:\n\n- preserving denser feature maps with atrous convolution;\n- aggregating context at several spatial scales;\n- refining localization through a conditional random field in early versions or a learned decoder in DeepLabv3+.\n\nThese mechanisms are not all present in every version."
    },
    {
      "heading": "Atrous convolution and output stride",
      "text": "Atrous convolution, also called dilated convolution, spaces the sampling positions of a convolutional filter apart. For a one-dimensional signal, it can be written as\n\n\\[y[i]=\\sum_{k=0}^{K-1}w[k]\\,x[i+rk],\\]\n\nwhere \\(x\\) is the input, \\(w\\) is a filter with \\(K\\) coefficients, and \\(r\\) is the dilation rate. Ordinary convolution is recovered when \\(r=1\\). The same principle applies in two spatial dimensions.\n\nFor a square filter of width \\(K\\), the effective span along either axis is\n\n\\[K_{\\mathrm{eff}}=K+(K-1)(r-1).\\]\n\nIncreasing the dilation rate therefore expands the area sampled by a filter without increasing its number of learned coefficients. This does not mean that maintaining high-resolution feature maps is computationally free: applying filters over larger feature maps still requires more computation and memory.\n\nDeepLab often removes selected downsampling operations from a classification backbone and increases dilation rates in subsequent layers. This preserves denser spatial features while retaining a broad field of view.\n\nA related parameter is output stride, the ratio between input-image resolution and the resolution of the relevant feature map. An output stride of 16 means that the feature map is approximately 16 times smaller along each spatial dimension. Output strides of 8 and 16 are common DeepLab configurations. Smaller output strides preserve more spatial detail but increase computational and memory costs."
    },
    {
      "heading": "Multiscale context aggregation",
      "text": "Semantic interpretation often depends on both local appearance and surrounding context. A small patch may be ambiguous in isolation but recognizable when its relationship to a road, building, or larger object is visible. Objects also occur at widely varying apparent sizes.\n\nDeepLabv2 introduced atrous spatial pyramid pooling (ASPP), which applies parallel filters with different dilation rates to the same feature map. These branches examine the image at different effective scales, allowing the model to combine local and broader contextual evidence.\n\nDeepLabv3 revised ASPP. A typical configuration includes:\n\n- a \\(1\\times1\\) convolutional branch;\n- several \\(3\\times3\\) atrous convolution branches with different rates;\n- an image-level pooling branch that supplies global context;\n- concatenation of the branch outputs followed by a projection layer.\n\nThe image-level branch helps address a limitation of very large dilation rates: on a finite feature map, many sampling positions may fall outside the image and encounter padding rather than useful features. Dilation rates are consequently tied to the feature-map resolution and output-stride configuration, rather than being universal constants."
    },
    {
      "heading": "Development of the model family",
      "text": "The principal versions form a sequence of architectural revisions rather than a single fixed network.\n\nVersion | Initial appearance | Main contribution\nDeepLabv1 | 2014 preprint; presented at ICLR 2015 | Combined a deep convolutional network using atrous convolution with fully connected conditional random field refinement\nDeepLabv2 | 2016 preprint | Introduced ASPP for multiscale context aggregation\nDeepLabv3 | 2017 | Systematically developed atrous-convolution modules and revised ASPP with image-level context and batch normalization\nDeepLabv3+ | 2018 | Added an explicit decoder for boundary refinement and explored efficient atrous separable convolution\n\nDeepLabv1: dense prediction and structured refinement\n\nThe original work, Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs, adapted a classification network for dense prediction using atrous convolution. Its coarse class predictions were upsampled and refined by a fully connected conditional random field (CRF).\n\nThe CRF combined the network’s predictions with pairwise relationships based on pixel positions and appearance. This encouraged spatial consistency while allowing label changes near strong appearance boundaries. In the original approach, it was a separate refinement stage rather than an integral part of the convolutional predictor.\n\nDeepLabv2: spatial pyramids\n\nDeepLabv2 made multiscale context modeling more explicit through ASPP and investigated stronger backbones, including residual networks. Its framework retained fully connected CRF refinement.\n\nParallel atrous branches provided an architectural means of handling scale variation. This is distinct from evaluating an image at several resized resolutions and combining the predictions, although both techniques can be used together.\n\nDeepLabv3: stronger contextual representations\n\nDeepLabv3 studied cascaded and parallel arrangements of atrous convolution and introduced a revised ASPP module. Its use of batch normalization and image-level features was important to the resulting design and training procedure.\n\nThe reported DeepLabv3 system achieved strong benchmark results without relying on the fully connected CRF postprocessing used in earlier versions. Its output nevertheless remained dependent on upsampling from relatively coarse feature maps.\n\nDeepLabv3+: decoder-based refinement\n\nDeepLabv3+ extended DeepLabv3 through an encoder–decoder architecture. The encoder extracts contextual features, while the decoder combines them with higher-resolution, low-level features from an earlier backbone stage.\n\nIn the standard design, the encoder output is upsampled and concatenated with low-level features after their channel count has been reduced by a \\(1\\times1\\) convolution. Further convolutions refine the combined representation before final upsampling. This gives the model access to spatial detail that is difficult to recover from high-level features alone.\n\nDeepLabv3+ also investigated a modified Xception backbone and atrous depthwise separable convolution. Separating spatial filtering from channel mixing reduces computation relative to a comparable standard convolution, although actual runtime depends on the implementation and hardware."
    },
    {
      "heading": "Training and evaluation",
      "text": "DeepLab is ordinarily trained using supervised learning with pixel-level annotations. A common loss function is pixelwise cross-entropy, with unlabeled or ignored pixels excluded from the objective. Implementations may modify the objective to address class imbalance or other dataset-specific requirements.\n\nCommon training practices include random cropping, scale variation, horizontal flipping, and other forms of data augmentation. Pretrained backbone weights are typically adapted through fine-tuning. Training crop size, output stride, dilation rates, normalization settings, and the optimization schedule can substantially affect results.\n\nEvaluation commonly uses mean intersection over union (mIoU). For each class, IoU compares the intersection of predicted and reference pixels with their union; mIoU averages this quantity across the evaluated classes. DeepLab papers reported results on benchmarks including PASCAL VOC 2012 and Cityscapes.\n\nA benchmark score describes more than the architecture alone. Comparisons depend on the training images and annotations used, backbone, input resolution, evaluation split, and inference procedure. Multiscale evaluation, horizontal-flip evaluation, or model ensembles can improve reported accuracy while increasing inference cost. Such results are not directly comparable to single-scale, single-model evaluation without qualification."
    },
    {
      "heading": "Applications and limitations",
      "text": "DeepLab has been used as a model or baseline in computer vision tasks involving scene understanding, road-scene parsing, remote-sensing imagery, and foreground–background separation. Its modular design allows different backbones and output-stride settings to support different accuracy and resource requirements. Implementations and adaptations are available in ecosystems such as TensorFlow and PyTorch.\n\nImportant limitations include:\n\n- Fine structures and boundaries. Thin objects, narrow gaps, and small regions can disappear during downsampling. Decoder refinement reduces but does not eliminate this problem.\n- Sparse sampling artifacts. Some arrangements of dilated filters produce uneven coverage, sometimes called gridding, which can weaken local continuity.\n- Fixed scale choices. ASPP combines several predetermined sampling scales, but these do not guarantee optimal treatment of every object size.\n- Resource demands. Dense feature maps and multiple context branches increase memory use and computation, particularly for high-resolution images.\n- Domain sensitivity. Performance can deteriorate under distribution shift, such as changes in imaging conditions, sensor characteristics, or scene composition.\n- Semantic rather than instance output. The standard formulation neither separates individual instances nor directly provides object identities across video frames.\n\nDeepLab is therefore best understood as a family of architectural techniques and reference models, not as one invariant system. A precise description of a DeepLab implementation includes its version, backbone, output stride, context module, decoder, training setup, and inference procedure."
    }
  ],
  "links": [
    "deep-learning",
    "semantic-segmentation",
    "pixel",
    "google",
    "instance-segmentation",
    "convolutional-neural-network",
    "fully-convolutional-network",
    "image-classification",
    "imagenet",
    "transfer-learning",
    "conditional-random-field",
    "residual-neural-network",
    "batch-normalization",
    "encoder-decoder-architecture",
    "supervised-learning",
    "loss-function",
    "data-augmentation",
    "fine-tuning",
    "intersection-over-union",
    "computer-vision",
    "tensorflow",
    "pytorch",
    "distribution-shift"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}