{
  "id": "multilabel-classification",
  "title": "Multilabel Classification",
  "lang": "en",
  "source_lang": "en",
  "summary": "Multilabel classification is a machine-learning task in which each example may be assigned multiple labels from a predefined set, rather than exactly one mutually exclusive class.",
  "category": "computer-science",
  "subcategory": "artificial-intelligence",
  "type": "concept",
  "aliases": [
    "Multi-label classification",
    "Multi-label learning",
    "Multiple-label classification"
  ],
  "generated_at": "2026-11-04",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/multilabel-classification/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/multilabel-classification/",
      "json": "https://aiwiki.page/zh/multilabel-classification.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Multilabel classification is a machine-learning task in which an input can belong to several categories simultaneously. A news article, for example, may be labeled “politics,” “economics,” and “international affairs,” while a photograph may be labeled “person,” “bicycle,” and “street.” In its standard supervised-learning formulation, a model learns from examples paired with sets of applicable labels and predicts a label set for each new input.\n\nThe defining feature is that labels are not mutually exclusive. This distinguishes the task from multiclass classification, where each example receives exactly one class from several alternatives. Multilabel methods must address both the relevance of individual labels and, where useful, relationships among them."
    },
    {
      "heading": "Formal definition",
      "text": "Let \\(\\mathcal{X}\\) be an input space and let \\(\\mathcal{L}=\\{\\ell_1,\\ldots,\\ell_L\\}\\) be a vocabulary of \\(L\\) labels. A multilabel classifier learns a mapping\n\n\\[f:\\mathcal{X}\\rightarrow 2^{\\mathcal{L}},\\]\n\nwhere \\(2^{\\mathcal{L}}\\) denotes the set of all subsets of \\(\\mathcal{L}\\). Depending on the application, an empty label set may be allowed.\n\nEach target set can be represented by a binary indicator vector:\n\n\\[\\mathbf{y}=(y_1,\\ldots,y_L)\\in\\{0,1\\}^{L}, \\qquad y_j= \\begin{cases} 1,&\\ell_j\\text{ applies},\\\\ 0,&\\ell_j\\text{ does not apply}. \\end{cases}\\]\n\nThis is often called a multi-hot representation, in contrast to one-hot encoding, which has exactly one active entry.\n\nMany models first produce a score \\(s_j(x)\\) or estimated marginal probability\n\n\\[p_j(x)=P(y_j=1\\mid x)\\]\n\nfor every label. A decision rule then converts these outputs into a label set. One common rule is\n\n\\[\\hat{Y}(x)=\\{\\ell_j:p_j(x)\\geq t_j\\},\\]\n\nwhere \\(t_j\\) is a global or label-specific threshold.\n\nTwo descriptive properties of a dataset are:\n\n- Label cardinality: the mean number of positive labels per example.\n- Label density: label cardinality divided by the number of available labels.\n\nThese properties describe different aspects of difficulty. A dataset with thousands of possible labels may have low density even when each example has several relevant labels."
    },
    {
      "heading": "Relationship to other learning tasks",
      "text": "In binary classification, the target is one of two alternatives. A multilabel task can be decomposed into multiple binary tasks, but its overall output is a set rather than one binary decision.\n\nMulticlass and multilabel classification are sometimes confused because both can involve many categories. The distinction is whether categories exclude one another. Classifying a document into exactly one primary department is multiclass classification; assigning every applicable subject tag is multilabel classification.\n\nMultilabel classification also differs from several related settings:\n\n- Multi-output classification is broader: each output may have its own set of possible values, rather than being a binary label indicator.\n- Label ranking orders labels by relevance without necessarily deciding which labels are present.\n- Hierarchical classification arranges categories in a taxonomy. It may be single-label or multilabel.\n- Structured prediction models outputs with internal dependencies. Multilabel classification can be treated as a structured-prediction problem when label relationships are represented explicitly.\n\nMultilabel image classification predicts which categories occur in an image but does not locate individual objects. Object detection additionally predicts object locations, while segmentation assigns categories or identities to image regions."
    },
    {
      "heading": "Main modeling approaches",
      "text": "Binary relevance\n\nBinary relevance trains one binary classifier for each label:\n\n\\[f_j:\\mathcal{X}\\rightarrow\\{0,1\\}, \\qquad j=1,\\ldots,L.\\]\n\nThe final prediction combines their outputs. The individual classifiers may use methods such as support vector machines, decision trees, or logistic regression.\n\nBinary relevance is simple, modular, and naturally parallelizable. Its principal limitation is that one label’s prediction does not explicitly depend on the others.\n\nThis limitation is sometimes described too strongly as an assumption that labels must be independent. Independent per-label decisions can be optimal for an additive objective such as Hamming loss even when labels are statistically dependent. What binary relevance lacks is an explicit mechanism for using those dependencies to improve joint decisions or share information across labels.\n\nClassifier chains\n\nA classifier chain orders the labels and predicts each one using the input and preceding labels:\n\n\\[P(\\mathbf{y}\\mid x) = \\prod_{j=1}^{L} P(y_j\\mid x,y_1,\\ldots,y_{j-1}).\\]\n\nThis factorization can represent label dependencies. During ordinary training, earlier ground-truth labels are used as features; during sequential prediction, earlier predictions are used instead.\n\nClassifier chains can exploit patterns such as the frequent co-occurrence of “beach” and “sea,” but their behavior depends on label order. Incorrect early predictions can also affect later decisions. Ensembles of chains use multiple orderings to reduce sensitivity to a single sequence. Exact search for the highest-probability label vector may be expensive, so practical systems often use greedy or approximate inference.\n\nLabel powerset\n\nThe label powerset transformation treats each distinct label combination observed in training as one multiclass category. For example, \\(\\{\\text{sport},\\text{business}\\}\\) and \\(\\{\\text{sport}\\}\\) become separate classes.\n\nThis approach directly preserves observed combinations, but the number of possible combinations is \\(2^L\\). Many combinations may have few training examples, and the basic method cannot output an unseen combination as a new class. Variants train on smaller label subsets or combine several transformed problems.\n\nAlgorithm adaptation and neural models\n\nOther approaches adapt an algorithm to produce multiple labels directly. Examples include neighbor-based methods that estimate label relevance from nearby examples and tree-based methods whose leaves contain label vectors or distributions.\n\nIn deep learning, a common architecture uses a shared representation network followed by \\(L\\) output units. Each output usually applies a sigmoid function:\n\n\\[p_j=\\sigma(z_j)=\\frac{1}{1+\\exp(-z_j)}.\\]\n\nUnlike a softmax output, these probabilities are not constrained to sum to one, so several labels can receive high probabilities simultaneously. Softmax is therefore generally unsuitable as the sole output normalization for unrestricted multilabel classification.\n\nShared representations can transfer useful information between labels without explicitly modeling a joint label distribution. More elaborate models use label embeddings, graph structures, attention mechanisms, or sequential decoders. In text applications, transformer encoders can supply document representations for multilabel prediction."
    },
    {
      "heading": "Training objectives and decision rules",
      "text": "A widely used loss function is the sum or mean of binary cross-entropies:\n\n\\[\\mathcal{L}_{\\mathrm{BCE}} = -\\frac{1}{L}\\sum_{j=1}^{L} \\left[ y_j\\log p_j+(1-y_j)\\log(1-p_j) \\right].\\]\n\nThis objective trains each output to estimate a label’s marginal probability. It does not, by itself, specify how the final label set should be selected.\n\nA threshold of \\(0.5\\) is appropriate for calibrated marginal probabilities when false positives and false negatives have equal cost under an additive per-label decision objective. It is not universally optimal. Different error costs, label frequencies, or evaluation metrics may favor other thresholds.\n\nCommon decision strategies include:\n\n- Global thresholding: one threshold for all labels.\n- Per-label thresholding: a separate threshold for each label.\n- Top-\\(k\\) selection: predict the \\(k\\) highest-scoring labels.\n- Cardinality prediction: estimate how many labels apply, then select that many.\n- Constrained decoding: enforce requirements such as parent–child consistency in a taxonomy.\n\nTop-\\(k\\) selection imposes a fixed output size unless \\(k\\) is predicted separately. This can be inappropriate when examples vary greatly in their number of relevant labels.\n\nThresholds and other hyperparameters are selected using a validation set, not the final test set. Choosing thresholds after inspecting test performance creates data leakage and makes reported results overly optimistic."
    },
    {
      "heading": "Evaluation",
      "text": "No single metric fully characterizes multilabel performance. Different metrics reward different kinds of correctness.\n\nExact-match accuracy\n\nAlso called subset accuracy, exact-match accuracy requires every predicted label decision for an example to be correct:\n\n\\[\\mathrm{ExactMatch} = \\frac{1}{N}\\sum_{i=1}^{N} \\mathbf{1}[\\hat{Y}_i=Y_i].\\]\n\nIt is stringent: an otherwise correct prediction receives no credit if it omits or adds one label. It measures complete-set correctness but may be low when the vocabulary is large.\n\nHamming loss\n\nHamming loss measures the fraction of individual label decisions that are incorrect:\n\n\\[\\mathrm{HammingLoss} = \\frac{1}{NL} \\sum_{i=1}^{N}\\sum_{j=1}^{L} \\mathbf{1}[\\hat y_{ij}\\neq y_{ij}].\\]\n\nIt treats each label decision equally. In sparse datasets, predicting no labels can yield a deceptively low Hamming loss because most label indicators are negative.\n\nPrecision, recall, and F-scores\n\nPrecision measures how many predicted positives are correct; recall measures how many true positives are recovered. Their harmonic mean is\n\n\\[F_1=\\frac{2TP}{2TP+FP+FN}.\\]\n\nFor multilabel data, the averaging convention matters:\n\n- Micro-averaging pools true positives, false positives, and false negatives across all labels before calculating the metric. Frequent labels have greater influence.\n- Macro-averaging calculates the metric separately for each label and averages the results equally. It gives rare labels equal weight but can be unstable when their support is small.\n- Weighted macro-averaging weights label-level results by their support.\n- Example-based averaging calculates a metric for each example and then averages across examples.\n\nAn example-based Jaccard score measures intersection over union between predicted and true sets:\n\n\\[J_i=\\frac{|\\hat{Y}_i\\cap Y_i|}{|\\hat{Y}_i\\cup Y_i|}.\\]\n\nCases involving empty sets or zero denominators require an explicitly stated convention.\n\nRanking metrics\n\nWhen scores are important independently of a chosen threshold, ranking metrics are useful. Average precision rewards ranking relevant labels above irrelevant ones. Other measures include precision at \\(k\\), recall at \\(k\\), and ranking loss.\n\nFor a fixed example, a ranking metric can compare labels against one another. For a fixed label, a ranking metric can compare examples. These are different evaluation axes, so reports must identify which convention is used."
    },
    {
      "heading": "Data challenges and limitations",
      "text": "Imbalance and rare labels\n\nMultilabel datasets often contain both common labels and a long tail of rare ones. A model can perform well overall while failing on infrequent categories. Class-weighted losses, resampling, shared representations, and transfer learning can help, but their effects depend on the dataset and evaluation objective.\n\nWeights can also change the interpretation or calibration of output scores. Better ranking performance does not necessarily imply that the scores are accurate probabilities.\n\nMissing and uncertain annotations\n\nAn absent annotation does not always mean that a label is false. Annotators may omit relevant labels, or a dataset may record only a subset of known positives. Treating all unrecorded labels as negatives can bias training and evaluation.\n\nWhere annotation status is known, unknown entries can be masked out of the loss. Other settings require assumptions about how labels are missing. These problems differ from fully annotated multilabel classification and cannot generally be resolved by changing a decision threshold alone.\n\nLabel dependence\n\nLabels can co-occur because of semantic relationships, causal relationships, annotation conventions, or dataset-specific artifacts. Modeling dependence may improve prediction, but correlations need not remain stable under distribution shift.\n\nFor example, a system may learn that two topics usually appear together and then fail when one occurs independently. Enforcing a genuine taxonomy constraint is different from treating a frequent co-occurrence as mandatory.\n\nScale\n\nIn extreme multilabel classification, the label vocabulary may contain hundreds of thousands or millions of labels. Scoring every label for every input can be expensive, and rare labels may have very little training data.\n\nCommon strategies include hierarchical label organization, label embeddings, candidate retrieval, and sparse outputs. Retrieval-based systems first select a manageable candidate set and then rank or classify those candidates. Their final recall is limited by the relevant labels retained during candidate selection."
    },
    {
      "heading": "Applications and development",
      "text": "Early multilabel research was closely associated with document categorization, where texts naturally belong to several subjects. The field expanded into image annotation, audio tagging, biological function prediction, and large-scale content organization.\n\nIn natural language processing, applications include assigning topics, intents, or content attributes to documents and messages. In computer vision, multilabel models identify the categories present in an image without necessarily locating each instance. Audio systems use them to describe overlapping sound events or multiple musical attributes. Information retrieval systems use related methods to assign categories and improve search organization.\n\nThe development of shared neural representations has made it practical to learn from many labels jointly. Nevertheless, binary relevance remains an important baseline because it is straightforward, scalable, and sometimes well matched to the evaluation objective. Comparisons between multilabel methods depend not only on model architecture but also on annotation completeness, label frequency, threshold selection, and the definition of prediction quality."
    }
  ],
  "links": [
    "machine-learning",
    "supervised-learning",
    "multiclass-classification",
    "one-hot-encoding",
    "binary-classification",
    "image-classification",
    "object-detection",
    "support-vector-machine",
    "decision-tree-learning",
    "algorithm",
    "deep-learning",
    "attention-mechanism",
    "transformer-architecture",
    "loss-function",
    "hyperparameter",
    "validation-set",
    "test-set",
    "data-leakage",
    "intersection-over-union",
    "average-precision",
    "transfer-learning",
    "distribution-shift",
    "natural-language-processing",
    "computer-vision",
    "information-retrieval"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}