{
  "id": "multiclass-classification",
  "title": "Multiclass Classification",
  "lang": "en",
  "source_lang": "en",
  "summary": "Multiclass classification is a machine-learning task that assigns each input to exactly one of three or more predefined classes.",
  "category": "computer-science",
  "subcategory": "artificial-intelligence",
  "type": "concept",
  "aliases": [
    "Multi-class classification",
    "Multinomial classification"
  ],
  "generated_at": "2026-11-03",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/multiclass-classification/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/multiclass-classification/",
      "json": "https://aiwiki.page/zh/multiclass-classification.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Multiclass classification is a supervised learning task in which a model assigns an input to exactly one of three or more predefined categories. Examples include recognizing a handwritten digit as one of ten digits, assigning a document to a single topic, and identifying the main object in an image. It extends binary classification, which distinguishes between only two classes, and differs from multilabel classification, in which an input may receive several labels simultaneously."
    },
    {
      "heading": "Problem formulation",
      "text": "A multiclass classification problem has an input space \\(\\mathcal{X}\\) and a finite label set \\(\\mathcal{Y}=\\{1,\\ldots,K\\}\\), with \\(K>2\\). Given training data\n\n\\[D=\\{(x_i,y_i)\\}_{i=1}^{n},\\]\n\nthe objective is to learn a classifier\n\n\\[f:\\mathcal{X}\\rightarrow\\mathcal{Y}\\]\n\nthat predicts labels accurately for previously unseen inputs.\n\nThe labels represent categories, not numerical quantities. Encoding three animal species as 1, 2, and 3 does not imply that the third species is greater than the second or that the distance between species is meaningful. One-hot encoding represents each class by a vector whose corresponding entry is 1 and whose other entries are 0, although many implementations accept integer class indices directly.\n\nA probabilistic classifier estimates conditional class probabilities,\n\n\\[p_k(x)=P(Y=k\\mid X=x), \\qquad \\sum_{k=1}^{K}p_k(x)=1.\\]\n\nWhen every incorrect classification has the same cost, the standard prediction rule selects the class with the largest estimated probability:\n\n\\[\\hat y(x)=\\operatorname*{arg\\,max}_{k}p_k(x).\\]\n\nWith the true conditional probabilities, this rule minimizes the probability of misclassification. If different errors have different costs, the appropriate rule instead minimizes expected cost.\n\nRelated classification settings\n\nSeveral neighboring problems have different output assumptions:\n\n- Multilabel classification: each input can belong to several classes, such as a photograph labeled both “beach” and “sunset.”\n- Ordinal classification: classes have an inherent order, such as low, medium, and high severity.\n- Hierarchical classification: classes occupy a taxonomy, and predictions may be made at different levels.\n- Open-set recognition: inputs may belong to classes absent from training, rather than necessarily belonging to one of the known classes.\n\nStandard multiclass classification is usually closed-set: the available classes are assumed to cover all admissible inputs. A classifier can nevertheless produce a label for an input outside this intended domain, so producing an answer does not establish that the input belongs to a known class."
    },
    {
      "heading": "Modeling approaches",
      "text": "Multiclass problems can be handled either by models that represent all classes jointly or by combinations of binary classifiers.\n\nJoint multiclass models\n\nSeveral machine-learning methods naturally support more than two classes:\n\n- Decision trees partition the input space and assign class labels or class distributions to their leaves.\n- Random forests combine predictions from multiple trees.\n- A naive Bayes classifier combines class priors with class-conditional feature likelihoods under a conditional-independence assumption.\n- The k-nearest neighbors algorithm predicts from the labels of nearby training examples.\n- Artificial neural networks can produce a score for each class and learn all scores jointly.\n\nMultinomial logistic regression uses a linear score for each class followed by a softmax transformation. Neural classifiers generalize this arrangement by learning nonlinear representations before the final class-scoring layer.\n\nOne-versus-rest\n\nThe one-versus-rest strategy trains \\(K\\) binary classifiers. Classifier \\(k\\) distinguishes class \\(k\\) from all other classes. At prediction time, the class with the highest decision score is typically selected.\n\nThis approach allows binary learning methods to be reused, but each binary problem may be imbalanced, especially when \\(K\\) is large. Independent binary predictions may also select several positive classes or none at all; a score-based selection rule resolves the final single-label prediction.\n\nScores from separately trained classifiers are not automatically comparable as probabilities. Their interpretation depends on the underlying model and any subsequent calibration.\n\nOne-versus-one\n\nThe one-versus-one strategy trains a classifier for every pair of classes, giving\n\n\\[\\frac{K(K-1)}{2}\\]\n\nbinary classifiers. Each classifier is trained using examples from only its two classes. Predictions are combined through voting or another aggregation procedure.\n\nAlthough this produces many classifiers, each uses a smaller training subset. It has been widely used with support vector machines. Its practical cost depends on the learning algorithm, class sizes, and prediction procedure, not just the number of classifiers.\n\nError-correcting output codes\n\nError-correcting output codes assign each class a binary or ternary codeword. Binary classifiers predict code components, and a decoding rule chooses the closest class codeword.\n\nInspired by error-correcting codes, this approach can provide redundancy: some component predictions may be wrong without changing the decoded class. Its effectiveness depends on code design, classifier accuracy, and diversity among component errors."
    },
    {
      "heading": "Softmax and training objectives",
      "text": "A common multiclass model produces real-valued scores, called logits, \\(z_1,\\ldots,z_K\\). The softmax function converts these into a normalized probability vector:\n\n\\[p_k=\\frac{\\exp(z_k)}{\\sum_{j=1}^{K}\\exp(z_j)}.\\]\n\nFor numerical stability, implementations commonly subtract the largest logit before exponentiation. This leaves the probabilities unchanged.\n\nThe standard loss function for probabilistic multiclass classification is categorical cross-entropy. For a training example whose correct class is \\(y\\),\n\n\\[\\mathcal{L}(x,y)=-\\log p_y(x).\\]\n\nFor one-hot targets \\(t_k\\), the equivalent expression is\n\n\\[\\mathcal{L}(x,t)=-\\sum_{k=1}^{K}t_k\\log p_k(x).\\]\n\nMinimizing average cross-entropy corresponds to maximizing the conditional likelihood of the observed labels. Neural networks commonly optimize this objective using backpropagation and gradient-based methods.\n\nCross-entropy is not the only option. Multiclass margin losses, including hinge-style objectives, encourage the correct class score to exceed competing scores by a specified margin. Class weighting changes the relative influence of examples from different categories. Label smoothing replaces hard one-hot targets with softened targets and can affect both learning and confidence estimates.\n\nRegularization constrains model fitting and can reduce overfitting. A softmax output alone does not ensure that reported probabilities accurately reflect real-world frequencies."
    },
    {
      "heading": "Evaluation",
      "text": "Evaluation requires data separate from model fitting. A validation set supports model selection, while a held-out test set estimates performance after those choices have been made. Duplicated or related observations crossing these boundaries can cause data leakage and inflate measured performance.\n\nConfusion matrix and accuracy\n\nA confusion matrix records the number of examples for every true-class and predicted-class pair. It reveals which categories are confused, although the convention for placing true labels on rows or columns must be specified.\n\nAccuracy is the fraction of correctly classified examples:\n\n\\[\\mathrm{Accuracy} =\\frac{1}{n}\\sum_{i=1}^{n}\\mathbf{1}[\\hat y_i=y_i].\\]\n\nIt is useful when classes and errors have comparable importance, but it can conceal poor performance on rare categories. A classifier that almost always predicts the largest class may achieve high accuracy without usefully distinguishing the others.\n\nPrecision, recall, and F-scores\n\nFor each class, all other classes can be treated as the negative category:\n\n\\[P_k=\\frac{TP_k}{TP_k+FP_k}, \\qquad R_k=\\frac{TP_k}{TP_k+FN_k}.\\]\n\nThe per-class F1 score is\n\n\\[F_{1,k}=\\frac{2P_kR_k}{P_k+R_k}.\\]\n\nResults can be aggregated in several ways:\n\n- Macro averaging gives every class equal weight.\n- Weighted averaging weights class scores by the number of true examples in each class.\n- Micro averaging pools counts across classes before computing a score.\n\nIn ordinary single-label multiclass evaluation, when all classes are included and every example receives exactly one prediction, micro-averaged precision, recall, and F1 all equal accuracy. Macro F1 is normally the mean of per-class F1 scores, not the F1 computed from macro-averaged precision and recall. Implementations must define how to handle zero denominators.\n\nRanked predictions and probability quality\n\nTop-\\(k\\) accuracy counts a prediction as correct when the true class appears among the \\(k\\) highest-scoring classes. It is useful when a system presents several candidates, but depends strongly on the number and difficulty of the available classes.\n\nLog loss and multiclass Brier scores evaluate predicted probabilities rather than only the winning class. Probability calibration concerns whether stated confidence corresponds to observed correctness. A model can classify accurately while being systematically overconfident."
    },
    {
      "heading": "Applications",
      "text": "Multiclass classification appears across computer vision, natural language processing, and speech recognition.\n\nIn image classification, a model may assign an image to one object category. In document processing, it may select a topic, language, or routing destination. Character recognition treats individual symbols as classes. In semantic segmentation, each pixel receives a class label, making multiclass prediction a component of a larger structured-output task.\n\nThe task depends on the labeling scheme rather than the data type alone. Assigning one topic to an article is multiclass classification; assigning every applicable topic is multilabel classification. Predicting a numerical property from the same article is regression."
    },
    {
      "heading": "Limitations and practical challenges",
      "text": "Class imbalance and scarce examples. Rare classes may contribute little to the overall training objective and have uncertain evaluation estimates. Weighting and resampling alter the learning problem, but their effects must be assessed against the intended deployment distribution.\n\nAmbiguous or inconsistent labels. A single-label formulation may force a choice between overlapping categories. Annotation errors and unclear class definitions can limit achievable agreement independently of model capacity.\n\nDistribution changes. Distribution shift can change input characteristics, class frequencies, or relationships between inputs and labels. Performance on a fixed test set does not guarantee equivalent performance after such changes.\n\nUnknown inputs and confidence. Closed-set models normally select a known class even for unfamiliar inputs. Rejection mechanisms and open-set methods address this separately; high softmax confidence is not reliable proof that an input is familiar.\n\nLarge label spaces. As class counts grow, output-layer computation, label storage, and data requirements can become substantial. Hierarchical prediction, candidate retrieval, and approximate normalization can reduce some costs, but introduce additional design choices and possible errors.\n\nUnequal error costs. Ordinary accuracy treats every mistake alike. Where confusing one pair of classes has different consequences from confusing another, evaluation and decision rules must reflect those costs rather than relying only on the most probable label."
    }
  ],
  "links": [
    "supervised-learning",
    "binary-classification",
    "training-data",
    "one-hot-encoding",
    "machine-learning",
    "decision-tree-learning",
    "random-forest",
    "naive-bayes-classifier",
    "k-nearest-neighbors-algorithm",
    "artificial-neural-network",
    "support-vector-machine",
    "error-correcting-code",
    "softmax-function",
    "loss-function",
    "backpropagation",
    "regularization",
    "overfitting",
    "validation-set",
    "test-set",
    "data-leakage",
    "confusion-matrix",
    "computer-vision",
    "natural-language-processing",
    "speech-recognition",
    "image-classification",
    "semantic-segmentation",
    "distribution-shift"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}