{
  "id": "confusion-matrix",
  "title": "Confusion Matrix",
  "lang": "en",
  "source_lang": "en",
  "summary": "A confusion matrix tabulates reference and predicted class labels, revealing correct classifications, error types, and the counts used to calculate classification metrics.",
  "category": "mathematics",
  "subcategory": "probability-statistics",
  "type": "concept",
  "aliases": [
    "Error matrix",
    "Classification error matrix"
  ],
  "generated_at": "2026-11-03",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/confusion-matrix/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/confusion-matrix/",
      "json": "https://aiwiki.page/zh/confusion-matrix.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "A confusion matrix is a table that compares the class labels assigned by a classification system with reference labels for the same observations. Its entries record how often each reference class is predicted as each possible class. Used in statistics and machine learning, it distinguishes correct predictions from particular kinds of mistakes rather than reducing performance to a single score. The reference labels are often called actual, true, or ground-truth labels, although they may themselves contain errors."
    },
    {
      "heading": "Structure and mathematical definition",
      "text": "For a classification task with \\(K\\) mutually exclusive classes, a confusion matrix is a \\(K \\times K\\) matrix. A common convention places reference classes in rows and predicted classes in columns:\n\n\\[C_{ij} = \\sum_{n=1}^{N} \\mathbf{1}\\{y_n=i,\\ \\hat y_n=j\\},\\]\n\nwhere \\(N\\) is the number of evaluated observations, \\(y_n\\) is the reference label, \\(\\hat y_n\\) is the predicted label, and \\(\\mathbf{1}\\) is an indicator function.\n\nUnder this convention:\n\n- \\(C_{ii}\\) counts correct predictions for class \\(i\\).\n- \\(C_{ij}\\), for \\(i\\ne j\\), counts observations of class \\(i\\) incorrectly assigned to class \\(j\\).\n- The row sum \\(n_i=\\sum_j C_{ij}\\) is the number of reference observations in class \\(i\\), often called its support.\n- The column sum \\(m_j=\\sum_i C_{ij}\\) is the number of predictions assigned to class \\(j\\).\n- The sum of all entries is \\(N\\).\n\nSome publications and software packages reverse the axes. Consequently, row and column labels must be checked before interpreting a table; the phrase “confusion matrix” alone does not establish its orientation.\n\nA weighted confusion matrix replaces each unit count with an observation weight \\(w_n\\):\n\n\\[C_{ij} = \\sum_{n=1}^{N} w_n\\mathbf{1}\\{y_n=i,\\ \\hat y_n=j\\}.\\]\n\nWeighted entries need not be integers. They may represent sampling weights, exposure, or another explicitly defined weighting scheme."
    },
    {
      "heading": "Binary classification",
      "text": "For two classes, one is conventionally designated positive and the other negative. These labels identify the outcome of interest; “positive” does not necessarily mean desirable.\n\nWith reference labels in rows, the matrix is:\n\nReference class | Predicted positive | Predicted negative\nPositive | True positive, \\(TP\\) | False negative, \\(FN\\)\nNegative | False positive, \\(FP\\) | True negative, \\(TN\\)\n\nA false positive is a negative observation classified as positive. A false negative is a positive observation classified as negative. The distinction matters whenever the two errors have different consequences, such as in spam filtering, fault detection, or document retrieval.\n\nThese errors are related to Type I and Type II errors in hypothesis testing when a positive decision corresponds to rejecting a negative null hypothesis. The correspondence depends on that setup; classification terminology does not itself specify a hypothesis test.\n\nMetrics derived from the matrix\n\nMany classification metrics are ratios of these four counts.\n\nMetric | Formula | Interpretation\nAccuracy | \\(\\frac{TP+TN}{N}\\) | Fraction of all predictions that are correct\nError rate | \\(\\frac{FP+FN}{N}\\) | Fraction of all predictions that are incorrect\nPrecision | \\(\\frac{TP}{TP+FP}\\) | Fraction of positive predictions that are reference-positive\nRecall, sensitivity, or true positive rate | \\(\\frac{TP}{TP+FN}\\) | Fraction of reference-positive observations detected\nSpecificity or true negative rate | \\(\\frac{TN}{TN+FP}\\) | Fraction of reference-negative observations correctly rejected\nFalse positive rate | \\(\\frac{FP}{FP+TN}\\) | Fraction of reference-negative observations incorrectly flagged\nFalse negative rate | \\(\\frac{FN}{FN+TP}\\) | Fraction of reference-positive observations missed\nNegative predictive value | \\(\\frac{TN}{TN+FN}\\) | Fraction of negative predictions that are reference-negative\n\nPrecision and recall condition on different groups: precision concerns predicted positives, whereas recall concerns reference positives. Neither determines the other.\n\nThe F1 score is their harmonic mean:\n\n\\[F_1 = \\frac{2\\,\\mathrm{Precision}\\,\\mathrm{Recall}} {\\mathrm{Precision}+\\mathrm{Recall}} = \\frac{2TP}{2TP+FP+FN}.\\]\n\nIt does not directly include true negatives. Binary balanced accuracy gives equal weight to the recall of each reference class:\n\n\\[\\mathrm{Balanced\\ accuracy} = \\frac{1}{2} \\left( \\frac{TP}{TP+FN} + \\frac{TN}{TN+FP} \\right).\\]\n\nA metric with a zero denominator is mathematically undefined. Software may return a missing value, issue a warning, or substitute a specified value. Such conventions can affect comparisons, especially when a class is absent or never predicted."
    },
    {
      "heading": "Worked example",
      "text": "Suppose a classifier is evaluated on 1,000 observations:\n\nReference class | Predicted positive | Predicted negative | Total\nPositive | 80 | 20 | 100\nNegative | 90 | 810 | 900\nTotal | 170 | 830 | 1,000\n\nIts accuracy is\n\n\\[\\frac{80+810}{1000}=89\\%.\\]\n\nIts recall is \\(80/100=80\\%\\), its specificity is \\(810/900=90\\%\\), and its precision is approximately \\(80/170=47.1\\%\\). Its F1 score is approximately \\(59.3\\%\\), while balanced accuracy is \\(85\\%\\).\n\nA system that always predicts negative would achieve 90% accuracy on these observations, but zero recall for the positive class. This illustrates why accuracy alone can obscure performance when class frequencies are unequal. Whether either system is preferable depends on the purpose of classification and the relative consequences of its errors."
    },
    {
      "heading": "Counts, normalization, and probability",
      "text": "The raw matrix preserves the number of observations behind each result. Normalized matrices emphasize proportions instead.\n\nOverall normalization\n\nDividing every entry by \\(N\\) produces\n\n\\[\\widehat p_{ij}=\\frac{C_{ij}}{N}.\\]\n\nThese values form the empirical joint probability distribution of reference and predicted labels. They sum to one across the whole matrix.\n\nRow normalization\n\nDividing each entry by its row total gives\n\n\\[R_{ij}=\\frac{C_{ij}}{n_i}.\\]\n\nFor nonempty rows, this estimates the conditional probability\n\n\\[P(\\hat Y=j\\mid Y=i).\\]\n\nEach such row sums to one. Diagonal entries are class-specific recalls, making this display useful for comparing how reliably different reference classes are recognized.\n\nColumn normalization\n\nDividing by column totals gives\n\n\\[Q_{ij}=\\frac{C_{ij}}{m_j},\\]\n\nwhich estimates\n\n\\[P(Y=i\\mid \\hat Y=j).\\]\n\nEach nonempty column sums to one, and diagonal entries are class-specific precisions.\n\nRow- and column-normalized matrices answer different questions. A normalized display should identify its denominator and, where possible, retain class counts so that a proportion based on a few observations is distinguishable from one based on thousands."
    },
    {
      "heading": "Multiclass and multilabel classification",
      "text": "Multiclass classification\n\nIn a single-label multiclass task, every observation belongs to exactly one of \\(K\\) classes and receives exactly one predicted class. The off-diagonal entries reveal which classes are confused with which others.\n\nEach class \\(i\\) can also be evaluated against all remaining classes:\n\n\\[TP_i=C_{ii}, \\qquad FN_i=\\sum_{j\\ne i}C_{ij},\\]\n\n\\[FP_i=\\sum_{j\\ne i}C_{ji}, \\qquad TN_i=N-TP_i-FN_i-FP_i.\\]\n\nThese quantities yield per-class precision, recall, and F-scores. Common summaries include:\n\n- Macro averaging: calculate a metric for each class and take the unweighted mean.\n- Support-weighted averaging: weight each class’s metric by its reference support.\n- Micro averaging: sum the relevant counts across classes before calculating the metric.\n\nIn ordinary single-label multiclass evaluation over all classes, micro-averaged precision, recall, and F1 all equal accuracy. Macro-averaged recall gives each class equal weight and is commonly called multiclass balanced accuracy.\n\nThe averaging rule matters. Macro F1, for example, usually means the mean of the class-specific F1 scores, not the harmonic mean of macro precision and macro recall.\n\nMultilabel classification\n\nIn multilabel classification, an observation may have several labels simultaneously. A single \\(K\\times K\\) table generally cannot represent all label-presence decisions.\n\nA common approach constructs a separate binary confusion matrix for each label, treating that label as present or absent. These matrices support per-label and aggregated metrics, but do not fully capture dependencies among labels or the accuracy of complete predicted label sets."
    },
    {
      "heading": "Thresholds and decision costs",
      "text": "Many classifiers, including logistic regression, produce scores or estimated probabilities before assigning labels. In binary classification, a threshold \\(t\\) may define the decision rule:\n\n\\[\\hat y= \\begin{cases} 1,&s(x)\\ge t,\\\\ 0,&s(x)<t. \\end{cases}\\]\n\nFor fixed scores and observations, lowering the threshold cannot decrease either the true positive count or the false positive count. A confusion matrix therefore describes performance at a particular decision rule, rather than every possible operating point. Receiver operating characteristic curves and precision–recall curves summarize performance across thresholds.\n\nError costs can be incorporated using a loss function. If \\(L_{ij}\\) is the cost of predicting class \\(j\\) when the reference class is \\(i\\), the average observed cost is\n\n\\[\\widehat R = \\frac{1}{N} \\sum_{i=1}^{K}\\sum_{j=1}^{K} C_{ij}L_{ij}.\\]\n\nWhen correct decisions have zero cost and every incorrect decision has unit cost, this reduces to the error rate. Different costs can favor different classifiers or thresholds even when their accuracies are similar, connecting confusion-matrix analysis with decision theory."
    },
    {
      "heading": "Applications and limitations",
      "text": "Confusion matrices are used to compare classifiers, inspect error patterns, identify poorly recognized classes, and investigate whether a model systematically substitutes one label for another. Applications include text classification, image recognition, remote sensing, speech recognition, and industrial inspection.\n\nTheir interpretation has several important limits.\n\nClass frequencies affect the results. Precision depends on the proportion of positive cases in the evaluated population. If that proportion is \\(\\pi\\), sensitivity is \\(r\\), and the false positive rate is \\(f\\), then Bayes’ theorem gives\n\n\\[P(Y=1\\mid\\hat Y=1) = \\frac{r\\pi}{r\\pi+f(1-\\pi)},\\]\n\nprovided the denominator is nonzero. Precision can therefore change across populations even if sensitivity and specificity remain unchanged.\n\nEvaluation data must represent the intended setting. Results on training data generally do not establish performance on unseen observations. Held-out evaluation and cross-validation address aspects of this problem, but distribution changes, nonrepresentative sampling, and information leakage can still undermine interpretation.\n\nReference labels may be imperfect. Disagreement with a reference label does not establish that the classifier is wrong in an absolute sense. Ambiguous definitions, annotation errors, and disagreements among annotators can contribute to off-diagonal counts.\n\nCounts have sampling uncertainty. Rates calculated from finite samples vary with the observations selected. Confidence intervals or bootstrap methods can quantify aspects of that uncertainty. Dependence among observations, such as repeated records from the same person, must be considered when estimating it.\n\nHard labels discard score information. Two models can have identical confusion matrices while assigning very different probabilities. The matrix alone cannot establish probability calibration or determine score-based losses such as cross-entropy.\n\nAggregation hides structure. A matrix does not preserve individual observations, subgroup differences, time ordering, or the circumstances of each error. Additional analyses are needed to determine where errors occur and whether performance differs across relevant groups.\n\nFor structured prediction tasks such as object detection, a confusion matrix also depends on how predictions are matched to reference objects. Rejected predictions, unknown classes, unmatched detections, or missing reference objects require explicit conventions; otherwise, apparently similar matrices may count different events."
    }
  ],
  "links": [
    "statistics",
    "machine-learning",
    "matrix",
    "indicator-function",
    "type-i-and-type-ii-errors",
    "precision-and-recall",
    "f-score",
    "joint-probability-distribution",
    "conditional-probability",
    "logistic-regression",
    "loss-function",
    "decision-theory",
    "bayes-theorem",
    "training-data",
    "cross-validation",
    "confidence-interval",
    "bootstrap-sampling",
    "cross-entropy"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}