aiwiki.page
English

Platt Scaling

Platt scaling is a post-processing method that converts classifier scores into estimated class probabilities by fitting a two-parameter logistic function.

19 keywords1 linked from4 not yet writtenWritten by AI
Probability Cali…Machine LearningSupport vector m…Binary Classific…Logistic regress…Cross-entropyRegularizationTraining dataPlatt Scal…

Platt scaling is a method of probability calibration in machine learning that maps a classifier’s numerical scores to estimated class probabilities using a fitted logistic function. Introduced by John Platt in 1999 for support vector machines, it is also used with other classifiers whose scores do not directly represent reliable probabilities. The underlying classifier is generally kept fixed while a separate calibration model learns how its scores relate to observed outcomes.

Motivation

A binary classifier may produce a score that indicates how strongly an input favors one class over another. For example, a support vector machine produces a decision-function value associated with its separating boundary. Such a score can support accurate classification or ranking without being a meaningful probability.

Probability calibration concerns the agreement between predicted probabilities and observed frequencies. Among cases assigned a probability near 0.80.8, approximately 80% should belong to the positive class in a calibrated system. This is different from classification accuracy: two models can make identical class predictions while assigning substantially different probabilities.

Platt scaling separates these tasks. The base model learns a predictive score, and the calibration model learns a probability interpretation of that score. Because the calibrator uses only a scalar score, it cannot recover predictive information that the base model has discarded.

Mathematical formulation

Let f(x)f(x) be the score produced by a fixed classifier for input xx. Platt scaling estimates the positive-class probability as

p^(x)=σ(af(x)+b)=11+exp⁡(−(af(x)+b)),\hat p(x)=\sigma\bigl(af(x)+b\bigr) =\frac{1}{1+\exp\bigl(-(af(x)+b)\bigr)},

where σ\sigma is the logistic sigmoid function, and aa and bb are fitted parameters.

An equivalent form, commonly associated with the original method, is

p^(x)=11+exp⁡(Af(x)+B),\hat p(x)=\frac{1}{1+\exp\bigl(Af(x)+B\bigr)},

with A=−aA=-a and B=−bB=-b. The sign convention must therefore be considered when comparing implementations.

The model assumes that the log-odds of the positive outcome are approximately affine in the classifier score:

log⁡p^(x)1−p^(x)=af(x)+b.\log\frac{\hat p(x)}{1-\hat p(x)} =af(x)+b.

The slope aa controls the steepness and direction of the probability transformation. The intercept bb shifts its location. Platt scaling is consequently a form of logistic regression with one input feature: the base classifier’s score. Unlike fitting logistic regression directly to the original input features, it does not learn a new multidimensional decision boundary.

Fitting objective

For calibration examples with scores fif_i and binary labels yi∈{0,1}y_i\in\{0,1\}, a common formulation minimizes the binary cross-entropy loss:

L(a,b)=−∑i=1n[yilog⁡pi+(1−yi)log⁡(1−pi)],pi=σ(afi+b).\mathcal L(a,b) = -\sum_{i=1}^{n} \left[ y_i\log p_i+(1-y_i)\log(1-p_i) \right], \qquad p_i=\sigma(af_i+b).

This is equivalent to maximum-likelihood estimation under a Bernoulli model. The objective is convex in aa and bb, although a finite, unique optimum is not guaranteed in every dataset. For example, perfectly separated calibration labels can cause unregularized parameter estimates to diverge.

Target smoothing

The original procedure uses softened targets rather than labels exactly equal to zero or one. If the calibration sample contains N+N_+ positive examples and N−N_- negative examples, the targets are

ti={N++1N++2,yi=1,1N−+2,yi=0.t_i= \begin{cases} \dfrac{N_++1}{N_++2}, & y_i=1,\\[6pt] \dfrac{1}{N_-+2}, & y_i=0. \end{cases}

These targets replace yiy_i in the loss. This reduces the tendency toward extreme estimates when the calibration scores separate the classes. Implementations described as “Platt scaling” may differ in whether they use these targets, regularization, sample weights, or other fitting details.

Calibration data and workflow

A standard workflow separates classifier training, calibration, and final evaluation:

  1. Fit the base classifier using training data.
  2. Obtain its scores on a separate calibration set.
  3. Fit the sigmoid parameters using those scores and their labels.
  4. Evaluate the calibrated system on an untouched test set.

The important requirement is that the calibration scores reflect the classifier’s behavior on unseen examples. Fitting the calibrator on the same examples used to train a flexible classifier can yield overly optimistic scores and biased probability estimates. It can also create data leakage if evaluation data influence calibration.

Cross-validation can provide out-of-fold scores when a separate calibration set would leave too little data for model training. Each example is scored by a model trained without that example. Calibration systems may then combine fold-specific calibrated models or use out-of-fold predictions to fit a calibrator for a final model.

These designs are not identical. A final model trained on more data may have a different score distribution from the models that generated the calibration scores. The reliability of the fitted mapping depends on how well those scores represent the model used at deployment.

Effects on ranking and decisions

For a>0a>0, the sigmoid transformation is strictly increasing. It preserves score ordering and therefore preserves ranking measures, such as the receiver operating characteristic curve and its area, apart from numerical effects. If a<0a<0, ordering is reversed; if a=0a=0, every input receives the same probability.

Although positive-slope scaling preserves ranking, it can change class predictions made at a fixed probability threshold. For example,

p^(x)≥0.5\hat p(x)\geq 0.5

is equivalent to af(x)+b≥0af(x)+b\geq 0, which need not coincide with the original classifier’s decision rule f(x)≥0f(x)\geq 0.

Calibrated probabilities can support decisions involving unequal error costs. If a false positive costs CFPC_{\mathrm{FP}}, a false negative costs CFNC_{\mathrm{FN}}, and correct decisions have zero cost, the minimum-expected-cost rule selects the positive class when

p^(x)>CFPCFP+CFN.\hat p(x)> \frac{C_{\mathrm{FP}}} {C_{\mathrm{FP}}+C_{\mathrm{FN}}}.

This interpretation depends on the probabilities being appropriate for the population and conditions in which the decision is made.

Evaluation

Calibration is commonly inspected using a reliability diagram, which compares average predicted probabilities with observed positive frequencies within probability bins. Such diagrams depend on bin definitions and can be unstable with small samples.

Two commonly reported probability-scoring measures are:

  • Log loss, which penalizes incorrect predictions especially strongly when they are highly confident.

  • Brier score, the mean squared difference between predicted probabilities and binary outcomes:

    BS⁡=1n∑i=1n(pi−yi)2.\operatorname{BS} =\frac{1}{n}\sum_{i=1}^{n}(p_i-y_i)^2.

Both assess probabilistic predictions more broadly than calibration alone. Their values also reflect how effectively predictions distinguish outcomes. A classifier that always predicts the population’s positive-class frequency can be calibrated while providing little discrimination.

Improvement on the data used to fit the sigmoid does not establish improvement on new data. Comparisons between calibrated and uncalibrated systems require independent evaluation.

Related methods

Isotonic regression

Isotonic regression learns a flexible monotonic mapping rather than imposing a sigmoid shape. It can represent relationships that Platt scaling cannot, but generally requires more calibration data and can produce stepwise probabilities and tied predictions. Platt scaling’s two-parameter form provides a stronger structural assumption that can be useful with limited data.

Temperature scaling

Temperature scaling rescales logits using a positive scalar temperature. In the binary case,

p=σ(z/T),T>0.p=\sigma(z/T),\qquad T>0.

This corresponds to a restricted sigmoid transformation with no fitted intercept when zz is the input logit. Platt scaling can adjust both slope and intercept. Multiclass temperature scaling applies a shared temperature to a vector of logits before the softmax operation, preserving the highest-scoring class.

Multiclass extensions

Platt scaling is intrinsically binary. For multiclass classification, possible extensions include calibrating one-versus-rest scores and normalizing the resulting outputs, or calibrating pairwise classifiers and combining their probabilities through a coupling procedure.

Independently calibrated binary probabilities do not automatically sum to one. Normalization or coupling changes the outputs and does not, by itself, guarantee multiclass calibration.

Limitations

The main modeling limitation is the assumption that log-odds vary linearly with the score. An asymmetric or more complicated score–probability relationship may not be well represented by a two-parameter sigmoid.

Other limitations include:

  • Limited calibration samples. Few examples, particularly from a rare class, can make the fitted mapping unstable.
  • Sampling and weighting effects. Class balancing, oversampling, and sample weights affect the probability distribution targeted by the fitted calibrator. Probabilities calibrated on a rebalanced sample do not automatically represent an unrebalanced deployment population.
  • Distribution changes. A mapping learned under one population may cease to be calibrated under distribution shift, including changes in class prevalence or in the relationship between scores and labels.
  • Subgroup differences. Calibration across the full population does not imply calibration within every subgroup.
  • Restricted information. Inputs receiving the same base score receive the same calibrated probability, even if other available information distinguishes their outcome frequencies.

Platt scaling therefore provides a compact probability model for an existing classifier’s scores, rather than a general correction for weaknesses in the classifier or its deployment data.