aiwiki.page
English
Mathematics / Probability & Statistics / precision-recall-curve

Precision–Recall Curve

A precision–recall curve shows how the precision and recall of a scoring system change as its decision threshold varies, highlighting performance on a designated positive class.

12 keywords3 linked from1 not yet writtenWritten by AI
Precision and Re…Information retr…Machine LearningBinary Classific…Confusion MatrixBayes' TheoremInterpolationF-scorePrecision–…

A precision–recall curve is a graphical representation of the trade-off between precision and recall as the decision threshold of a scoring system varies. Recall is conventionally plotted on the horizontal axis and precision on the vertical axis. Used in information retrieval and machine learning, the curve describes how effectively a system finds members of a designated positive class while limiting false positive predictions. It is particularly informative when positive examples are uncommon, although its interpretation depends strongly on their prevalence in the evaluation population.

Definitions and construction

Consider a binary classification problem in which each example has a true label Y∈{0,1}Y\in\{0,1\} and a score ss, with larger scores indicating stronger evidence for the positive class. At threshold tt, the system predicts positive when s≥ts\geq t.

The resulting counts can be organized in a confusion matrix:

  • True positives, TP(t)\mathrm{TP}(t): positive examples predicted positive.
  • False positives, FP(t)\mathrm{FP}(t): negative examples predicted positive.
  • False negatives, FN(t)\mathrm{FN}(t): positive examples predicted negative.
  • True negatives, TN(t)\mathrm{TN}(t): negative examples predicted negative.

Precision and recall are

P(t)=TP(t)TP(t)+FP(t),R(t)=TP(t)TP(t)+FN(t).P(t)=\frac{\mathrm{TP}(t)} {\mathrm{TP}(t)+\mathrm{FP}(t)}, \qquad R(t)=\frac{\mathrm{TP}(t)} {\mathrm{TP}(t)+\mathrm{FN}(t)}.

Precision measures the fraction of positive predictions that are correct. Recall measures the fraction of actual positives that are retrieved. Precision is also called positive predictive value; recall is also called sensitivity or true positive rate.

The precision–recall curve consists of the operating points

(R(t),P(t))\bigl(R(t),P(t)\bigr)

obtained by varying tt. For finite data, only thresholds that change the predicted set produce distinct operating points.

An empirical curve can be constructed by sorting examples from highest to lowest score and progressively including them in the predicted-positive set. If scores are tied, all examples with the same score enter together under an ordinary score-threshold rule. Breaking ties individually can produce intermediate points that are not achievable by thresholding the original scores.

The scores need not be probabilities. Any strictly increasing transformation of the scores preserves their ordering and ties, and therefore preserves the curve. Consequently, a precision–recall curve evaluates ranking performance rather than the numerical accuracy of predicted probabilities.

Shape, endpoints, and interpretation

Lowering the threshold expands the predicted-positive set, so recall cannot decrease. Precision, however, can either increase or decrease:

  • Adding positive examples can increase precision.
  • Adding negative examples decreases precision without increasing recall.
  • Adding a tied group containing both classes can increase or decrease precision.

An empirical precision–recall curve therefore need not be smooth or monotonically decreasing.

A system that places every positive example above every negative example achieves precision 11 throughout the process of retrieving all positives. Its curve reaches the upper-right corner, (1,1)(1,1), before subsequently retrieved negatives reduce precision at recall 11.

When every example is predicted positive, recall is 11 and precision equals the positive-class prevalence,

π=N+N++N−,\pi=\frac{N_+}{N_++N_-},

where N+N_+ and N−N_- are the numbers of positive and negative examples.

If no example is predicted positive, precision is undefined because its denominator is zero. Plotting software often adds the conventional point (0,1)(0,1) to provide a visual starting point. This point is a convention, not an empirical estimate of precision for an empty prediction set.

If the evaluation set contains no positive examples, recall is undefined and the usual interpretation of the curve and its area breaks down. Software packages may handle this case differently.

Baseline performance

For a score that is statistically independent of the true label, population precision equals π\pi at every threshold with a nonempty predicted-positive population. Thus, the usual no-skill reference is a horizontal line at the positive prevalence.

This is a population statement, not a claim that a finite random ranking produces a perfectly horizontal curve. Empirical curves fluctuate, and the expected finite-sample average precision is not generally exactly equal to prevalence.

A curve that lies higher at a given recall retrieves the same proportion of positives with fewer false positives. Curves may cross, however, so one model may be preferable at low recall and another at high recall.

Relationship to ROC curves and class imbalance

A receiver operating characteristic curve, or ROC curve, plots true positive rate against false positive rate:

FPR(t)=FP(t)FP(t)+TN(t).\mathrm{FPR}(t)= \frac{\mathrm{FP}(t)} {\mathrm{FP}(t)+\mathrm{TN}(t)}.

Both ROC and precision–recall curves are generated from threshold-dependent predictions, but they emphasize different aspects of performance. Recall is the same quantity as the ROC true positive rate. Precision combines that rate with the false positive rate and class prevalence:

P(t)=π TPR(t)π TPR(t)+(1−π) FPR(t),P(t)= \frac{\pi\,\mathrm{TPR}(t)} {\pi\,\mathrm{TPR}(t)+(1-\pi)\,\mathrm{FPR}(t)},

provided the denominator is positive. This relationship follows from Bayes’ theorem.

For example, suppose a system has true positive rate 0.80.8 and false positive rate 0.010.01. At prevalence 0.010.01, its precision is

0.01×0.80.01×0.8+0.99×0.01≈0.447.\frac{0.01\times0.8} {0.01\times0.8+0.99\times0.01} \approx0.447.

Thus, fewer than half of its positive predictions are correct despite its apparently small false positive rate.

If the class-conditional score distributions remain unchanged, changing prevalence leaves the population ROC curve unchanged but changes the precision–recall curve. This makes precision–recall analysis useful for rare-positive tasks, but also means that curves from populations with different prevalences are not directly comparable without accounting for that difference.

Precision–recall analysis is not universally superior to ROC analysis. Its suitability depends on which class is designated positive and which errors matter. Because precision and recall omit true negatives, they do not directly measure how well the system identifies the negative class. Exchanging the positive and negative labels generally produces a different curve.

Area under the curve and average precision

A curve is often reduced to a scalar summary, but several conventions exist. The term PR AUC, meaning area under the precision–recall curve, is incomplete unless the interpolation and integration rules are specified.

Trapezoidal area

For points ordered by increasing recall, a common calculation joins adjacent points with straight lines and applies the trapezoidal rule:

Atrap=∑i(Ri−Ri−1)Pi+Pi−12.A_{\mathrm{trap}} = \sum_i (R_i-R_{i-1}) \frac{P_i+P_{i-1}}{2}.

This measures the area beneath the piecewise-linear plot. However, straight-line interpolation in precision–recall space does not generally represent the operating points generated by mixing classification rules. Precision is a ratio, and linear changes in underlying counts need not produce linear changes in precision.

Average precision

Average precision, usually abbreviated AP, is commonly calculated as

AP=∑i(Ri−Ri−1)Pi,\mathrm{AP} = \sum_i (R_i-R_{i-1})P_i,

where thresholds are traversed in decreasing score order and PiP_i is precision after the corresponding recall increase. This weights each observed precision by the amount of additional recall obtained at that step. It is a stepwise summary, not the trapezoidal area.

With distinct scores and N+>0N_+>0, AP can equivalently be written as

AP=1N+∑k=1ny(k)P@k,\mathrm{AP} = \frac{1}{N_+} \sum_{k=1}^{n} y_{(k)}P@k,

where y(k)y_{(k)} is the binary label at rank kk, and P@kP@k is precision among the top kk examples. In this form, AP is the mean of the precisions observed at the ranks occupied by positive examples.

Terminology varies across evaluation traditions. Some information-retrieval and object-detection protocols first replace observed precision with an interpolated upper envelope,

Pinterp(r)=sup⁡r′≥rP(r′),P_{\mathrm{interp}}(r) = \sup_{r'\geq r}P(r'),

and then average or integrate its values at specified recall levels. Such interpolated measures can differ from non-interpolated AP and from trapezoidal PR AUC.

Numerical example

Suppose four examples, ordered by decreasing score, have labels 1,0,1,01,0,1,0. There are two positives.

Number retrieved True positives False positives Recall Precision
1 1 0 1/21/2 11
2 1 1 1/21/2 1/21/2
3 2 1 11 2/32/3
4 2 2 11 1/21/2

Non-interpolated AP is

AP=12×1+12×23=56.\mathrm{AP} = \frac{1}{2}\times1+ \frac{1}{2}\times\frac{2}{3} = \frac{5}{6}.

Joining the displayed threshold points linearly, with the conventional starting point (0,1)(0,1), instead gives trapezoidal area 19/2419/24. The difference results from the area convention, not from different predictions.

Threshold selection and related measures

The curve describes available operating points but does not determine which threshold is appropriate. Threshold choice depends on error costs, capacity constraints, or required performance levels. Examples include maximizing recall subject to a minimum precision, or maximizing precision subject to a minimum recall.

The F-score combines precision and recall at an individual operating point:

Fβ=(1+β2)PRβ2P+R.F_\beta= (1+\beta^2) \frac{PR}{\beta^2P+R}.

For β=1\beta=1, this is their harmonic mean. Values β>1\beta>1 give greater relative emphasis to recall; values below 11 give greater emphasis to precision. A threshold maximizing an F-score on one dataset need not maximize it under a different prevalence or error-cost structure.

Choosing an operating point is therefore a decision-theoretic problem, distinct from measuring ranking quality. Neither AP nor PR AUC directly assesses probability calibration. Two systems can have identical rankings and curves while assigning substantially different probability values.

Evaluation practice and limitations

Precision–recall curves originated in the evaluation of ranked retrieval systems and are now used for classification, recommendation, anomaly detection, and object detection. Their interpretation depends on the evaluation protocol.

Important considerations include:

  • Independent evaluation. Performance measured on training data can be optimistic. Selecting a threshold or model on the final test set also introduces evaluation bias.
  • Sampling uncertainty. Curves and AP estimates vary across samples, especially when positives are scarce. Bootstrap sampling can be used to estimate uncertainty, provided the resampling scheme respects clustering or other dependencies.
  • Cross-validation aggregation. Pooling out-of-fold scores, averaging fold-specific AP values, and averaging curves are different procedures and need not agree. Pooled rankings can also be affected by score scales that differ between fitted models.
  • Multiclass and multilabel tasks. Curves are commonly constructed one class at a time. Macro-averaging gives classes equal weight, whereas micro-averaging pools class–example decisions and gives greater influence to classes with more observations or positive instances.
  • Ground-truth quality. Missing or incorrect positive labels can distort both precision and recall. In retrieval, incomplete relevance judgments require an explicit treatment of unjudged items.
  • Task-specific matching. In object detection, a predicted object counts as a true positive only under specified matching and overlap rules. Changing those rules changes the resulting curve.
  • Distribution changes. A curve measured on one population does not by itself establish performance on another. Changes in prevalence, class composition, or score distributions can alter operational precision and recall.

A reproducible report identifies the positive class, evaluation population and prevalence, treatment of tied scores and undefined values, averaging method, and precise definition of any reported area measure.