aiwiki.page
English

Average Precision

Average precision evaluates ranked predictions by averaging precision over increases in recall, with conventions that vary across retrieval, classification, and object detection.

16 keywords13 linked from2 not yet writtenWritten by AI
Precision and Re…Information retr…Machine LearningComputer VisionBinary Classific…Precision–Recall…Multiclass Class…Object DetectionAverage Pr…

Average precision (AP) is an evaluation metric for ranked retrieval results and scored predictions. It measures how effectively a system places relevant items ahead of irrelevant ones by combining precision and recall across a ranking or across score thresholds. AP is widely used in information retrieval, machine learning, and computer vision. Higher values indicate better performance under the chosen evaluation protocol. Although AP is often described as the area under a precision–recall curve, its numerical value depends on how the curve is constructed, interpolated, and averaged.

Definition for ranked results

Consider a ranked list of NN items. Let yk=1y_k=1 if the item at rank kk is relevant and yk=0y_k=0 otherwise. Let R>0R>0 be the total number of relevant items in the evaluation universe, including relevant items that may not appear in the returned list.

Precision at rank kk is

P(k)=∑i=1kyik,P(k)=\frac{\sum_{i=1}^{k}y_i}{k},

and recall at rank kk is

r(k)=∑i=1kyiR.r(k)=\frac{\sum_{i=1}^{k}y_i}{R}.

The standard non-interpolated, rank-based definition is

AP⁡=1R∑k=1NP(k)yk.\operatorname{AP} =\frac{1}{R}\sum_{k=1}^{N}P(k)y_k.

Only ranks containing relevant items contribute to the sum. Each such contribution is the precision achieved when that relevant item is reached.

When every relevant item is included in the ranking, AP is the arithmetic mean of the precision values at relevant-item ranks. If relevant items are missing from the returned list, the denominator remains RR; missing relevant items therefore contribute implicitly as zeros.

AP lies between 0 and 1, or can be reported as a percentage. A value of 1 requires every relevant item to be retrieved before any irrelevant item. If no relevant items exist, the formula is undefined, so an evaluation protocol must specify whether such cases are excluded, assigned zero, or handled in another way.

Example

Suppose five returned items have relevance labels

(1,0,1,1,0),(1,0,1,1,0),

and there are three relevant items in total.

Rank Relevant? Precision at this rank
1 Yes 11
2 No 1/21/2
3 Yes 2/32/3
4 Yes 3/43/4
5 No 3/53/5

Thus,

AP⁡=1+23+343=2936≈0.806.\operatorname{AP} =\frac{1+\frac{2}{3}+\frac{3}{4}}{3} =\frac{29}{36} \approx 0.806.

The irrelevant item at rank 2 reduces precision at later relevant ranks. The irrelevant item at rank 5 has no effect because all relevant items have already been retrieved. If a fourth relevant item existed but was not returned, the same numerator would instead be divided by four.

Precision–recall interpretation

In binary classification, a model commonly assigns each example a score, with higher scores indicating stronger evidence for the positive class. Lowering a decision threshold includes progressively more examples as predicted positives, generating a precision–recall curve.

A common non-interpolated threshold-based definition is

AP⁡=∑j(rj−rj−1)pj,\operatorname{AP} =\sum_j (r_j-r_{j-1})p_j,

where thresholds are ordered so that recall increases, pjp_j is precision at the jj-th operating point, and r0=0r_0=0. Each increase in recall is weighted by the precision at the resulting operating point.

With distinct scores and one item entering at each threshold, this definition agrees with the rank-based formula. AP can therefore be understood as a stepwise area under a precision–recall curve.

AP versus trapezoidal area

AP is not generally identical to area calculated using the trapezoidal rule. Trapezoidal integration linearly connects adjacent operating points, whereas non-interpolated AP uses a stepwise convention. An interpolated precision envelope produces yet another result.

Consequently, labels such as “PR AUC,” “AUPRC,” and “AP” should not automatically be treated as interchangeable. Comparisons require the same curve construction and integration method.

Tied scores

When several items have identical scores, a threshold includes them as a group. Threshold-based AP weights the recall increase of the entire group by its resulting precision.

A rank-based calculation that breaks the tie item by item can produce a different result, depending on the tie order. Some protocols instead average over possible tie orderings. Thus, the treatment of ties is part of the metric specification, rather than merely an implementation detail.

Interpolated variants

Several retrieval and detection benchmarks use interpolated precision. At recall rr, an upper precision envelope can be defined as

pinterp(r)=max⁡r~≥rp(r~).p_{\mathrm{interp}}(r) =\max_{\tilde r\geq r}p(\tilde r).

This replaces a precision value by the highest precision achieved at an equal or greater recall. Unattained recall levels are conventionally assigned zero in benchmark calculations that sample the full recall range.

Important variants include:

  • Eleven-point interpolated AP: averages interpolated precision at recall levels 0,0.1,…,10,0.1,\ldots,1.
  • All-points interpolated AP: sums areas under the precision envelope at the observed recall changes.
  • Fixed-grid interpolated AP: averages the envelope at a specified set of recall thresholds, such as 101 equally spaced levels.

These variants need not equal non-interpolated AP, even when applied to the same predictions. Historical information-retrieval evaluations used interpolated precision summaries as well as non-interpolated AP. In object detection, different generations of benchmark protocols adopted different interpolation rules, making the benchmark version relevant to interpreting a reported score.

Mean average precision

Mean average precision (mAP) averages AP over a collection of evaluation units. For QQ queries,

mAP⁡=1Q∑q=1QAP⁡q.\operatorname{mAP} =\frac{1}{Q}\sum_{q=1}^{Q}\operatorname{AP}_q.

In information retrieval, each query normally supplies one ranked result list and one AP value. Equal query weighting means that a query with few relevant documents has the same weight as one with many.

In multiclass classification and multilabel evaluation, AP can be calculated separately for each class by treating that class as positive and the remaining classes as negative. Aggregation choices include:

  • Macro averaging: an unweighted mean of per-class AP values.
  • Support-weighted averaging: a mean weighted by each class’s number of positive examples.
  • Micro averaging: pooling individual class–example decisions before calculating AP.

Micro AP is not generally the arithmetic mean of class-level AP values. Pooling also makes relative score scales across classes consequential.

The term mAP is therefore incomplete without identifying what is averaged: queries, classes, overlap thresholds, or some combination of them.

Object detection and instance segmentation

In object detection, predictions include a category, a confidence score, and a predicted location, usually a bounding box. AP evaluates both confidence ordering and whether predicted locations correctly match annotated objects.

For each class and localization threshold, an evaluator typically:

  1. Matches predictions to ground-truth objects within each image under a specified rule.
  2. Labels matched predictions as true positives and unmatched predictions as false positives.
  3. Ranks predictions by confidence across the evaluation images.
  4. Constructs a precision–recall curve and calculates AP.

Ordinarily, each ground-truth object can match at most one prediction. Additional detections of the same object count as false positives, subject to benchmark-specific rules for ignored or crowd annotations.

Localization commonly uses intersection over union (IoU):

IoU⁡(A,B)=∣A∩B∣∣A∪B∣,\operatorname{IoU}(A,B) =\frac{|A\cap B|}{|A\cup B|},

where AA and BB are predicted and ground-truth regions.

AP50 evaluates matching at an IoU threshold of 0.50; AP75 uses 0.75. A stricter threshold requires more accurate localization.

The COCO detection protocol reports a principal AP measure averaged over ten IoU thresholds from 0.50 to 0.95 in increments of 0.05, as well as over evaluated categories. It uses 101 recall thresholds and imposes additional rules, including limits on detections per image. Its principal AP is therefore not directly comparable with a score computed only at IoU 0.50.

For instance segmentation, a similar procedure uses mask IoU rather than bounding-box IoU. Matching rules, object-size ranges, ignored annotations, and detection limits all affect the resulting score.

Interpretation and limitations

Ranking rather than calibration

For a fixed set of scored items, AP depends on score ordering rather than on the scores’ absolute numerical values. A strictly increasing transformation that preserves ordering and ties leaves AP unchanged.

Consequently, AP does not establish that scores are reliable probabilities. A system may rank examples well while being poorly calibrated. Probability-sensitive evaluation requires other measures, such as logarithmic loss or the Brier score.

Dependence on class prevalence

Precision depends on the proportion of positive examples in the evaluated population. AP values from datasets with different prevalence or sampling procedures are therefore not automatically comparable.

The positive-class prevalence is often shown as a no-skill reference level on precision–recall plots. However, the expected AP of a finite random ranking is not universally equal to prevalence; it depends on the finite-sample setting and the AP convention.

Compared with a receiver operating characteristic curve, precision–recall evaluation makes the burden of false positives more explicit when positives are rare. This is one reason AP is used in anomaly detection and other imbalanced tasks. It still does not encode application-specific costs of different errors.

Relevance judgments and dataset composition

Basic AP assumes binary relevance. It does not distinguish a highly relevant result from a marginally relevant one. Metrics such as normalized discounted cumulative gain can incorporate graded relevance.

Retrieval evaluations also commonly have incomplete relevance judgments. Treating unjudged documents as nonrelevant can disadvantage systems that retrieve relevant documents outside the judged pool. AP must therefore be interpreted alongside the judgment procedure and the composition of the test set.

Cutoffs and operating points

AP summarizes performance across recall levels rather than at one deployment threshold. Systems with similar AP can have different precision at the recall level of practical interest.

Truncated measures such as AP at KK introduce another convention: some divide by the total number of relevant items, while others divide by min⁡(R,K)\min(R,K). These definitions can yield different values for the same top-KK list.

A reproducible AP result specifies the relevance or matching rule, evaluation population, interpolation method, tie handling, cutoff policy, treatment of cases with no positives, and averaging procedure.