{
  "id": "normalized-discounted-cumulative-gain",
  "title": "Normalized Discounted Cumulative Gain",
  "lang": "en",
  "source_lang": "en",
  "summary": "Normalized discounted cumulative gain is a ranking metric that measures graded relevance while giving greater weight to items near the top of a result list.",
  "category": "computer-science",
  "subcategory": "artificial-intelligence",
  "type": "concept",
  "aliases": [
    "NDCG",
    "nDCG",
    "NDCG@k",
    "Normalised Discounted Cumulative Gain"
  ],
  "generated_at": "2026-11-05",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/normalized-discounted-cumulative-gain/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/normalized-discounted-cumulative-gain/",
      "json": "https://aiwiki.page/zh/normalized-discounted-cumulative-gain.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Normalized discounted cumulative gain (NDCG) is a measure of ranking quality used in information retrieval, recommender systems, and learning to rank. It evaluates an ordered list by assigning gains to relevant items, discounting gains at lower positions, and dividing the resulting score by the score of an ideal ranking. Unlike measures that distinguish only relevant from irrelevant results, NDCG can use graded relevance, such as “irrelevant,” “partially relevant,” and “highly relevant.” It is commonly reported at a fixed rank cutoff, written NDCG@\\(k\\)."
    },
    {
      "heading": "Definition",
      "text": "NDCG combines three ideas:\n\n- Gain: more relevant items contribute more value.\n- Discounting: an item contributes less when it appears farther down the ranking.\n- Normalization: the ranking is compared with the best possible ordering for the same evaluation context.\n\nLet \\(r_i\\) be the relevance grade of the item at rank \\(i\\), with ranks beginning at 1. A widely used definition of discounted cumulative gain at cutoff \\(k\\) is\n\n\\[\\operatorname{DCG}@k = \\sum_{i=1}^{k} \\frac{2^{r_i}-1}{\\log_2(i+1)}.\\]\n\nThe numerator \\(2^{r_i}-1\\) is the gain function. The denominator discounts results according to their positions. Rank 1 has discount weight 1, rank 2 has weight approximately \\(0.631\\), and rank 3 has weight \\(0.5\\).\n\nThe ideal discounted cumulative gain, \\(\\operatorname{IDCG}@k\\), is calculated by placing the available items in descending order of relevance and applying the same formula. Then\n\n\\[\\operatorname{NDCG}@k = \\frac{\\operatorname{DCG}@k} {\\operatorname{IDCG}@k}.\\]\n\nWith nonnegative gains, a positive denominator, and a consistently defined evaluation universe, NDCG lies between 0 and 1. A score of 1 indicates an ideal ordering at the evaluated cutoff, allowing for interchangeable items with equal relevance.\n\nGain and discount variants\n\nNDCG is a family of closely related conventions rather than a completely uniform formula. A general form is\n\n\\[\\operatorname{DCG}@k = \\sum_{i=1}^{k} g(r_i)d(i),\\]\n\nwhere \\(g\\) is an increasing gain function and \\(d\\) is a positive, nonincreasing position weight.\n\nTwo common gain functions are:\n\n- Linear gain: \\(g(r)=r\\).\n- Exponential gain: \\(g(r)=2^r-1\\).\n\nExponential gain places greater emphasis on distinguishing highly relevant items from moderately relevant ones. For binary grades \\(r\\in\\{0,1\\}\\), these two functions are identical.\n\nDiscount conventions also vary. An early formulation leaves the first result undiscounted and uses a logarithmic discount based on rank for subsequent results. Many modern implementations instead use \\(\\log_2(i+1)\\) uniformly. Scores from different conventions are not necessarily interchangeable.\n\nFor the uniform \\(\\log_b(i+1)\\) formulation, changing the logarithm base multiplies both DCG and IDCG by the same constant and therefore leaves NDCG unchanged."
    },
    {
      "heading": "Worked example",
      "text": "Suppose three items have relevance grades 3, 2, and 0, but a system returns them in the order 2, 3, 0. With exponential gains and cutoff 3,\n\n\\[\\operatorname{DCG}@3 = \\frac{2^2-1}{\\log_2 2} + \\frac{2^3-1}{\\log_2 3} + \\frac{2^0-1}{\\log_2 4} \\approx 7.4165.\\]\n\nThe ideal order is 3, 2, 0:\n\n\\[\\operatorname{IDCG}@3 = 7+\\frac{3}{\\log_2 3} \\approx 8.8928.\\]\n\nTherefore,\n\n\\[\\operatorname{NDCG}@3 \\approx \\frac{7.4165}{8.8928} \\approx 0.834.\\]\n\nThe ranking contains the same items as the ideal list, but receives a lower score because the most relevant item appears second. NDCG can thus distinguish orderings even when their sets of returned items are identical."
    },
    {
      "heading": "Relevance judgments and evaluation scope",
      "text": "Graded relevance\n\nRelevance grades may come from human assessments or from behavioral observations transformed into evaluation labels. In a search engine, assessors might judge whether a document fully answers a query, provides partial information, or is unrelated. Recommendation evaluations may use explicit ratings or labels derived from interactions.\n\nThe grade scale and gain function jointly determine the meaning of the metric. Exponential gain on grades from 0 to 4 assigns gains of 0, 1, 3, 7, and 15. These differences express a particular valuation of relevance levels; they are not automatically justified by the fact that the labels are ordered.\n\nBehavioral labels also require care. A click is not a direct measurement of relevance: it can depend on presentation, previous ranking position, and whether the user noticed the item.\n\nThe ideal ranking\n\nThe items eligible for the ideal ranking must be specified. In retrieval evaluation, IDCG may be computed from the judged relevant documents for a query, including documents absent from the system’s returned list. In reranking evaluation, it may instead be computed only from a fixed candidate set.\n\nThese choices measure different capabilities. Normalizing within a candidate set evaluates ordering within that set, but does not fully measure whether an earlier retrieval stage omitted valuable items. Allowing the ideal set to vary between systems can therefore make comparisons misleading.\n\nCutoffs and missing results\n\nSmall cutoffs emphasize the first few results; larger cutoffs evaluate deeper portions of the list. NDCG@5 and NDCG@100 can rank systems differently.\n\nA list shorter than \\(k\\) is commonly treated as having zero gain at missing positions. However, the ideal denominator should still follow the specified evaluation universe rather than automatically shrinking to the returned list.\n\nZero denominators and negative labels\n\nIf every eligible item has zero gain, then IDCG is zero and the ratio is mathematically undefined. Evaluation tools may assign zero, exclude the query, or use another explicit convention. This choice affects aggregate scores.\n\nThe usual bounded interpretation assumes nonnegative gains. Negative relevance labels, when passed through common gain functions, can produce negative gains and invalidate the standard interpretation of NDCG as a score between 0 and 1."
    },
    {
      "heading": "Aggregation and implementation",
      "text": "NDCG is usually calculated separately for each query or user and then averaged:\n\n\\[\\operatorname{MeanNDCG}@k = \\frac{1}{|Q|} \\sum_{q\\in Q} \\operatorname{NDCG}_q@k,\\]\n\nwhere \\(Q\\) is the set of evaluated queries or users after applying the chosen inclusion rules.\n\nThis macro-average gives each evaluation unit equal weight unless explicit weights are introduced. It differs from dividing the sum of all DCG values by the sum of all IDCG values, which implicitly gives greater weight to units with larger ideal gains.\n\nTied prediction scores also require a convention. An implementation may break ties deterministically or calculate the expected value over possible orderings within tied groups. Different treatments can change the result, particularly when a tie crosses the cutoff.\n\nMeaningful comparisons therefore specify the cutoff, gain and discount functions, candidate universe, relevance judgments, treatment of unjudged items, zero-denominator policy, tie handling, and aggregation method."
    },
    {
      "heading": "Uses in machine learning",
      "text": "NDCG is widely used as an offline evaluation metric for machine-learning ranking models. It is especially useful when results have multiple relevance levels and top-ranked mistakes matter more than mistakes near the bottom.\n\nBecause NDCG depends on discrete rank positions, it is not ordinarily a smooth function of prediction scores. Small changes in scores may leave the ranking unchanged, while a swap can cause an abrupt change in the metric. Ranking methods therefore often optimize a surrogate loss function rather than NDCG directly.\n\nSome methods weight training updates according to the change in NDCG that would result from exchanging two items. Such approaches connect model training to the evaluation objective without treating the metric as an ordinary differentiable loss.\n\nAs with other evaluation measures, repeatedly selecting models against the same test set can compromise its role as an independent assessment. A separate validation set is commonly used for model selection."
    },
    {
      "heading": "Strengths and limitations",
      "text": "NDCG accommodates graded relevance, emphasizes early results, and normalizes for differences in the amount of attainable gain across queries. However, normalization does not make every query equally difficult or every dataset directly comparable.\n\nIts main limitations include:\n\n- Dependence on judgments. Incomplete or inconsistent assessments can distort both the observed gain and the ideal denominator. Treating unjudged items as irrelevant may disadvantage systems that retrieve previously unassessed material.\n- Dependence on candidate construction. Scores calculated against sampled negative items may differ substantially from scores calculated against a full catalog.\n- Assumed position effects. Logarithmic discounting represents a particular model of how value decreases with rank, not a universal description of user attention.\n- Additive item value. Standard NDCG scores each item independently. It does not directly penalize redundant answers or reward coverage of different user intents.\n- Limited interpretation of differences. An increase from 0.70 to 0.75 does not mean that users are five percentage points more satisfied.\n- Limited coverage of system quality. NDCG alone does not measure latency, fairness, novelty, factual correctness, or other deployment requirements.\n\nCompared with average precision, which conventionally uses binary relevance, NDCG directly supports multiple relevance levels. Nevertheless, the appropriate metric depends on the intended task and the meaning of the available labels."
    },
    {
      "heading": "Historical development",
      "text": "The cumulative-gain framework was developed in information-retrieval research by Kalervo Järvelin and Jaana Kekäläinen around 2000 and elaborated in their 2002 work on cumulated gain-based evaluation. It addressed a limitation of evaluation methods that reduced relevance to a binary distinction despite the availability of graded judgments. Discounted and normalized cumulative-gain measures subsequently became established tools in search evaluation, recommendation, and learning-to-rank research."
    }
  ],
  "links": [
    "information-retrieval",
    "recommender-system",
    "learning-to-rank",
    "search-engine",
    "expected-value",
    "machine-learning",
    "loss-function",
    "test-set",
    "validation-set",
    "average-precision"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}