{
  "id": "continuous-ranked-probability-score",
  "title": "Continuous Ranked Probability Score",
  "lang": "en",
  "source_lang": "en",
  "summary": "The continuous ranked probability score evaluates univariate probabilistic forecasts by comparing a predicted cumulative distribution with an observed outcome; lower scores indicate better performance.",
  "category": "mathematics",
  "subcategory": "probability-statistics",
  "type": "concept",
  "aliases": [
    "CRPS",
    "Continuous ranked probability scoring rule"
  ],
  "generated_at": "2026-11-05",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/continuous-ranked-probability-score/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/continuous-ranked-probability-score/",
      "json": "https://aiwiki.page/zh/continuous-ranked-probability-score.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "The continuous ranked probability score (CRPS) is a proper scoring rule for evaluating forecasts expressed as a probability distribution over a real-valued outcome. It measures the discrepancy between the forecast’s cumulative distribution and the cumulative distribution corresponding to the observed value. Lower scores are better. Unlike a score for a point prediction, CRPS assesses the entire predictive distribution, including its location and spread. It is widely used in weather forecasting and in other applications involving probabilistic predictions of continuous or ordered numerical quantities."
    },
    {
      "heading": "Definition",
      "text": "Let \\(F\\) be the forecast cumulative distribution function and let \\(y\\) be the observed outcome. The CRPS is\n\n\\[\\operatorname{CRPS}(F,y) = \\int_{-\\infty}^{\\infty} \\left(F(z)-\\mathbf{1}\\{y\\le z\\}\\right)^2\\,dz,\\]\n\nwhere \\(\\mathbf{1}\\{\\cdot\\}\\) is the indicator function.\n\nThe function \\(\\mathbf{1}\\{y\\le z\\}\\), considered as a function of \\(z\\), is the cumulative distribution function of a distribution concentrated entirely at \\(y\\). Thus, CRPS is the integrated squared difference between the forecast CDF and the CDF of the realized observation.\n\nAn equivalent expression is\n\n\\[\\operatorname{CRPS}(F,y) = \\int_{-\\infty}^{y}F(z)^2\\,dz + \\int_y^\\infty [1-F(z)]^2\\,dz.\\]\n\nThis form shows how the score penalizes forecast probability below an observation that turns out to be larger, and forecast probability above an observation that turns out to be smaller.\n\nThe score is nonnegative. It is zero precisely when the forecast puts all its probability at the observed value. A finite first absolute moment is a standard sufficient condition for finiteness and for the expectation-based formulas below.\n\nDespite its name, CRPS does not require the forecast distribution to be continuous: it also applies to discrete and mixed distributions on the real line."
    },
    {
      "heading": "Equivalent representations",
      "text": "Expected absolute distances\n\nLet \\(X\\) and \\(X'\\) be independent random variables with distribution \\(F\\). If \\(\\mathbb{E}|X|<\\infty\\), then\n\n\\[\\operatorname{CRPS}(F,y) = \\mathbb{E}|X-y| - \\frac{1}{2}\\mathbb{E}|X-X'|.\\]\n\nHere \\(\\mathbb{E}\\) denotes expectation. The first term measures the average absolute distance between a forecast draw and the observation. The second adjusts for the spread of the forecast distribution.\n\nThis adjustment distinguishes CRPS from simply averaging the absolute errors of forecast draws. Without it, minimizing the expected score would favor a point mass at a median of the outcome distribution rather than an honest representation of uncertainty.\n\nIntegrated threshold scores\n\nFor any threshold \\(z\\), the forecast probability of the event \\(Y\\le z\\) is \\(F(z)\\), and its observed outcome is \\(\\mathbf{1}\\{y\\le z\\}\\). Therefore,\n\n\\[\\left(F(z)-\\mathbf{1}\\{y\\le z\\}\\right)^2\\]\n\nis the Brier score for that binary event. CRPS integrates these binary scores over all thresholds.\n\nThis interpretation connects the evaluation of a numerical outcome to the evaluation of all threshold-exceedance or threshold-nonexceedance events associated with it.\n\nIntegrated quantile loss\n\nLet \\(F^{-1}(\\tau)\\) denote the generalized quantile function, and define the quantile loss, or pinball loss, by\n\n\\[\\rho_\\tau(u) = u\\left(\\tau-\\mathbf{1}\\{u<0\\}\\right).\\]\n\nThen\n\n\\[\\operatorname{CRPS}(F,y) = 2\\int_0^1 \\rho_\\tau\\!\\left(y-F^{-1}(\\tau)\\right)\\,d\\tau.\\]\n\nCRPS can therefore be viewed either as an integral over thresholds or as an integral over quantile levels. The latter representation is useful when a forecasting model produces quantiles rather than a density or CDF."
    },
    {
      "heading": "Propriety and interpretation",
      "text": "A scoring rule is proper if reporting the true outcome distribution minimizes the expected score. It is strictly proper if that distribution is the unique minimizer.\n\nSuppose the true distribution is \\(G\\), and \\(F\\) and \\(G\\) have finite first absolute moments. Then\n\n\\[\\mathbb{E}_{Y\\sim G}\\operatorname{CRPS}(F,Y) - \\mathbb{E}_{Y\\sim G}\\operatorname{CRPS}(G,Y) = \\int_{-\\infty}^{\\infty}[F(z)-G(z)]^2\\,dz.\\]\n\nThe right-hand side is nonnegative and vanishes only when \\(F=G\\). Hence CRPS is strictly proper on this class of distributions.\n\nThe expected score can also be written as\n\n\\[\\mathbb{E}_{Y\\sim G}\\operatorname{CRPS}(F,Y) = \\int_{-\\infty}^{\\infty}[F(z)-G(z)]^2\\,dz + \\int_{-\\infty}^{\\infty}G(z)[1-G(z)]\\,dz.\\]\n\nThe second term represents uncertainty inherent in the outcome distribution; it does not depend on the forecast. Even a correct, nondegenerate forecast therefore has a positive expected score.\n\nStrict propriety is a statement about expected performance across outcomes. An incorrect forecast can outperform a correct one for a particular observation. Assessment consequently usually averages scores over many forecast–observation pairs.\n\nRelation to point-forecast error\n\nIf the forecast assigns probability one to a point \\(x\\), then\n\n\\[\\operatorname{CRPS}(F,y)=|x-y|.\\]\n\nCRPS thus extends absolute error from point forecasts to distributional forecasts.\n\nIt has the same physical units as the predicted quantity. For a transformed variable \\(aX+b\\),\n\n\\[\\operatorname{CRPS}(F_{aX+b},ay+b) = |a|\\operatorname{CRPS}(F_X,y).\\]\n\nRaw scores are therefore not directly comparable across quantities measured on different scales."
    },
    {
      "heading": "Computation",
      "text": "Finite ensembles\n\nFor an equally weighted ensemble \\(x_1,\\ldots,x_m\\), interpreted as an empirical predictive distribution,\n\n\\[\\operatorname{CRPS}(F_m,y) = \\frac{1}{m}\\sum_{i=1}^{m}|x_i-y| - \\frac{1}{2m^2} \\sum_{i=1}^{m}\\sum_{j=1}^{m}|x_i-x_j|.\\]\n\nThis expression is exact for the empirical distribution. Direct evaluation of the pairwise term requires quadratic work, but sorting the ensemble gives\n\n\\[\\operatorname{CRPS}(F_m,y) = \\frac{1}{m}\\sum_{i=1}^{m}|x_i-y| - \\frac{1}{m^2} \\sum_{i=1}^{m}(2i-m-1)x_{(i)},\\]\n\nwhere \\(x_{(i)}\\) are the sorted members. The resulting computation takes \\(O(m\\log m)\\) time.\n\nFor nonnegative weights \\(w_i\\) summing to one,\n\n\\[\\operatorname{CRPS}(F,y) = \\sum_i w_i|x_i-y| - \\frac{1}{2}\\sum_{i,j}w_iw_j|x_i-x_j|.\\]\n\nA distinction arises when ensemble members are independent draws used to approximate an underlying distribution rather than the forecast distribution being evaluated. In that case, an unbiased estimator of the underlying CRPS is\n\n\\[\\widehat{\\operatorname{CRPS}}_{\\mathrm{fair}} = \\frac{1}{m}\\sum_i|x_i-y| - \\frac{1}{2m(m-1)} \\sum_{i\\ne j}|x_i-x_j|, \\qquad m>1.\\]\n\nThis correction requires appropriate sampling assumptions; it is not automatically suitable for dependent or otherwise structured ensembles.\n\nNormal forecasts\n\nFor a normal distribution with mean \\(\\mu\\) and standard deviation \\(\\sigma>0\\), define\n\n\\[a=\\frac{y-\\mu}{\\sigma}.\\]\n\nIf \\(\\Phi\\) and \\(\\phi\\) are the standard normal CDF and probability density function, respectively, then\n\n\\[\\operatorname{CRPS}(F,y) = \\sigma\\left[ a\\bigl(2\\Phi(a)-1\\bigr) + 2\\phi(a) - \\frac{1}{\\sqrt{\\pi}} \\right].\\]\n\nClosed-form expressions are also available for various other distribution families. Otherwise, the defining integral, quantile representation, or sample representation can be evaluated numerically."
    },
    {
      "heading": "Applications",
      "text": "CRPS is used to compare probabilistic weather forecasts, hydrological forecasts, economic predictions, and time-series models. It can also assess a posterior predictive distribution in Bayesian modeling.\n\nFor forecast–observation pairs \\((F_t,y_t)\\), performance is commonly summarized by\n\n\\[\\overline{\\operatorname{CRPS}} = \\frac{1}{n}\\sum_{t=1}^{n} \\operatorname{CRPS}(F_t,y_t).\\]\n\nA related continuous ranked probability skill score compares a model with a reference forecast:\n\n\\[\\operatorname{CRPSS} = 1- \\frac{\\overline{\\operatorname{CRPS}}_{\\mathrm{model}}} {\\overline{\\operatorname{CRPS}}_{\\mathrm{reference}}},\\]\n\nprovided the denominator is positive. Positive values indicate improvement over the reference; zero indicates equal mean scores; negative values indicate worse performance.\n\nCRPS can also serve as a loss function for estimating predictive-distribution parameters. Minimizing empirical CRPS is an alternative to maximum likelihood estimation. Their objectives differ, so the resulting parameter estimates need not coincide."
    },
    {
      "heading": "Extensions and limitations",
      "text": "Threshold weighting\n\nA threshold-weighted version is\n\n\\[\\operatorname{twCRPS}(F,y) = \\int_{-\\infty}^{\\infty} w(z)\\left(F(z)-\\mathbf{1}\\{y\\le z\\}\\right)^2\\,dz,\\]\n\nwhere \\(w(z)\\ge0\\) is fixed independently of the realized observation. It can emphasize thresholds associated with unusually high or low outcomes.\n\nThis construction remains proper under suitable integrability conditions. If the weight vanishes over a region, however, differences between distributions there may go undetected, so strict propriety for the entire distribution can be lost. Simply weighting ordinary scores according to the observed outcome does not generally preserve propriety.\n\nMultivariate outcomes\n\nOrdinary CRPS concerns a single real-valued variable. Scoring several variables separately assesses their marginal distributions but does not fully assess their dependence.\n\nThe energy score provides a multivariate extension by replacing absolute distances with Euclidean distances:\n\n\\[\\operatorname{ES}(F,\\mathbf y) = \\mathbb{E}\\|\\mathbf X-\\mathbf y\\| - \\frac{1}{2}\\mathbb{E}\\|\\mathbf X-\\mathbf X'\\|.\\]\n\nIn one dimension, this is CRPS. In higher dimensions, sensitivity to dependence errors can be limited, making complementary assessments useful.\n\nDiagnostic limitations\n\nA single mean CRPS does not reveal whether forecast errors primarily concern location, dispersion, or tail behavior. Distinct forecasting systems can obtain similar scores while making different kinds of errors.\n\nCRPS should also be distinguished from a standalone test of calibration. Its strict propriety rewards correct distributions in expectation, but a finite-sample score does not by itself establish that predicted probabilities agree with observed frequencies.\n\nBecause ordinary CRPS integrates discrepancies uniformly over thresholds, performance for rare extremes can be obscured by performance elsewhere. Moreover, aggregate scores depend on the evaluation cases: changes in forecast horizon, outcome variability, or case mix can change average CRPS without reflecting the same change in forecasting ability. Out-of-sample evaluation, including appropriately designed cross-validation, separates predictive assessment from in-sample fitting."
    }
  ],
  "links": [
    "proper-scoring-rule",
    "probability-distribution",
    "cumulative-distribution-function",
    "indicator-function",
    "statistical-independence",
    "random-variable",
    "expected-value",
    "median",
    "brier-score",
    "quantile",
    "normal-distribution",
    "probability-density-function",
    "time-series",
    "posterior-predictive-distribution",
    "loss-function",
    "maximum-likelihood-estimation",
    "euclidean-distance",
    "cross-validation"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}