crps.numeric: Continuous Ranked Probability Score for Parametric Forecast...

In scoringRules: Scoring Rules for Parametric and Simulated Distribution Forecasts

Continuous Ranked Probability Score for Parametric Forecast Distributions

Description

Calculate the Continuous Ranked Probability Score (CRPS) given observations and parameters of a family of distributions.

Usage

## S3 method for class 'numeric'
crps(y, family, ...)

Arguments

y	vector of realized values.
family	string which specifies the parametric family; current options: "2pexp", "2pnorm", "beta", "binom", "clogis", "cnorm", "ct", "exp", "expM", "exponential", "gamma", "gev", "gpd", "gtclogis", "gtcnorm", "gtct", "hyper", "lapl", "laplace", "llapl", "llogis", "lnorm", "log-laplace", "log-logistic", "log-normal", "logis", "logistic", "mixnorm", "mixture-normal", "nbinom", "negative-binomial", "norm", "normal", "pois", "poisson", "t", "tlogis", "tnorm", "tt", "two-piece-exponential", "two-piece-normal", "unif", "uniform".
...	vectors of parameter values; expected input depends on the chosen family. See details below.

Details

Mathematical details are available in Appendix A of the vignette Evaluating probabilistic forecasts with scoringRules that accompanies the package.

The parameters supplied to each of the functions are numeric vectors:

1. Distributions defined on the real line:

   • "laplace" or "lapl": location (real-valued location parameter), scale (positive scale parameter); see crps_lapl

   • "logistic" or "logis": location (real-valued location parameter), scale (positive scale parameter); see crps_logis

   • "normal" or "norm": mean, sd (mean and standard deviation); see crps_norm

   • "normal-mixture" or "mixture-normal" or "mixnorm": m (mean parameters), s (standard deviations), w (weights); see crps_mixnorm; note: matrix-input for parameters

   • "t": df (degrees of freedom), location (real-valued location parameter), scale (positive scale parameter); see crps_t

   • "two-piece-exponential" or "2pexp": location (real-valued location parameter), scale1, scale2 (positive scale parameters); see crps_2pexp

   • "two-piece-normal" or "2pnorm": location (real-valued location parameter), scale1, scale2 (positive scale parameters); see crps_2pnorm

2. Distributions for non-negative random variables:

   • "exponential" or "exp": rate (positive rate parameter); see crps_exp

   • "gamma": shape (positive shape parameter), rate (positive rate parameter), scale (alternative to rate); see crps_gamma

   • "log-laplace" or "llapl": locationlog (real-valued location parameter), scalelog (positive scale parameter); see crps_llapl

   • "log-logistic" or "llogis": locationlog (real-valued location parameter), scalelog (positive scale parameter); see crps_llogis

   • "log-normal" or "lnorm": locationlog (real-valued location parameter), scalelog (positive scale parameter); see crps_lnorm

3. Distributions with flexible support and/or point masses:

   • "beta": shape1, shape2 (positive shape parameters), lower, upper (lower and upper limits); see crps_beta

   • "uniform" or "unif": min, max (lower and upper limits), lmass, umass (point mass in lower or upper limit); see crps_unif

   • "expM": location (real-valued location parameter), scale (positive scale parameter), mass (point mass in location); see crps_expM

   • "gev": location (real-valued location parameter), scale (positive scale parameter), shape (real-valued shape parameter); see crps_gev

   • "gpd": location (real-valued location parameter), scale (positive scale parameter), shape (real-valued shape parameter), mass (point mass in location); see crps_gpd

   • "tlogis": location (location parameter), scale (scale parameter), lower, upper (lower and upper limits); see crps_tlogis

   • "clogis": location (location parameter), scale (scale parameter), lower, upper (lower and upper limits); see crps_clogis

   • "gtclogis": location (location parameter), scale (scale parameter), lower, upper (lower and upper limits); lmass, umass (point mass in lower or upper limit); see crps_gtclogis

   • "tnorm": location (location parameter), scale (scale parameter), lower, upper (lower and upper limits); see crps_tnorm

   • "cnorm": location (location parameter), scale (scale parameter), lower, upper (lower and upper limits); see crps_cnorm

   • "gtcnorm": location (location parameter), scale (scale parameter), lower, upper (lower and upper limits); lmass, umass (point mass in lower or upper limit); see crps_gtcnorm

   • "tt": df (degrees of freedom), location (location parameter), scale (scale parameter), lower, upper (lower and upper limits); see crps_tt

   • "ct": df (degrees of freedom), location (location parameter), scale (scale parameter), lower, upper (lower and upper limits); see crps_ct

   • "gtct": df (degrees of freedom), location (location parameter), scale (scale parameter), lower, upper (lower and upper limits); lmass, umass (point mass in lower or upper limit); see crps_gtct

4. Distributions of discrete variables:

   • "binom": size (number of trials (zero or more)), prob (probability of success on each trial); see crps_binom

   • "hyper": m (the number of white balls in the urn), n (the number of black balls in the urn), k (the number of balls drawn from the urn); see crps_hyper

   • "negative-binomial" or "nbinom": size (positive dispersion parameter), prob (success probability), mu (mean, alternative to prob); see crps_nbinom

   • "poisson" or "pois": lambda (positive mean); see crps_pois

All numerical arguments should be of the same length. An exception are scalars of length 1, which will be recycled.

Value

Vector of score values. A lower score indicates a better forecast.

Author(s)

Alexander Jordan, Fabian Krueger, Sebastian Lerch

References

Closed form expressions of the CRPS for specific distributions:

Baran, S. and S. Lerch (2015): 'Log-normal distribution based Ensemble Model Output Statistics models for probabilistic wind-speed forecasting', Quarterly Journal of the Royal Meteorological Society 141, 2289-2299. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1002/qj.2521")} (Log-normal)

Friederichs, P. and T.L. Thorarinsdottir (2012): 'Forecast verification for extreme value distributions with an application to probabilistic peak wind prediction', Environmetrics 23, 579-594. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1002/env.2176")} (Generalized Extreme Value, Generalized Pareto)

Gneiting, T., Larson, K., Westvelt III, A.H. and T. Goldman (2005): 'Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation', Monthly Weather Review 133, 1098-1118. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1175/mwr2904.1")} (Normal)

Gneiting, T., Larson, K., Westrick, K., Genton, M.G. and E. Aldrich (2006): 'Calibrated probabilistic forecasting at the stateline wind energy center: The regime-switching space-time method', Journal of the American Statistical Association 101, 968-979. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1198/016214506000000456")} (Censored normal)

Gneiting, T. and T.L. Thorarinsdottir (2010): ‘Predicting inflation: Professional experts versus no-change forecasts’, arXiv preprint arXiv:1010.2318. (Two-piece normal)

Grimit, E.P., Gneiting, T., Berrocal, V.J. and N.A. Johnson (2006): 'The continuous ranked probability score for circular variables and its application to mesoscale forecast ensemble verification', Quarterly Journal of the Royal Meteorological Society 132, 2925-2942. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1256/qj.05.235")} (Mixture of normals)

Scheuerer, M. and D. Moeller (2015): 'Probabilistic wind speed forecasting on a grid based on ensemble model output statistics', Annals of Applied Statistics 9, 1328-1349. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1214/15-aoas843")} (Gamma)

Thorarinsdottir, T.L. and T. Gneiting (2010): 'Probabilistic forecasts of wind speed: ensemble model output statistics by using heteroscedastic censored regression', Journal of the Royal Statistical Society (Series A) 173, 371-388. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1111/j.1467-985x.2009.00616.x")} (Truncated normal)

Wei, W. and L. Held (2014): ‘Calibration tests for count data’, TEST 23, 787-205. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1007/s11749-014-0380-8")} (Poisson, Negative Binomial)

Independent listing of closed-form solutions for the CRPS:

Taillardat, M., Mestre, O., Zamo, M. and P. Naveau (2016): 'Calibrated ensemble forecasts using quantile regression forests and ensemble model output statistics', Monthly Weather Review 144, 2375-2393. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1175/mwr-d-15-0260.1")}

See Also

logs.numeric

Examples

crps(y = 1, family = "normal", mean = 0, sd = 2)
crps(y = rnorm(20), family = "normal", mean = 1:20, sd = sqrt(1:20))

## Arguments can have different lengths:
crps(y = rnorm(20), family = "normal", mean = 0, sd = 2)
crps(y = 1, family = "normal", mean = 1:20, sd = sqrt(1:20))

## Mixture of normal distributions requires matrix input for parameters:
mval <- matrix(rnorm(20*50), nrow = 20)
sdval <- matrix(runif(20*50, min = 0, max = 2), nrow = 20)
weights <- matrix(rep(1/50, 20*50), nrow = 20)
crps(y = rnorm(20), family = "mixnorm", m = mval, s = sdval, w = weights)

scoringRules documentation built on Sept. 18, 2024, 5:09 p.m.