scoringfunctions-package: Overview of the functions in the scoringfunctions package

In scoringfunctions: Loss Functions for Assessing Point Forecasts

Overview of the functions in the scoringfunctions package

Description

The scoringfunctions package implements consistent scoring (loss) functions and identification functions.

Details

The table below lists a selection of predictive functionals alongside their pointwise scoring functions (_sf) / realised average scores (_rs), and identification functions (_if). The complete listing of the package functions is given in the numbered sections below the table:

Target Functional	Loss Functions (_sf / _rs)	Identification (_if)
Mean	serr_sf / mse, bregman1_sf / bregman1_rs	mean_if
	bregman2_sf / bregman2_rs, bregman3_sf / qlike
	bregman4_sf / bregman4_rs
Expectile (p)	expectile_sf / expectile_rs	expectile_if
Median	aerr_sf / mae, maelog_sf / maelog_rs	quantile_if (p=0.5)
	maesd_sf / maesd_rs
\beta-Median	bmedian_sf / bmedian_rs	---
Quantile (p)	quantile_sf / quantile_rs, gpl1_sf / gpl1_rs	quantile_if
	gpl2_sf / gpl2_rs
Huber Mean	huber_sf / huber_rs	hubermean_if
Huber Quantile	ghuber_sf / ghuber_rs	huberquantile_if
L_q-Mean	lqmean_sf / lqmean_rs	---
L_q-Quantile	lqquantile_sf / lqquantile_rs	---
Interval (p)	interval_sf	---
Mean - Variance	mv_sf	mv_if
Error - Spread	errorspread_sf	---
Relative Error	relerr_sf / mre	---
Percentage Error	aperr_sf / mape	---

The package functions are categorised into six classes, each of which has its own section below:

1. Scoring functions

2. Realised (average) score functions

3. Skill score functions

4. Identification functions

5. Functions for sample levels

6. Supporting functions

1. Scoring functions

1.1. Consistent scoring functions for one-dimensional functionals

1.1.1. Consistent scoring functions for the mean

bregman1_sf: Bregman scoring function (type 1)

bregman2_sf: Bregman scoring function (type 2, Patton scoring function)

bregman3_sf: Bregman scoring function (type 3, QLIKE scoring function)

bregman4_sf: Bregman scoring function (type 4, Patton scoring function)

serr_sf: Squared error scoring function

1.1.2. Consistent scoring functions for expectiles

expectile_sf: Asymmetric piecewise quadratic scoring function (expectile scoring function, expectile loss function)

1.1.3. Consistent scoring functions for the median

aerr_sf: Absolute error scoring function

maelog_sf: MAE-LOG scoring function

maesd_sf: MAE-SD scoring function

1.1.4. Consistent scoring functions for quantiles

gpl1_sf: Generalized piecewise linear power scoring function (type 1)

gpl2_sf: Generalized piecewise linear power scoring function (type 2)

quantile_sf: Asymmetric piecewise linear scoring function (quantile scoring function, quantile loss function)

1.1.5. Consistent scoring functions for Huber functionals

ghuber_sf: Generalized Huber scoring function

huber_sf: Huber scoring function

1.1.6. Consistent scoring functions for other functionals

aperr_sf: Absolute percentage error scoring function

bmedian_sf: \beta-median scoring function

linex_sf: LINEX scoring function

lqmean_sf: L_q-mean scoring function

lqquantile_sf: L_q-quantile scoring function

nmoment_sf: n-th moment scoring function

obsweighted_sf: Observation-weighted scoring function

powerweighted_sf: Power-weighted squared error scoring function

relerr_sf: Relative error scoring function (MAE-PROP scoring function)

serrexp_sf: Squared error exp scoring function

serrlog_sf: Squared error log scoring function

serrpower_sf: Squared error of power transformations scoring function

serrsq_sf: Squared error of squares scoring function

sperr_sf: Squared percentage error scoring function

srelerr_sf: Squared relative error scoring function

1.2. Consistent scoring functions for two-dimensional functionals

interval_sf: Interval scoring function (Winkler scoring function)

mv_sf: Mean - variance scoring function

1.3. Consistent scoring functions for multi-dimensional functionals

errorspread_sf: Error - spread scoring function

2. Realised (average) score functions

2.1. Realised (average) score functions for one-dimensional functionals

2.1.1. Realised (average) score functions for the mean

bregman1_rs: Realised Bregman score (type 1)

bregman2_rs: Realised Bregman score (type 2, Patton score)

bregman4_rs: Realised Bregman score (type 4, Patton score)

mse: Mean squared error (MSE)

qlike: QLIKE

2.1.2. Realised (average) score functions for expectiles

expectile_rs: Realised expectile score

2.1.3. Realised (average) score functions for the median

mae: Mean absolute error (MAE)

maelog_rs: Realised MAE-LOG score

maesd_rs: Realised MAE-SD score

2.1.4. Realised (average) score functions for quantiles

gpl1_rs: Realised generalized piecewise linear power score (type 1)

gpl2_rs: Realised generalized piecewise linear power score (type 2)

quantile_rs: Realised quantile score

2.1.5. Realised (average) score functions for Huber functionals

ghuber_rs: Realised generalized Huber score

huber_rs: Realised Huber score

2.1.6. Realised (average) score functions for other functionals

bmedian_rs: Realised \beta-median score

linex_rs: Realised LINEX score

lqmean_rs: Realised L_q-mean score

lqquantile_rs: Realised L_q-quantile score

mape: Mean absolute percentage error (MAPE)

mre: Mean relative error (MRE)

mspe: Mean squared percentage error (MSPE)

msre: Mean squared relative error (MSRE)

nmoment_rs: Realised n-th moment score

obsweighted_rs: Realised observation-weighted score

serrexp_rs: Realised squared error exp score

serrlog_rs: Realised squared error log score

serrpower_rs: Realised squared error of power transformations score

serrsq_rs: Realised squared error of squares score

3. Skill score functions

3.1. Skill score functions for one-dimensional functionals

3.1.1. Skill score functions for the mean

nse: Nash-Sutcliffe efficiency (NSE)

4. Identification functions

4.1. Identification functions for one-dimensional functionals

expectile_if: Expectile identification function

hubermean_if: Huber mean identification function

huberquantile_if: Huber quantile identification function

mean_if: Mean identification function

meanexp_if: Exp-transformed identification function

meanlog_if: Log-transformed identification function

meanpower_if: Power-transformed identification function

nmoment_if: n-th moment identification function

powerweighted_if: Power-weighted identification function

quantile_if: Quantile identification function

4.2. Identification functions for two-dimensional functionals

mv_if: Mean - variance identification function

5. Functions for sample levels

quantile_level: Sample quantile level function

6. Supporting functions

capping_function: Capping function

References

Banerjee A, Guo X, Wang H (2005) On the optimality of conditional expectation as a Bregman predictor. IEEE Transactions on Information Theory 51(7):2664–2669. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1109/TIT.2005.850145")}.

Bellini F, Klar B, Muller A, Rosazza Gianin E (2014) Generalized quantiles as risk measures. Insurance: Mathematics and Economics 54:41–48. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1016/j.insmatheco.2013.10.015")}.

Brehmer JR, Gneiting T (2021) Scoring interval forecasts: Equal-tailed, shortest, and modal interval. Bernoulli 27(3):1993–2010. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.3150/20-BEJ1298")}.

Chen Z (1996) Conditional L_p-quantiles and their application to the testing of symmetry in non-parametric regression. Statistics and Probability Letters 29(2):107–115. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1016/0167-7152(95)00163-8")}.

Christensen HM, Moroz IM, Palmer TN (2015) Evaluation of ensemble forecast uncertainty using a new proper score: Application to medium-range and seasonal forecasts. Quarterly Journal of the Royal Meteorological Society 141(687)(Part B):538–549. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1002/qj.2375")}.

Dimitriadis T, Fissler T, Ziegel JF (2024) Osband's principle for identification functions. Statistical Papers 65:1125–1132. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1007/s00362-023-01428-x")}.

Dunsmore IR (1968) A Bayesian approach to calibration. Journal of the Royal Statistical Society, Series B (Methodological) 30(2):396–405. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1111/j.2517-6161.1968.tb00740.x")}.

Ferguson TS (1967) Mathematical Statistics: A Decision-Theoretic Approach. Academic Press, New York.

Fissler T, Pesenti SM (2023) Sensitivity measures based on scoring functions. European Journal of Operational Research 307(3):1408–1423. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1016/j.ejor.2022.10.002")}.

Fissler T, Ziegel JF (2016) Higher order elicitability and Osband's principle. The Annals of Statistics 44(4):1680–1707. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1214/16-AOS1439")}.

Fissler T, Ziegel JF (2019) Order-sensitivity and equivariance of scoring functions. Electronic Journal of Statistics 13(1):1166–1211. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1214/19-EJS1552")}.

Gerber HU (1974) On additive premium calculation principles. ASTIN Bulletin: The Journal of the IAA 7(3):215–222. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1017/S0515036100006061")}.

Gneiting T (2011) Making and evaluating point forecasts. Journal of the American Statistical Association 106(494):746–762. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1198/jasa.2011.r10138")}.

Gneiting T, Raftery AE (2007) Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association 102(477):359–378. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1198/016214506000001437")}.

Houghton-Carr HA (1999) Assessment criteria for simple conceptual daily rainfall-runoff models. Hydrological Sciences Journal 44(2):237–261. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1080/02626669909492220")}.

Huber PJ (1964) Robust estimation of a location parameter. Annals of Mathematical Statistics 35(1):73–101. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1214/aoms/1177703732")}.

Koenker R, Bassett Jr G (1978) Regression quantiles. Econometrica 46(1):33–50. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.2307/1913643")}.

Nash JE, Sutcliffe JV (1970) River flow forecasting through conceptual models Part I - A discussion of principles. Journal of Hydrology 10(3):282–290. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1016/0022-1694(70)90255-6")}.

Newey WK, Powell JL (1987) Asymmetric least squares estimation and testing. Econometrica 55(4):819–847. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.2307/1911031")}.

Park H, Stefanski LA (1998) Relative-error prediction. Statistics and Probability Letters 40(3):227–236. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1016/S0167-7152(98)00088-1")}.

Patton AJ (2011) Volatility forecast comparison using imperfect volatility proxies. Journal of Econometrics 160(1):246–256. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1016/j.jeconom.2010.03.034")}.

Raiffa H, Schlaifer R (1961) Applied Statistical Decision Theory. Colonial Press, Clinton.

Saerens M (2000) Building cost functions minimizing to some summary statistics. IEEE Transactions on Neural Networks 11(6):1263–1271. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1109/72.883416")}.

Savage LJ (1971) Elicitation of personal probabilities and expectations. Journal of the American Statistical Association 66(336):783–801. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1080/01621459.1971.10482346")}.

Taggart RJ (2022) Point forecasting and forecast evaluation with generalized Huber loss. Electronic Journal of Statistics 16(1):201–231. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1214/21-EJS1957")}.

Thirel G, Santos L, Delaigue O, Perrin C (2024) On the use of streamflow transformations for hydrological model calibration. Hydrology and Earth System Sciences 28(21):4837–4860. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.5194/hess-28-4837-2024")}.

Thomson W (1979) Eliciting production possibilities from a well-informed manager. Journal of Economic Theory 20(3):360–380. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1016/0022-0531(79)90042-5")}.

Tyralis H, Papacharalampous G (2026) Variable transformations in consistent loss functions. Knowledge-Based Systems 336:115202. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1016/j.knosys.2025.115202")}.

Varian HR (1975) A Bayesian approach to real estate assessment. In: Fienberg SE, Zellner A (eds) Studies in Bayesian Econometrics and Statistics in Honor of Leonard J. Savage. Amsterdam: North-Holland, pp 195–208.

Winkler RL (1972) A decision-theoretic approach to interval estimation. Journal of the American Statistical Association 67(337):187–191. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1080/01621459.1972.10481224")}.

Winkler RL, Murphy AH (1979) The use of probabilities in forecasts of maximum and minimum temperatures. Meteorological Magazine 108(1288):317–329.

Yeh C-C, Yeh H-W, Chan W (2008) Some equivalent forms of the arithematic-geometric mean inequality in probability: A survey. Journal of Inequalities and Applications 2008:386715. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1155/2008/386715")}.

Zellner A (1986) Bayesian estimation and prediction using asymmetric loss functions. Journal of the American Statistical Association 81(394):446–451. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1080/01621459.1986.10478289")}.

scoringfunctions documentation built on Aug. 30, 2026, 5:07 p.m.