scores_quantiles: Quantile and interval scores

In FK83/scoringRules: Scoring Rules for Parametric and Simulated Distribution Forecasts

Quantile and interval scores

Description

Compute quantile and interval scores, for given quantile predictions

Usage

qs_quantiles(y, x, alpha)

ints_quantiles(y, x_lower, x_upper, target_coverage)

qs_sample(y, dat, alpha, w = NULL, type = 7, show_messages = TRUE)

ints_sample(y, dat, target_coverage, w = NULL, type = 7, show_messages = TRUE)

Arguments

y	vector of observations
x	vector of quantile predictions
alpha	quantile level of interest
x_lower, x_upper	vector of quantile predictions (lower and upper endpoints of prediction intervals)
target_coverage	target (i.e., nominal) coverage level of prediction interval
dat	vector or matrix (depending on y; see details) of simulation draws from forecast distribution.
w	vector of observation weights (currently ignored)
type	integer between 1 and 9 that is passed on to stats function quantile (specifies algorithm for empirical quantile estimation; defaults to 7)
show_messages	logical; display of messages (does not affect warnings and errors).

Details

For a vector y of length n, dat should be given as a matrix with n rows. If y has length 1, then dat may be a vector.

Value

A vector of score values. Smaller values indicate better forecasts. Note that the interval score refers to the central prediction interval at level target_coverage.

References

Quantile score: Koenker, R. and G. Bassett (1978): ‘Regression quantiles’, Econometrica 46, 33-50. \Sexpr[results=rd]{tools:::Rd_expr_doi("https://doi.org/10.2307/1913643")} Interval score: Gneiting, T. and A.E. Raftery (2007): ‘Strictly proper scoring rules, prediction and estimation’, Journal of the American Statistical Association 102, 359-378. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1198/016214506000001437")}

See Also

The syntax of qs_sample and ints_sample is analogous to the functions documented on scores_sample_univ.

Examples

# Interval score is proportional to sum of two quantile scores
target_coverage <- .8
# corresponding quantile levels
alpha_1 <- .5*(1-target_coverage)
alpha_2 <- 1-.5*(1-target_coverage)
# compute intervalsscore
ints_quantiles(y = 1, x_lower = qnorm(alpha_1), 
x_upper = qnorm(alpha_2), target_coverage = target_coverage)
# compute sum of quantile scores (scaled by 2/(1-target_coverage))
(2/(1-target_coverage))*(qs_quantiles(y = 1, x = qnorm(alpha_1), alpha = alpha_1) + 
qs_quantiles(y = 1, x = qnorm(alpha_2), alpha = alpha_2))
# Compare exact to simulated quantile forecast from standard normal distribution
qs_quantiles(y = 1, x = qnorm(.1), alpha = .1)
qs_sample(y = 1, dat = rnorm(500), alpha = .1)

FK83/scoringRules documentation built on Feb. 20, 2024, 8:01 p.m.