scores_sample_univ: Scoring Rules for Simulated Forecast Distributions

In scoringRules: Scoring Rules for Parametric and Simulated Distribution Forecasts

Scoring Rules for Simulated Forecast Distributions

Description

Calculate scores (CRPS, LogS, DSS) given observations and draws from the predictive distributions.

Usage

crps_sample(
  y,
  dat,
  method = "edf",
  w = NULL,
  bw = NULL,
  num_int = FALSE,
  show_messages = TRUE
)

logs_sample(y, dat, bw = NULL, show_messages = FALSE)

dss_sample(y, dat, w = NULL)

Arguments

y	vector of realized values.
dat	vector or matrix (depending on y; see details) of simulation draws from forecast distribution.
method	string; approximation method. Options: "edf" (empirical distribution function) and "kde" (kernel density estimation).
w	optional; vector or matrix (matching dat) of weights for method "edf".
bw	optional; vector (matching y) of bandwidths for kernel density estimation; see details.
num_int	logical; if TRUE numerical integration is used for method "kde".
show_messages	logical; display of messages (does not affect warnings and errors).

Details

For a vector y of length n >= 2, dat should be given as a matrix with n rows. If y has length 1, then dat may be a vector.

crps_sample employs an empirical version of the quantile decomposition of the CRPS (Laio and Tamea, 2007) when using method = "edf". For method = "kde", it uses kernel density estimation using a Gaussian kernel. The logarithmic score always uses kernel density estimation.

The bandwidth (bw) for kernel density estimation can be specified manually, in which case it must be a vector (matching y) of positive numbers. If bw == NULL, the bandwidth is selected using the core function bw.nrd. Numerical integration may speed up computation for crps_sample in case of large samples dat.

Value

Value of the score. A lower score indicates a better forecast.

Author(s)

Alexander Jordan, Fabian Krueger, Sebastian Lerch

References

Evaluating simulation based forecast distributions:

Krueger, F., Lerch, S., Thorarinsdottir, T.L. and T. Gneiting (2021): ‘Predictive inference based on Markov chain Monte Carlo output’, International Statistical Review 89, 274-301. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1111/insr.12405")}

Empirical quantile decomposition of the CRPS:

Laio, F. and S. Tamea (2007): 'Verification tools for probabilistic forecasts of continuous hydrological variables', Hydrology and Earth System Sciences, 11, 1267-1277. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.5194/hess-11-1267-2007")}

See Also

scores_sample_univ_weighted for weighted versions of the scoring rules documented here.

Examples

## Not run: 

# y has length greater than 1
y <- 1:2
sample <- matrix(rnorm(20), nrow = 2)
crps_sample(y = y, dat = sample)
logs_sample(y = y, dat = sample)

y <- 1:2
sample <- rnorm(10)
crps_sample(y = y, dat = sample) # error

# y has length 1
y <- 1
sample <- rnorm(10)
crps_sample(y = y, dat = sample)

sample <- matrix(rnorm(10), nrow = 1)
crps_sample(y = y, dat = sample)

sample <- matrix(rnorm(20), nrow = 2)
crps_sample(y = y, dat = sample) # error

## End(Not run)

scoringRules documentation built on Sept. 18, 2024, 5:09 p.m.