proper_score: Calculate proper scores

In finnlindgren/StatCompLab: Workshop Material For UoE Statistical Computing

Calculate proper scores

Description

Calculates proper scores for probabilistic predictions

Usage

proper_score(type, obs, ...)

Arguments

type	A string indicating the type of score to calculate. One of "se", "ae", "ds", or "interval". See Details below.
obs	A vector of observations
...	Additional named arguments needed for each type of score, see Details below.

Details

Each score type takes additional arguments that define the prediction model information to be used in the score calculations. Unless specified otherwise, the extra arguments should either be scalars (applying to all the predictions) or vectors of the same length as the obs vector.

se

Squared error score. Requires mean, defining the prediction mean

ae

Absolute error score. Requires median, defining the prediction median

ds

Dawid-Sebastiani score. Requires mean and sd, defining the mean and standard deviation of the predictions

interval

Squared error score. Requires lwr and upr, defining the lower and upper prediction interval endpoints, and alpha, defining the prediction error probability targeted by the prediction intervals

The scores S_{type}(F_i,y_i) are defined as in the lecture notes, with obs[i] equal to y_i and each prediction distribution F_i defined by additional named arguments.

Value

A vector of calculated scores, one for each observation

Examples

# Five realisations of N(3, 3^2)
obs <- rnorm(5, mean = 3, sd = 3)

# One prediction for each observation. Only one of the predictions describes
# the true model
F_mean <- 1:5
F_median <- 1:5
F_sd <- 1:5
F_lwr <- F_mean - F_sd * qnorm(0.9)
F_upr <- F_mean - F_sd * qnorm(0.1)

# Compute the scores
data.frame(
  se = proper_score("se", obs, mean = F_mean),
  ae = proper_score("ae", obs, median = F_median),
  ds = proper_score("ds", obs, mean = F_mean, sd = F_sd),
  interval = proper_score("interval", obs,
                          lwr = F_lwr, upr = F_upr, alpha = 0.2)
)

finnlindgren/StatCompLab documentation built on March 23, 2023, 11:47 a.m.