Calculate average score differences for competing ensemble forecasts.

Share:

Description

Calculate the average ranked probability score (RPS) difference between two competing ensemble forecasts for the same observation. Approximate sampling quantiles of the average score difference and p-value of the paired one-sided t-test are provided. The difference between the score of 'ens.ref' and 'ens' is calculated. The higher the score difference, the higher the improvement of 'ens' over 'ens.ref'.

Usage

1
EnsRpsDiff(ens, ens.ref, obs, probs)

Arguments

ens

N*K matrix. ens[i,j] is the number of ensemble members that predict category j at time i.

ens.ref

N*K matrix, similar to ens

obs

N*K matrix. obs[i,j] = 1 if category j is observed at time i, 0 otherwise.

probs

vector of probabilities. The probabilities of estimated sampling quantiles of the average score difference. Can be used to construct confidence intervals.

Value

A list with the following elements:

"rps.diff": The value of the average score difference.

"sampling.quantiles": The quantiles of the sampling distribution of the average score difference corresponding to the 'probs' argument. The sampling quantiles are approximated by a t-distribution as follows:

qt(probs, df=N-1) * sd(score.diff) / sqrt(N) + mean.score.diff

"p.value": p value of the one-sided paired t-test.

Examples

1
2
3
4
5
  # Example:
  ens <- matrix(c(1,2,3, 1,3,2), byrow=TRUE, nrow=2)
  ens.ref <- matrix(c(2,2,2, 2,2,2), byrow=TRUE, nrow=2)
  obs <- matrix(c(0,0,1, 0,1,0), byrow=TRUE, nrow=2)
  EnsRpsDiff(ens, ens.ref, obs, probs=c(0.025, 0.975))

Want to suggest features or report bugs for rdrr.io? Use the GitHub issue tracker.