Calculate the average ranked probability score (RPS) difference between two competing ensemble forecasts for the same observation. Approximate sampling quantiles of the average score difference and p-value of the paired one-sided t-test are provided. The difference between the score of 'ens.ref' and 'ens' is calculated. The higher the score difference, the higher the improvement of 'ens' over 'ens.ref'.

1 | ```
EnsRpsDiff(ens, ens.ref, obs, probs)
``` |

`ens` |
N*K matrix. ens[i,j] is the number of ensemble members that predict category j at time i. |

`ens.ref` |
N*K matrix, similar to ens |

`obs` |
N*K matrix. obs[i,j] = 1 if category j is observed at time i, 0 otherwise. |

`probs` |
vector of probabilities. The probabilities of estimated sampling quantiles of the average score difference. Can be used to construct confidence intervals. |

A list with the following elements:

"rps.diff": The value of the average score difference.

"sampling.quantiles": The quantiles of the sampling distribution of the average score difference corresponding to the 'probs' argument. The sampling quantiles are approximated by a t-distribution as follows:

qt(probs, df=N-1) * sd(score.diff) / sqrt(N) + mean.score.diff

"p.value": p value of the one-sided paired t-test.

1 2 3 4 5 |

Questions? Problems? Suggestions? Tweet to @rdrrHQ or email at ian@mutexlabs.com.

All documentation is copyright its authors; we didn't write any of that.