score_predictions: Scoring rules to evaluate prediction accuracy

In opisthokonta/goalmodel: Prediction Models For Goals Scored

Scoring rules to evaluate prediction accuracy

Description

This function provides scoring rules for evaluating prediction accuracy.

Usage

score_predictions(predictions, observed, score)

Arguments

predictions	A matrix or data frame with probabilities, with one column for each outcome, and one row for each prediction.
observed	Numeric or character vector of the same length as the number of predictions. It must contain an indicator of the observed outcome, either a column number or a column name.
score	Character vector of the scoring functions to use. Currently 'log', 'brier', and 'rps' are available.

Details

Currently three scoring rules are available: The log score, Brier score, and the Ranked Probability Score (RPS).

The log score is just the negative logarithm of the probability of the observed outcome.

For all three scoring functions, a lower score means a better predictions. They will attain a lowest possible score of 0 if the observed outcome were predicted with a 100 The RPS has an upper limit of 1, which indicates the worst possible prediction. The Brier score has an upper limit of 2. The log score has no upper limit, and will be infinite if the observed outcome were predicted with probability 0.

References

Wheatcroft, E. (2021) Evaluating probabilistic forecasts of football matches: the case against the ranked probability score. https://doi.org/10.1515/jqas-2019-0089

Constantinou, A. C., and N. E. Fenton (2012) Solving the Problem of Inadequate Scoring Rules for Assessing Probabilistic Football Forecast Models. https://doi.org/10.1515/1559-0410.1418

opisthokonta/goalmodel documentation built on April 3, 2024, 1:32 a.m.