In eco4cast/score4cast: Scoring Utilities for Forecast Competitions

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.path = "man/figures/README-",
  out.width = "100%"
)

score4cast

The goal of score4cast is to provide a convenient interface to score ecological forecasts that conform the EFI standard. score4cast emphasizes the use of strictly proper scores (see scoringRules R package or Gneiting & Raferty's landmark 2007 paper) for probablistic forecasts. The EFI format provides a simple but flexible way to express both ensemble and parametric forecasts in a standard tabular layout.

Installation

You can install the development version of score4cast from GitHub with:

# install.packages("devtools")
devtools::install_github("eco4cast/score4cast")

Example

A forecast (in standardized format) is scored against a target (in standardized format):

library(score4cast)
ex_data <- system.file("extdata/standard-format-examples.R", package="score4cast")
source(ex_data)

scores <- score(ex_forecast, ex_target)
scores

library(tidyverse)

Example using Bernoulli distribution and extra columns for grouping

forecast <- tibble(datetime = as_date("2023-01-02"),
             site_id = "fcre",
             depth = c(1,2),
             model_id = "test",
             reference_datetime = as_date("2023-01-02"),
             variable = "temp",
             family = "bernoulli",
             parameter = "prob",
             prediction = c(0.3, 0.1))

target <- tibble(datetime = as_date("2023-01-02"),
             site_id = "fcre",
             depth = c(1,2),
             variable = "temp",
             observation = c(1,0))

crps_logs_score(forecast,target, extra_groups = "depth")