compute_actual_vs_nominal_prob: Compute calibration of a quantile forecaster
In dshemetov/evalcast-mirror: Tools For Evaluating COVID Forecasters

Description Usage Arguments Details Value

Compute calibration of a quantile forecaster

compute_actual_vs_nominal_prob(
  predictions_cards,
  geo_type,
  backfill_buffer = 10,
  grp_vars = c("forecaster", "forecast_date", "ahead"),
  avg_vars = c("geo_value")
)

`predictions_cards`	tibble containing at least columns actual, quantile, value and any grouping or averaging variables named in the next arguments
`geo_type`	String indicating geographical type, such as "county", or "state". See the COVIDcast Geographic Coding documentation for available options.
`backfill_buffer`	How many days until response is deemed trustworthy enough to be taken as correct?
`grp_vars`	character vector of named columns in the score_card at which average performance will be returned
`avg_vars`	character vector of named columns in the score_card over which averaging performance will be computed

Note that no checks are performed to ensure that averaging variables all contain the same predicted quantiles. avg_vars and grp_vars must have an empty intersection.

A tibble containing grp_vars, nominal_prob (the claimed interval coverage), prop_below (the proportion of actual values falling below the forecast quantile) and prop_above (the proportion of actual values falling above the forecast quantile). All proportions are calculated for each available probability at each combination of grouping variables by averaging over any variables listed in avg_vars.

dshemetov/evalcast-mirror documentation built on Feb. 4, 2022, 8:52 a.m.