compute_coverage: Compute observed coverage from a quantile forecaster
In dshemetov/evalcast-mirror: Tools For Evaluating COVID Forecasters

Description Usage Arguments Details Value

Compute observed coverage from a quantile forecaster

compute_coverage(
  predictions_cards,
  geo_type = c("county", "hrr", "msa", "dma", "state", "hhs", "nation"),
  grp_vars = c("forecaster", "forecast_date", "ahead"),
  avg_vars = c("geo_value"),
  backfill_buffer = 0
)

`predictions_cards`	tibble of predictions that are all for the same prediction task, meaning they are for the same response, incidence period,and geo type. Forecasts may be for a different forecast date or forecaster. A predictions card may be created by the function `get_predictions()`, downloaded with `get_covidhub_predictions()` or possibly created manually.
`geo_type`	String indicating geographical type, such as "county", or "state". See the COVIDcast Geographic Coding documentation for available options.
`grp_vars`	character vector of named columns in the score_card at which average performance will be returned
`avg_vars`	character vector of named columns in the score_card over which averaging performance will be computed
`backfill_buffer`	How many days until response is deemed trustworthy enough to be taken as correct?

Checks are performed to ensure that averaging variables all contain the same confidence intervals though these may vary over grouping variables. avg_vars and grp_vars must have an empty intersection.

A tibble containing grp_vars, nominal_prob (the claimed interval coverage), prop_below (the proportion of actual values falling below the lower end of the confidence interval), prop_above (the proportion of actual values falling above the upper end of the confidence interval), and prop_covered (the proportion falling inside the interval). All proportions are calculated for each available symmetric interval at each combination of grouping variables by averaging over any variables listed in avg_vars.

dshemetov/evalcast-mirror documentation built on Feb. 4, 2022, 8:52 a.m.