verify_index: Function for verification of hindcasts of indices

In Climandes/ClimIndVis: Calculation, verification and visualisation of climate indices

Description

Verifies daily seasonal hindcasts (re-forecasts) of indices for gridded or station data. It is based on the packages easyVerification and SpecsVerification.

Usage

verify_index(
  obs,
  hc,
  hc_ref = NULL,
  index,
  index_args = list(),
  selyears = NULL,
  veri_metrics,
  veri_args = list()
)

Arguments

obs	Observations. "climindvis" object of type p/grid as output from make_object by using daily observational or Re-analysis data.
hc	Hindcasts. "climindvis_index" object of type hc_p/hc_grid as output from make_object using an ensemble of daily seasonal hindcasts.
hc_ref	Reference forecast for calculation of skill scores. "climindvis" object of type hc_p/hc_grid as output from make_object sing an ensemble of daily seasonal hindcasts . If hc_ref=NULL climatology will be taken as reference, Default = NULL.
index	Name of index to be calculated., e.g. index = "dd" or index = "tnn". For list of indices see indices_list
index_args	List of arguments for index. See indices_list.
selyears	Integer array of years to be selected from data. For indices with quantiles, years as defined in index_args are used for quantile calculation, but only selected years are plotted. For functions using seasonal forecast data, selyears is only applied to hindcasts.
veri_metrics	Character array of verification metrics as defined in the easyVerification package, e.g. c("EnsCorr","EnsRpss"). Also see "Verification metrics" below.
veri_args	Optional list of additional arguments for verification, see veri_args

Value

Object of class index and "climindvis_verification" with following entries:

• verification: Named list with verification outputs for each metric in veri_metric.

• veri_info: Information needed for plotting of verification results, such as limits and dimensions of metric

• years: years used for verification

• lon: Array of longitudes

• lat: Array of latitudes

• data_info$obs and $hc ($hc_ref):Data_info taken from obs and hc input objects, e.g. name of dataset

Cateogry forecasts

For category skill scores (e.g. ROC area and Rps(s)) forecasts have to be provided in categories. The easyVerification package automatically converts continuous forecast to category forecasts. The optional parameter veri_args$ncat determines the number of equidistant forecast categories (e.g. 3:terciles, 4:quartiles, 5:quintiles). This corresponds to setting the variable prob of easyVerification{veriApply} to prob=c(1:((ncat-1)/ncat)). However, if you want to use other probability thresholds you can use the argument veri_args$prob. The use of absolute thresholds (as possible in easyVerification{veriApply}) is currently not implemented.

Verification metrics

The verify_index function can be used of calculating all of the skill metrics defined in the packages easyVerification and SpecsVerification. Skill scores ending in "ss" denote the corresponding skill scores (see below). Currently the following verification metrics from the easyVerification package can be calculated (the ending "ss" denotes the corresponding skill scores):

Ens2AFC

Generalized Discrimination Score

EnsCorr

Correlation with Ensemble Mean

EnsMae/EnsMaess

Ensemble mean absolute error

EnsMse/EnsMsess

Ensemble mean squared error

EnsRmse/EnsRmsess

Square root of ensemble mean squared error

EnsRoca/EnsRocss

Area Under the ROC Curve

EnsSprErr

Spread to Error Ratio

FairSprErr

Fair Spread to Error Ratio

EnsRps/EnsRpss

Ranked probability skill score

FairRps/FairRpss

Fair Ranked probability skill score

EnsCrps/EnsCrpss

Continuous Ranked probability skill score

FairCrps/FairCrpss

Fair Continuous Ranked probability skill score

More information about the skill scores can be found in the respective package.

Skill scores are defined as (mean score - mean score for reference) / (perfect score - mean score for reference). The skill score is zero if the mean score of the forecast equals the mean score of the reference forecast, and equals one if the mean score of the forecast equals the best possible score. If hc_ref is not defined, the climatology is taken as reference.

For more information about Fair scores, see e.g. Ferro, C. A. T. (2014). Fair scores for ensemble forecasts. Quarterly Journal of the Royal Meteorological Society, 140(683), 1917-1923.

See Also

easyVerification, calc_index

Examples

## load example hindcast and observational data for stations
## (see \code{\link{example_climindvis_objects}}):

data("object_hc_st", "object_st")

## verify hindcasts of indices using the skill metrics
## "EnsRmsess", "EnsRpss", "EnsCorr":

veri<-verify_index(obs = object_st, hc = object_hc_st, index="txx", index_args=list(aggt="seasonal") , veri_metrics = c("EnsRmsess", "EnsRpss", "EnsCorr"))

Climandes/ClimIndVis documentation built on Oct. 24, 2021, 10:52 a.m.