vr.hist: Verification Rank Histogram

vr.histR Documentation

Verification Rank Histogram

Description

This function plots the Verification Rank Histogram (VRH) given observations of an univariate variable and samples of a predictive distribution.

Usage

vr.hist(
  y,
  x,
  bins = NULL,
  type = "relative",
  title = "Verification Rank Histogram",
  reliability = FALSE,
  entropy = FALSE,
  na.rm = TRUE
)

Arguments

y

vector of observations

x

matrix of samples of a predictive distribution (depending on y; see details)

bins

numeric; if NULL the number of bins is equal to ncol(x)+1; otherwise bins must be chosen so that (ncol(x)+1)/bins is an integer; default: NULL (see details)

type

character; "relative", "absolute" and "density"; default: "relative" (see details)

title

character; title of the plot; default: "Verification Rank Histogram"

reliability

logical; if TRUE the reliability index is calculated for the plot (see details); if FALSE the reliability index is not calculated; default: FALSE

entropy

logical; if TRUE the entropy is calculated for the plot (see details); if FALSE the entropy is not calculated; default: FALSE

na.rm

logical; if TRUE NA are stripped before the rank computation proceeds; if FALSE NA are used in the rank computation; default: FALSE

Details

For a vector y of length n, x should be given as matrix with n rows, where the i-th entry of y belongs to the i-th row of x. The columns of x represent the samples of a predictive distribution. Only finite values of y and x are used.

The parameter bins specifies the number of columns for the VRH. For "large" ncol(x) it is often reasonable to reduce the resolution of the VRH by using bins so that (ncol(x)+1)/bins is an integer.

If type is "relative" the relative frequencies of the bins are plotted. If type is "absolute" the absolute frequencies of the bins are plotted. If type is "density" the relative densities of the bins are plotted.

An uniform VRH indicates a calibrated predictive distribution. A ∩-shape in the VRH indicates overdispersion and a ∪-shape indicates underdispersion of the predictive distribution. A systematic bias of the predictive distribution results in a triangular shaped VRH histogram.

The deviation from uniformity of the VRH can be quantified by the reliability index (RI). The smaller the RI, the better is the calibration of the forecast. The optimal value of the RI is 0.

The entropy is a tool to assess the calibration of a forecast. The optimal value of the entropy is 1, representing a calibrated forecast.

Value

ggplot object with a plot of the Verification Rank Histogram.

Author(s)

David Jobst

References

Anderson, J. (1996). A method for producing and evaluating probabilistic forecasts from ensemble model integrations. Journal of Climate, 9, 1518-1530.

Candille, G. and Talagrand, O. (2005). Evaluation of probabilistic prediction systems for a scalar variable. Quarterly Journal of the Royal Meteorological Society, 131(609), 2131-2150.

Delle Monache, L., Hacker, J., Zhou, Y., Deng, X. and Stull, R., (2006). Probabilistic aspects of meteorological and ozone regional ensemble forecasts. Journal of Geophysical Research: Atmospheres, 111, D24307.

Hamill, T. and Colucci, S. (1997). Verification of Eta-RSM short-range ensemble forecasts. Monthly Weather Review, 125, 1312-1327.

Hamill, T. (2001). Interpretation of rank histograms for verifying ensemble forecasts. Monthly Weather Review, 129, 550-560.

Taillardat, M., Mestre, O., Zamo, M. and Naveau, P., (2016). Calibrated Ensemble Forecasts Using Quantile Regression Forests and Ensemble Model Output Statistics. American Meteorological Society, 144, 2375-2393.

Tribus, M. (1969). Rational Descriptions, Descisions and Designs. Pergamon Press.

Talagrand, O., Vautard, R. and Strauss, B. (1997). Evaluation of probabilistic prediction systems. Workshop on Predictability (ECMWF), 1-25.

Examples

# simulated data
n <- 30
m <- 50
y <- rnorm(n)
x <- matrix(rnorm(n*m), ncol = m)

# vr.hist plot
vr.hist(y = y, x = x)
vr.hist(y = y, x = x, bins = 17, title = "VRH", reliability = TRUE, entropy = FALSE)
vr.hist(y = y, x = x, bins = 3, type = "absolute", reliability = FALSE, entropy = TRUE)
vr.hist(y = y, x = x, bins = 3, type = "density", reliability = TRUE, entropy = TRUE)


jobstdavid/eppverification documentation built on May 13, 2024, 5:20 p.m.