crps: Continuous Ranked Probability Score

In ensembleBMA: Probabilistic Forecasting using Ensembles and Bayesian Model Averaging

Continuous Ranked Probability Score

Description

Computes the continuous ranked probability score (CRPS) for univariate ensemble forecasting models.

Usage

crps( fit, ensembleData, dates=NULL, nSamples=NULL, seed=NULL, ...)
CRPS( fit, ensembleData, dates=NULL, nSamples=NULL, seed=NULL, ...)

Arguments

fit	A model fit to ensemble forecasting data.
ensembleData	An ensembleData object that includes ensemble forecasts, verification observations and possibly dates. Missing values (indicated by NA) are allowed. \ This need not be the data used for the model fit, although it must include the same ensemble members.
nSamples	The number of simulation samples for CRPS via simulation. For the normal model, the default is analytic computation of the CRPS. For the gamma model with a point mass at 0 (precipitation), the CRPS is always computed by simulation, with default nSamples = 10000.
seed	Argument to set.seed for random number generation in simulation.
dates	The dates for which the CRPS will be computed. These dates must be consistent with fit and ensembleData. The default is to use all of the dates in fit. The dates are ignored if fit originates from fitBMA, which also ignores date information.
...	Included for generic function compatibility.

Details

These methods are generic, and can be applied to all ensemble forecasting models.
For gamma0 model for precipitation and the gamma model for wind speed the CRPS is only available through simulation. The default number of simulation samples is 10,000.
Note that the gamma0 model for precipitation and the gamma model for wind speed may have been applied to a power transformation of the data.
For normal models for temperature and pressure, analytic computation of the CRPS is the default. CRPS will be computed via simulation for normal models only if nSamples is set to a positive value.
For the bivariate normal model for wind speed and direction, the CRPS is computed for the marginal wind speed distribution.

Value

crps is a matrix giving the CRPS for each instance in the data for both the raw ensemble and the median probabilistic forecast.
CRPS is a vector giving the mean of the CRPS over all instances for the raw ensemble and the median probabilistic forecast.

References

T. Gneiting and A. E. Raftery, Strictly proper scoring rules, prediction and estimation, Journal of the American Statistical Association 102:359–378, 2007.

C. Fraley, A. E. Raftery, T. Gneiting and J. M. Sloughter, ensembleBMA: An R Package for Probabilistic Forecasting using Ensembles and Bayesian Model Averaging, Technical Report No. 516R, Department of Statistics, University of Washington, 2007 (revised 2010).

See Also

ensembleBMA, fitBMA

Examples

  data(ensBMAtest)

  ensMemNames <- c("gfs","cmcg","eta","gasp","jma","ngps","tcwb","ukmo")

  obs <- paste("T2","obs", sep = ".")
  ens <- paste("T2", ensMemNames, sep = ".")

  tempTestData <- ensembleData( forecasts = ensBMAtest[,ens],
                                dates = ensBMAtest[,"vdate"],
                                observations = ensBMAtest[,obs],
                                station = ensBMAtest[,"station"],
                                forecastHour = 48,
                                initializationTime = "00")

## Not run: # R check
  tempTestFit <- ensembleBMAnormal( tempTestData, trainingDays = 30)

## End(Not run)

# for quick run only; use more training days for forecasting
  tempTestFit <- ensembleBMAnormal( tempTestData[1:20,], trainingDays = 8)

  crpsValues <- crps( tempTestFit, tempTestData)
  colMeans(crpsValues)

  CRPS( tempTestFit, tempTestData)

ensembleBMA documentation built on Sept. 2, 2022, 9:05 a.m.