brierScore: Brier Scores

In ensembleBMA: Probabilistic Forecasting using Ensembles and Bayesian Model Averaging

Brier Scores

Description

Computes Brier Scores for climatology, raw ensemble, and ensemble forecasting models given observation thresholds.

Usage

brierScore( fit, ensembleData, thresholds, dates = NULL, ...)

Arguments

fit	A model fit to ensemble forecasting data.
ensembleData	An ensembleData object including ensemble forecasts, verification observations and possibly dates. Missing values (indicated by NA) are allowed. \ This need not be the data used for the model fit, although it must include the same ensemble members.
thresholds	One or more threshold values for the Brier score computations.
dates	The dates for which the Brier score will be computed. These dates must be consistent with fit and ensembleData. The default is to use all of the dates in fit. The dates are ignored if fit originates from fitBMA, which also ignores date information.
...	Included for generic function compatibility.

Value

A data frame giving the Brier Scores for climatology (empirical distribution of the verifying observations), ensemble (voting), and ensemble foreacsting models for the specified thresholds.
A logistic Brier score is also given for the BMAgamma0 model.

References

G. W. Brier, Verification of forecasts expressed in terms of probability, Monthly Weather Review, 78:1–3, 1950.

T. Gneiting and A. E. Raftery, Strictly proper scoring rules, prediction and estimation, Journal of the American Statistical Association 102:359–378, 2007.

C. Fraley, A. E. Raftery, T. Gneiting and J. M. Sloughter, ensembleBMA: An R Package for Probabilistic Forecasting using Ensembles and Bayesian Model Averaging, Technical Report No. 516R, Department of Statistics, University of Washington, 2007 (revised 2010).

See Also

ensembleBMA

Examples

  data(ensBMAtest)

  ensMemNames <- c("gfs","cmcg","eta","gasp","jma","ngps","tcwb","ukmo")
  
  obs <- paste("PCP24","obs", sep = ".")
  ens <- paste("PCP24", ensMemNames, sep = ".")

  prcpTestData <- ensembleData( forecasts = ensBMAtest[,ens],
                                dates = ensBMAtest[,"vdate"], 
                                observations = ensBMAtest[,obs], 
                                station = ensBMAtest[,"station"],
                                forecastHour = 48,
                                initializationTime = "00")

## Not run:  # R check
  prcpTestFit <- ensembleBMAgamma0( prcpTestData, trainingDays = 30)

## End(Not run) 

  hist(prcpTestData$obs)

  brierScore(prcpTestFit, prcpTestData,
             thresholds = seq(from = 0, to = .5, by = .1))

ensembleBMA documentation built on Sept. 2, 2022, 9:05 a.m.