Description Usage Arguments Details Value Author(s) References See Also Examples
Calculate Brier scores, average Brier scores by a grouping variable, and Brier score decompositions for two-alternative forecasts.
1 2 |
object |
an object of class "formula", of the form
|
data |
an optional data frame or list containing the
variables in the formula. If not found in |
group |
the name of a grouping variable within |
decomp |
if |
bounds |
a vector of length 2 corresponding to the desired minimum and maximum Brier score, respectively. |
reverse |
if |
wt |
a vector of weights for computing a weighted Brier score. If |
decompControl |
a list of additional settings for the Brier decomposition. See options below. |
If decomp=TRUE
or group
is supplied, the function returns
a list (see value section). Otherwise, the function returns a numeric
vector containing the Brier score associated with each forecast.
Some decompControl
arguments are specifically designed for forecasting tournaments and may not be useful in other situations. Possible arguments for decompControl
include:
A vector of weights, for performing a weighted Brier
decomposition (could also use the simple wt
argument).
A vector of question ids, for use with the qtype
argument.
If TRUE
(default), forecasts are binned prior to decomposition. If FALSE
, the original forecasts are maintained.
A data frame with columns qid
, ord
, squo
. For each unique question id in the qid
argument above, this describes whether or not the question is ordinal (1=yes,0=no) and whether or not the question has a "status quo" interpretation (1=yes,0=no).
Should Brier components be rescaled, such that 1 is
always best and 0 is always worst? Defaults to FALSE
.
To what value should forecasts be rounded (necessary for Murphy decomposition)? Defaults to .1, meaning that forecasts are rounded to the nearest .1.
Method for ensuring that each forecast sums to 1. If equal to 1 (default), the smallest forecast is one minus the sum of the other forecasts. If equal to 2, the forecast furthest from its rounded value is one minus the sum of other forecasts.
Desired number of Brier resamples (useful for questions with inconsistent alternatives). Defaults to 0; see Merkle & Hartman reference for more detail.
Depending on input arguments, brierscore
may return an object of
class numeric
containing raw Brier scores. It may also return
a list containing the objects below.
rawscores |
an object of class |
brieravg |
an object of class |
decomp |
an object of class |
Ed Merkle
Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78, 1-3.
Merkle, E. C. & Hartman, R. (2018). Weighted Brier score decompositions for topically heterogenous forecasting tournaments. Working paper.
Murphy, A. H. (1973). A new vector partition of the probability score. Journal of Applied Meteorology, 12, 595-600.
Yates, J. F. (1982). External correspondence: Decompositions of the mean probability score. Organizational Behavior and Human Performance, 30, 132-156.
Young, R. M. B. (2010). Decomposition of the Brier score for weighted forecast-verification pairs. Quarterly Journal of the Royal Meteorological Society, 136, 1364-1370.
1 2 3 4 5 6 7 8 9 10 | data("WorldEvents")
## Raw Brier scores
brier1 <- brierscore(answer ~ forecast, data=WorldEvents)
## Raw Brier scores plus group means and decompositions
brier2 <- brierscore(answer ~ forecast, data=WorldEvents,
group="forecaster", decomp=TRUE)
## Obtain Brier scores via calcscore
brier3 <- calcscore(answer ~ forecast, data=WorldEvents,
param=2, fam="pow")
all.equal(brier1, brier3)
|
[1] TRUE
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.