vxstats: Some Common Traditional Forecast Verification Statistics.

Description Usage Arguments Details Value Warning Note Author(s) References See Also Examples

View source: R/Hoods2d.R

Description

Calculates some common traditional forecast verification statistics.

Usage

1
2
3
vxstats(X, Xhat, which.stats = c("bias", "ts", "ets", "pod",
                 "far", "f", "hk", "bcts", "bcets", "mse"), subset =
                 NULL)

Arguments

X,Xhat

k by m matrix of verification and forecast values, resp.

which.stats

character vector giving the names of the desired statistics. See Details below.

subset

numeric vector indicating a subset of the verification set over which to calculate the verification statistics.

Details

Computes several traditional verification statistics (see Wilks, 2006, Ch. 7; Jolliffe and Stephenson, 2012 for more on these forecast verification statistics; as well as the Issues, Methods and FAQ web page of the Joint Working Group on Forecast Verification of the World Meteorological Organization at: https://www.cawcr.gov.au/projects/verification/) The possible statistics that can be computed, as determined by which.stats are:

“bias” the number of forecast events divided by the number of observed events (sometimes called frequency bias).

“ts” threat score, given by hits/(hits + misses + false alarms)

“ets” equitable threat score, given by (hits - hits.random)/(hits + misses + false alarms - hits.random), where hits.random is the number of observed events times the number of forecast events divided by the total number of forecasts.

“pod” probability of detecting an observed event (aka, hit rate). It is given by hits/(hits + misses).

“far” false alarm ratio, given by (false alarms)/(hits + false alarms).

“f” false alarm rate (aka probability of false detection) is given by (false alarms)/(correct rejections + false alarms).

“hk” Hanssen-Kuipers Score is given by the difference between the hit rate (“pod”) and the false alarm rate (“f”).

“bcts”, “bcets”, Bias Corrected Threat Score (Equitable Threat Score) as introduced in Mesinger (2008); see also Brill and Mesinger (2009). Also referred to as the dHdA versions of these scores.

“mse” mean square error (not a contingency table statistic, but can be used with binary fields). This is the only statistic that can be calculated here that does not require binary fields for Fcst and Obs.

Value

A list with components determined by which.stats, which may include any or all of the following.

bias

numeric giving the frequency bias.

ts

numeric giving the threat score.

ets

numeric giving the equitable threat score, also known as the Gilbert Skill Score.

pod

numeric giving the probability of decking an event, also known as the hit rate.

far

numeric giving the false alarm ratio.

f

numeric giving the false alarm rate.

hk

numeric giving the Hanssen and Kuipers statistic.

bcts, bcets

numeric giving the bias corrected version of the threat- and/or equitable threat score.

mse

numeric giving the mean square error.

Warning

It is up to the user to provide the appropriate type of fields for the given statistics to be computed. For example, they must be binary for all types of which.stats except mse.

Note

See the web page: https://www.cawcr.gov.au/projects/verification/ for more details about these statistics, and references.

Author(s)

Eric Gilleland

References

Brill, K. F. and Mesinger, F. (2009) Applying a general analytic method for assessing bias sensitivity to bias-adjusted threat and equitable threat scores. Wea. Forecasting, 24, 1748–1754.

Jolliffe, I. T. and Stephenson, D. B., Edts. (2012) Forecast Verification: A Practitioner's Guide in Atmospheric Science, 2nd edition. Chichester, West Sussex, U.K.: Wiley, 274 pp.

Mesinger, F. (2008) Bias adjusted precipitation threat scores. Adv. Geosci., 16, 137–142.

Wilks, D. S. (2006) Statistical Methods in the Atmospheric Sciences. 2nd Edition, Academic Press, Burlington, Massachusetts, 627pp.

See Also

hoods2d

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Calculate the traditional verification scores for the first geometric case
# of the ICP.
data( "geom001" )
data( "geom000" )

rmse <- sqrt(vxstats( geom001, geom000, which.stats="mse")$mse)
rmse
vxstats( geom001 > 0, geom000 > 0, which.stats=c("bias", "ts", "ets", "pod", "far", "f", "hk"))

data( "geom005" )
vxstats( geom005 > 0, geom000 >0, which.stats=c("ts","ets","bcts","bcets"))

Example output

Loading required package: spatstat
Loading required package: spatstat.data
Loading required package: nlme
Loading required package: rpart

spatstat 1.59-0       (nickname: 'J'ai omis les oeufs de caille') 
For an introduction to spatstat, type 'beginner' 


Note: R version 3.4.4 (2018-03-15) is more than 9 months old; we strongly recommend upgrading to the latest version
Loading required package: fields
Loading required package: spam
Loading required package: dotCall64
Loading required package: grid
Spam version 2.2-2 (2019-03-07) is loaded.
Type 'help( Spam)' or 'demo( spam)' for a short introduction 
and overview of this package.
Help for individual functions is also obtained by adding the
suffix '.spam' to the function name, e.g. 'help( chol.spam)'.

Attaching package: 'spam'

The following objects are masked from 'package:base':

    backsolve, forwardsolve

Loading required package: maps
See https://github.com/NCAR/Fields for
 an extensive vignette, other supplements and source code 
Loading required package: smoothie
Loading required package: smatr
Loading required package: turboEM
Loading required package: doParallel
Loading required package: foreach
Loading required package: iterators
Loading required package: parallel
Loading required package: numDeriv
Loading required package: quantreg
Loading required package: SparseM

Attaching package: 'SparseM'

The following object is masked from 'package:spam':

    backsolve

The following object is masked from 'package:base':

    backsolve


Attaching package: 'turboEM'

The following objects are masked from 'package:numDeriv':

    grad, hessian

[1] 13.83465
$bias
[1] 1

$ts
[1] 0

$ets
[1] -0.013148

$pod
[1] 0

$far
[1] 1

$f
[1] 0.02664635

$hk
[1] -0.02664635

attr(,"class")
[1] "vxstats"
$ts
[1] 0.1073921

$ets
[1] 0.08397798

$bcts
[1] 0.4183034

$bcets
[1] 0.5821826

attr(,"class")
[1] "vxstats"

SpatialVx documentation built on March 28, 2021, 1:10 a.m.