FQI: Forecast Quality Index

Description Usage Arguments Details Value Author(s) References See Also Examples

View source: R/functionals.R

Description

Functions for calculating the Forecast Quality Index (FQI) and its components.

Usage

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
FQI(object, surr = NULL, k = 4, time.point = 1, obs = 1, model = 1, ...)

UIQI(X, Xhat, ...)

ampstats(X, Xhat, only.nonzero = FALSE)

## S3 method for class 'fqi'
print(x, ...)

## S3 method for class 'fqi'
summary(object, ...)

Arguments

object

list object of class “SpatialVx”. In the case of the summary method, object is the list object returned by FQI.

X,Xhat

numeric matrices giving the fields for the verification set.

x

list object of class “fqi” as returned by FQI.

surr

three-dimesnional array containing surrogate fields for X, e.g. as returned by surrogater2d. If NULL, these will be calculated using surrogater2d.

only.nonzero

logical, should the means and variances of only the non-zero values of the fields be calculated (if so, the covariance is returned as NA)?

k

numeric vector for use with the partial Hausdorff distance. For k that are whole numerics or integers >= 1, then the k-th highest value is returned by locmeasures2d. If 0 <= k < 1, then the corresponding quantile is returned.

time.point

numeric or character indicating which time point from the “SpatialVx” verification set to select for analysis.

obs, model

numeric indicating which forecast model to select for the analysis.

...

In the case of FQI, additional arguments to surrogater2d. Only used if surr is NULL. In the case of UIQI, additional arguments to ampstats. In the case of summary.fqi, these are not used.

Details

The FQI was proposed as a spatial verification metric (a true metric in the mathematical sense) by Venugopal et al. (2005) to combine amplitude and displacement error information in a single summary statistic. It is given by

FQI = (PHD_k(X, Xhat)/mean( PHD_k(X, surr_i); i in 1 to number of surrogates)) / (brightness * distortion)

where the numerator is a normalized partial Hausdorff distance (see help file for locperf), brightness (also called bias) is given by 2*(mu1*mu2)/(mu1^2+mu2^2), where mu1 (mu2) is the mean value of X (Xhat), and the distortion term is given by 2*(sig1*sig2)/(sig1^2+sig2^2), where sig1^2 (sig2^2) is the variance of X (Xhat) values. The denominator is a modified UIQI (Universal Image Quality Index; Wang and Bovik, 2002), which itself is given by

UIQI = cor(X,Xhat)*brightness*distortion.

Note that if only.nonzero is TRUE in the call to UIQI, then the modified UIQI used in the FQI formulation is returned (i.e., without multiplying by the correlation term).

The print method so far just calls the summary method.

Value

FQI returns a list with with the following components:

phd.norm

matrix of normalized partial Hausdorff distances for each value of k (rows) and each threshold (columns).

uiqi.norm

numeric vector of modified UIQI values for each threshold.

fqi

matrix of FQI values for each value of k (rows) and each threshold (columns).

It will also have the same attributes as the “SpatialVx” object with additional attributes defining the arguments specific to parameters used by the function.

UIQI returns a list with components:

data.name

character vector giving the names of the two fields.

cor

single numeric giving the correlation between the two fields.

brightness.bias

single numeric giving the brightness (bias) value.

distortion.variability

single numeric giving the distortion (variability) value.

UIQI

single numeric giving the UIQI (or modified UIQI if only.nonzero is set to TRUE) value.

ampstats returns a list object with components:

mean.fcst,mean.vx

single numerics giving the mean of Xhat and X, resp.

var.fcst,var.vx

single numerics giving the variance of Xhat and X, resp.

cov

single numeric giving the covariance between Xhat and X (if only.nonzero is TRUE, this will be NA).

Author(s)

Eric Gilleland

References

Venugopal, V., Basu, S. and Foufoula-Georgiou, E. (2005) A new metric for comparing precipitation patterns with an application to ensemble forecasts. J. Geophys. Res., 110, D08111, 11 pp., doi:10.1029/2004JD005395.

Wang, Z. and Bovik, A. C. (2002) A universal image quality index. IEEE Signal Process. Lett., 9, 81–84.

See Also

locperf, surrogater2d, locmeasures2d

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
data( "ExampleSpatialVxSet" )

x <- ExampleSpatialVxSet$vx
xhat <- ExampleSpatialVxSet$fcst

# Now, find surrogates of the simulated field.
z <- surrogater2d(x, zero.down=TRUE, n=10)

u <- list( X = cbind( quantile( c(x), c(0.75, 0.9)) ),
            Xhat = cbind( quantile( c(xhat), c(0.75, 0.9) ) ) )

hold <- make.SpatialVx(x, xhat, thresholds = u,
    field.type = "Example", units = "none",
    data.name = "ExampleSpatialVxSet", 
    obs.name = "X", model.name = "Xhat" )

FQI(hold, surr = z, k = c(4, 0.75) )

Example output

Loading required package: spatstat
Loading required package: nlme
Loading required package: rpart

spatstat 1.52-1       (nickname: 'Apophenia') 
For an introduction to spatstat, type 'beginner' 


Note: R version 3.4.1 (2017-06-30) is more than 9 months old; we strongly recommend upgrading to the latest version
Loading required package: fields
Loading required package: spam
Loading required package: dotCall64
Loading required package: grid
Spam version 2.1-1 (2017-07-02) is loaded.
Type 'help( Spam)' or 'demo( spam)' for a short introduction 
and overview of this package.
Help for individual functions is also obtained by adding the
suffix '.spam' to the function name, e.g. 'help( chol.spam)'.

Attaching package: 'spam'

The following objects are masked from 'package:base':

    backsolve, forwardsolve

Loading required package: maps
Loading required package: smoothie
Loading required package: smatr
Loading required package: plyr

Attaching package: 'plyr'

The following object is masked from 'package:maps':

    ozone

Loading required package: turboEM
Loading required package: doParallel
Loading required package: foreach
Loading required package: iterators
Loading required package: parallel
Loading required package: numDeriv
Loading required package: quantreg
Loading required package: SparseM

Attaching package: 'SparseM'

The following object is masked from 'package:spam':

    backsolve

The following object is masked from 'package:base':

    backsolve


Attaching package: 'turboEM'

The following objects are masked from 'package:numDeriv':

    grad, hessian

Comparison for:
  ExampleSpatialVxSet X Xhat 

 Thresholds are:  threshold 1 threshold 2 

 normalized Partial Hausdorff distance
           threshold 1 threshold 2
k = 4;       0.8865963   0.9929434
k = 0.75;    1.0353367   1.0732379

 modified universal image quality index
     threshold 1 threshold 2
[1,]   0.9204923   0.9352584

 Forecast Quality Index (FQI)
           threshold 1 threshold 2
k = 4;       0.9631763    1.061678
k = 0.75;    1.1247641    1.147531
NULL

SpatialVx documentation built on Jan. 4, 2019, 5:15 p.m.