Numerical Goodness-of-fit measures

Description

Numerical goodness-of-fit measures between sim and obs, with treatment of missing values. Several performance indices for comparing two vectors, matrices or data.frames

Usage

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
gof(sim, obs, ...)

## Default S3 method:
gof(sim, obs, na.rm=TRUE, do.spearman=FALSE, do.pbfdc=FALSE,
             j=1, norm="sd", s=c(1,1,1), method=c("2009", "2012"), lQ.thr=0.7, 
             hQ.thr=0.2, digits=2,...)
## S3 method for class 'matrix'
gof(sim, obs, na.rm=TRUE, do.spearman=FALSE, do.pbfdc=FALSE,
             j=1, norm="sd", s=c(1,1,1), method=c("2009", "2012"), lQ.thr=0.7, 
             hQ.thr=0.2, digits=2,...)
## S3 method for class 'data.frame'
gof(sim, obs, na.rm=TRUE, do.spearman=FALSE, do.pbfdc=FALSE,
             j=1, norm="sd", s=c(1,1,1), method=c("2009", "2012"), lQ.thr=0.7, 
             hQ.thr=0.2, digits=2,...)
## S3 method for class 'zoo'
gof(sim, obs, na.rm=TRUE, do.spearman=FALSE, do.pbfdc=FALSE,
             j=1, norm="sd", s=c(1,1,1), method=c("2009", "2012"), lQ.thr=0.7, 
             hQ.thr=0.2, digits=2,...)                                       

Arguments

sim

numeric, zoo, matrix or data.frame with simulated values

obs

numeric, zoo, matrix or data.frame with observed values

na.rm

a logical value indicating whether 'NA' should be stripped before the computation proceeds.
When an 'NA' value is found at the i-th position in obs OR sim, the i-th value of obs AND sim are removed before the computation.

do.spearman

logical. Indicates if the Spearman correlation has to be computed. The default is FALSE.

do.pbfdc

logical. Indicates if the Percent Bias in the Slope of the midsegment of the Flow Duration Curve (pbiasfdc) has to be computed. The default is FALSE.

j

argument passed to the mNSE function

norm

argument passed to the nrmse function

s

argument passed to the KGE function

method

argument passed to the KGE function

lQ.thr

argument passed to the (optional) pbiasfdc function

hQ.thr

argument passed to the (optional) pbiasfdc function

digits

decimal places used for rounding the goodness-of-fit indexes.

...

further arguments passed to or from other methods.

Value

The output of the gof function is a matrix with one column only, and the following rows:

me

Mean Error

mae

Mean Absolute Error

mse

Mean Squared Error

rmse

Root Mean Square Error

nrmse

Normalized Root Mean Square Error ( -100% <= nrms <= 100% )

PBIAS

Percent Bias

pbiasfdc

PBIAS in the slope of the midsegment of the Flow Duration Curve

RSR

Ratio of RMSE to the Standard Deviation of the Observations, RSR = rms / sd(obs). ( 0 <= RSR <= +Inf )

rSD

Ratio of Standard Deviations, rSD = sd(sim) / sd(obs)

NSE

Nash-Sutcliffe Efficiency ( -Inf <= NSE <= 1 )

mNSE

Modified Nash-Sutcliffe Efficiency

rNSE

Relative Nash-Sutcliffe Efficiency

d

Index of Agreement ( 0 <= d <= 1 )

d1

Modified Index of Agreement

rd

Relative Index of Agreement

cp

Persistence Index ( 0 <= PI <= 1 )

r

Pearson Correlation coefficient ( -1 <= r <= 1 )

r.Spearman

Spearman Correlation coefficient ( -1 <= r.Spearman <= 1 )

R2

Coefficient of Determination ( 0 <= R2 <= 1 ).
Gives the proportion of the variance of one variable that is predictable from the other variable

bR2

R2 multiplied by the coefficient of the regression line between sim and obs
( 0 <= bR2 <= 1 )

KGE

Kling-Gupta efficiency between sim and obs
( 0 <= KGE <= 1 )

VE

Volumetric efficiency between sim and obs
( -Inf <= VE <= 1)

Note

obs and sim has to have the same length/dimension.

Missing values in obs and/or sim can be removed before the computations, depending on the value of na.rm.

Although r and r2 have been widely used for model evaluation, these statistics are over-sensitive to outliers and insensitive to additive and proportional differences between model predictions and measured data (Legates and McCabe, 1999)

Author(s)

Mauricio Zambrano Bigiarini <mzb.devel@gmail.com>

References

Legates, D. R., and G. J. McCabe Jr. (1999), Evaluating the Use of "Goodness-of-Fit" Measures in Hydrologic and Hydroclimatic Model Validation, Water Resour. Res., 35(1), 233–241

Krause P., Boyle D.P., and B\"ase F., Comparison of different efficiency criteria for hydrological model assessment, Advances in Geosciences 5 (2005), pp. 89–97

Moriasi, D.N., Arnold, J.G., Van Liew, M.W., Bingner, R.L., Harmel, R.D., Veith, T.L. 2007. Model evaluation guidelines for systematic quantification of accuracy in watershed simulations
Transactions of the ASABE. 50(3):885-900

Boyle, D. P., H. V. Gupta, and S. Sorooshian (2000), Toward Improved Calibration of Hydrologic Models: Combining the Strengths of Manual and Automatic Methods, Water Resour. Res., 36(12), 3663–3674

Kitanidis, P. K., and R. L. Bras (1980), Real-Time Forecasting With a Conceptual Hydrologic Model 2. Applications and Results, Water Resour. Res., 16(6), 1034–1044

J.E. Nash and J.V. Sutcliffe, River flow forecasting through conceptual models. Part 1: a discussion of principles, J. Hydrol. 10 (1970), pp. 282–290

Yapo P. O., Gupta H. V., Sorooshian S., 1996. Automatic calibration of conceptual rainfall-runoff models: sensitivity to calibration data. Journal of Hydrology. v181 i1-4. 23–48

Yilmaz, K. K., H. V. Gupta, and T. Wagener (2008), A process-based diagnostic approach to model evaluation: Application to the NWS distributed hydrologic model, Water Resour. Res., 44, W09417, doi:10.1029/2007WR006716

Hoshin V. Gupta, Harald Kling, Koray K. Yilmaz, Guillermo F. Martinez. Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling. Journal of Hydrology, Volume 377, Issues 1-2, 20 October 2009, Pages 80-91. DOI: 10.1016/j.jhydrol.2009.08.003. ISSN 0022-1694

Criss, R. E. and Winston, W. E. (2008), Do Nash values have value? Discussion and alternate proposals. Hydrological Processes, 22: 2723-2725. doi: 10.1002/hyp.7072

See Also

me, mae, rmse, nrmse, pbias, pbiasfdc, rSD, NSE, mNSE, rNSE, d, md, rd, cp, br2, KGE, VE

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
sim <- 1:10
obs <- 1:10
gof(sim, obs)

sim <- 2:11
obs <- 1:10
gof(sim, obs)

##################
# Loading daily streamflows of the Ega River (Spain), from 1961 to 1970
data(EgaEnEstellaQts)
obs <- EgaEnEstellaQts

# Generating a simulated daily time series, initially equal to the observed series
sim <- obs 

# Getting the numeric goodness of fit for the "best" (unattainable) case
gof(sim=sim, obs=obs)

# Randomly changing the first 2000 elements of 'sim', by using a normal distribution 
# with mean 10 and standard deviation equal to 1 (default of 'rnorm').
sim[1:2000] <- obs[1:2000] + rnorm(2000, mean=10)

# Getting the new numeric goodness of fit
gof(sim=sim, obs=obs)

# Storing a matrix object with all the GoFs:
g <-  gof(sim, obs)

# Getting only the RMSE
g[4,1]
g["RMSE",]

## Not run: 
# Writing all the GoFs into a TXT file
write.table(g, "GoFs.txt", col.names=FALSE, quote=FALSE)

# Getting the graphical representation of 'obs' and 'sim' along with the 
# numeric goodness of fit 
ggof(sim=sim, obs=obs)

## End(Not run)