modStats  R Documentation 
Function to calculate common numerical model evaluation statistics with flexible conditioning.
modStats(
mydata,
mod = "mod",
obs = "obs",
statistic = c("n", "FAC2", "MB", "MGE", "NMB", "NMGE", "RMSE", "r", "COE", "IOA"),
type = "default",
rank.name = NULL,
...
)
mydata 
A data frame. 
mod 
Name of a variable in 
obs 
Name of a variable in 
statistic 
The statistic to be calculated. See details below for a description of each. 
type 
It is also possible to choose More than one type can be considered e.g. 
rank.name 
Simple model ranking can be carried out if 
... 
Arguments passed on to

This function is under development and currently provides some common model evaluation statistics. These include (to be mathematically defined later):
n
, the number of complete pairs of data.
FAC2
, fraction of predictions within a factor of two.
MB
, the mean bias.
MGE
, the mean gross error.
NMB
, the normalised mean bias.
NMGE
, the normalised mean gross error.
RMSE
, the root mean squared error.
r
, the Pearson correlation coefficient. Note, can also supply and
argument method
e.g. method = "spearman"
. Also returned is the
P value of the correlation coefficient, P
, which may present as 0
for
very low values.
COE
, the Coefficient of Efficiency based on Legates and
McCabe (1999, 2012). There have been many suggestions for measuring model
performance over the years, but the COE is a simple formulation which is easy
to interpret.
A perfect model has a COE = 1. As noted by Legates and McCabe although the COE has no lower bound, a value of COE = 0.0 has a fundamental meaning. It implies that the model is no more able to predict the observed values than does the observed mean. Therefore, since the model can explain no more of the variation in the observed values than can the observed mean, such a model can have no predictive advantage.
For negative values of COE, the model is less effective than the observed mean in predicting the variation in the observations.
IOA
, the
Index of Agreement based on Willmott et al. (2011), which spans between 1
and +1 with values approaching +1 representing better model performance.
An IOA of 0.5, for example, indicates that the sum of the errormagnitudes is one half of the sum of the observeddeviation magnitudes. When IOA = 0.0, it signifies that the sum of the magnitudes of the errors and the sum of the observeddeviation magnitudes are equivalent. When IOA = 0.5, it indicates that the sum of the errormagnitudes is twice the sum of the perfect modeldeviation and observeddeviation magnitudes. Values of IOA near 1.0 can mean that the modelestimated deviations about O are poor estimates of the observed deviations; but, they also can mean that there simply is little observed variability  so some caution is needed when the IOA approaches 1.
All statistics are based on complete pairs of mod
and obs
.
Conditioning is possible through setting type
, which can be a vector
e.g. type = c("weekday", "season")
.
Returns a data frame with model evaluation statistics.
David Carslaw
Legates DR, McCabe GJ. (1999). Evaluating the use of goodnessoffit measures in hydrologic and hydroclimatic model validation. Water Resources Research 35(1): 233241.
Legates DR, McCabe GJ. (2012). A refined index of model performance: a rejoinder, International Journal of Climatology.
Willmott, C.J., Robeson, S.M., Matsuura, K., 2011. A refined index of model performance. International Journal of Climatology.
Other model evaluation functions:
TaylorDiagram()
,
conditionalEval()
,
conditionalQuantile()
## the example below is somewhat artificial  assuming the observed
## values are given by NOx and the predicted values by NO2.
modStats(mydata, mod = "no2", obs = "nox")
## evaluation stats by season
modStats(mydata, mod = "no2", obs = "nox", type = "season")
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.