multModEv: Multiple model evaluation

Description Usage Arguments Value Author(s) See Also Examples

View source: R/multModEv.R

Description

If you have a list of GLM model objects (created, e.g., with the multGLM function of the 'fuzzySim' R-Forge package), or a data frame with presence-absence data and the corresponding predicted values for a set of species, you can use the multModEv function to get a set of evaluation measures for all models simultaneously, as long as they all have the same sample size.

Usage

1
2
3
multModEv(models = NULL, obs.data = NULL, pred.data = NULL,
measures = modEvAmethods("multModEv"), standardize = FALSE, thresh = NULL,
bin.method = NULL, verbosity = 0, ...)

Arguments

models

a list of model object(s) of class "glm", all applied to the same data set. Evaluation is based on the cases included in the models.

obs.data

a data frame with observed (training or test) binary data. This argument is ignored if models is provided.

pred.data

a data frame with the corresponding predicted (training or test) values, with both rows and columns in the same order as in obs.data. This argument is ignored if models is provided. Note that, for calibration measures (based on HLfit or MillerCalib), the results are only valid if the input predictions represent probability.

measures

character vector of the evaluation measures to calculate. The default is all implemented measures, which you can check by typing modEvAmethods("multModEv"). But beware: calibration measures (i.e., HL and Miller) are only valid if your predicted values reflect actual presence probability (not favourability, habitat suitability or others); you should exclude them otherwise.

standardize

logical, whether to standardize measures that vary between -1 and 1 to the 0-1 scale (see standard01). The default is FALSE.

thresh

argument to pass to threshMeasures if any of measures is calculated by that function. The default is NULL, but a valid method must be specified if any of measures is threshold-based - i.e., any of those in modEvAmethods("threshMeasures").

bin.method

the method with which to divide the data into groups or bins, for calibration or reliability measures such as HLfit. The default is NULL, but a valid method must be specified if measures includes "HL" or "HL.p". Type modEvAmethods("getBins") for available options), and see HLfit and getBins for more information.

verbosity

integer specifying the amount of messages or warnings to display. Defaults to 0, but can also be 1 or 2 for more messages from the functions within.

...

optional arguments to pass to HLfit (if "HL" or "HL.p" are included in measures), namely n.bins, fixed.bin.size, min.bin.size, min.prob.interval or quantile.type.

Value

A data frame with the value of each evaluation measure for each model.

Author(s)

A. Marcia Barbosa

See Also

threshMeasures

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
data(rotif.mods)

eval1 <- multModEv(models = rotif.mods$models[1:6], thresh = 0.5, 
bin.method = "n.bins", fixed.bin.size = TRUE)

head(eval1)


eval2 <- multModEv(models = rotif.mods$models[1:6], thresh = "preval", 
measures = c("AUC", "CCR", "Sensitivity", "TSS"))

head(eval2)


# you can also calculate evaluation measures for a set of 
# observed vs predicted data, rather than from model objects:

obses <- sapply(rotif.mods$models, `[[`, "y")
preds <- sapply(rotif.mods$models, `[[`, "fitted.values")

eval3 <- multModEv(obs.data = obses[ , 1:4], pred.data = preds[ , 1:4], 
thresh = "preval", bin.method = "prob.bins")

head(eval3)

modEvA documentation built on May 29, 2017, 10:47 a.m.