Evaluate a model

In SDMtune: Species Distribution Model Selection

knitr::opts_chunk$set(collapse = TRUE,
                      comment = "#>",
                      fig.align = "center")
options(knitr.table.format = "html")

default_model <- SDMtune:::bm_maxent
files <- list.files(path = file.path(system.file(package = "dismo"), "ex"), 
                    pattern = "grd", 
                    full.names = TRUE)
predictors <- terra::rast(files)

Intro

In the previous articles you have learned how to prepare the data for the analysis, how to train a model and how to make predictions using SDMtune. In this article you will learn how to evaluate your model using three different metrics.

SDMtune implements three evaluation metrics:

• AUC: Area Under the ROC curve [@Fielding1997]
• TSS: True Skill Statistic [@ALLOUCHE2006]
• AICc: Akaike Information Criterion corrected for small sample size [@Warren2011a]

We will compute the value of the metrics on the training dataset, using the default model that we trained in a previous article.

AUC

As usually we first load the SDMtune package:

library(SDMtune)

The AUC can be calculated using the function auc():

auc(default_model)

We can also plot the ROC curve using the function plotROC():

plotROC(default_model)

TSS

The TSS is computed with the function tss():

tss(default_model)

AICc

For the AICc we use the function aicc(). In this case we need to pass to the env argument the 'predictors' raster object that we created in the first article:

aicc(default_model, 
     env = predictors)

Try yourself

Try to compute the three metrics using a model trained using the Maxnet method.

Conclusion

In this article you have learned:

• how to calculate the AUC;
• how to plot the ROC curve;
• how to calculate the TSS;
• how to calculate the AICc.

In the next article you will learn two different strategies that can be used to correctly evaluate the model performance.

References

SDMtune documentation built on July 9, 2023, 6:03 p.m.