accuracy: Measures of Model Accuracy

In SDMTools: Species Distribution Modelling Tools: Tools for processing data associated with species distribution modelling exercises

Description

accuracy estimates six measures of accuracy for presence-absence or presence-psuedoabsence data. These include AUC, ommission rates, sensitivity, specificity, proportion correctly identified and Kappa.

Note: this method will exclude any missing data.

Usage

accuracy(obs, pred, threshold = 0.5)

Arguments

obs	a vector of observed values which must be 0 for absences and 1 for occurrences
pred	a vector of the same length as obs representing the predicted values. Values must be between 0 & 1 prepresenting a likelihood.
threshold	this can be: a) a single value representing a single threshold between 0 & 1; b) a vector of threshold values between 0 & 1; OR c) an integer value representing the number of equal interval threshold values between 0 & 1

Value

a data.frame with seven columns:

threshold	the threshold values representing each row of data
AUC	the AUC given the defined threshold value
ommission.rate	the ommission rate as a proportion of true occurrences misidentified given the defined threshold value
sensitivity	the sensitivity given the defined threshold value
specificity	the specificity given the defined threshold value
prop.correct	the proportion of the presence and absence records correctly identified given the defined threshold value
Kappa	the Kappa statistic of the model given the defined threshold value

Author(s)

Jeremy VanDerWal [e-mail]

See Also

auc, Kappa, omission, sensitivity, specificity, prop.correct, confusion.matrix

Examples

#create some data
obs = c(sample(c(0,1),20,replace=TRUE),NA); obs = obs[order(obs)]
pred = runif(length(obs),0,1); pred = pred[order(pred)]

#calculate accuracy of the model with a single threshold value
accuracy(obs,pred,threshold=0.5)

#calculate accuracy given several defined thresholds
accuracy(obs,pred,threshold=c(0.33,0.5,0.66))

#calculate accuracy given a number of equal interval thresholds
accuracy(obs,pred,threshold=20)

Example output

  threshold       AUC omission.rate sensitivity specificity prop.correct
1       0.5 0.8636364     0.2727273   0.7272727           1         0.85
      Kappa
1 0.7058824
Warning message:
In accuracy(obs, pred, threshold = 0.5) :
  1 data points removed due to missing data
  threshold       AUC omission.rate sensitivity specificity prop.correct
1      0.33 0.8888889     0.0000000   1.0000000   0.7777778         0.90
2      0.50 0.8636364     0.2727273   0.7272727   1.0000000         0.85
3      0.66 0.8181818     0.3636364   0.6363636   1.0000000         0.80
      Kappa
1 0.7938144
2 0.7058824
3 0.6116505
Warning message:
In accuracy(obs, pred, threshold = c(0.33, 0.5, 0.66)) :
  1 data points removed due to missing data
    threshold       AUC omission.rate sensitivity specificity prop.correct
1  0.00000000 0.5000000     0.0000000   1.0000000   0.0000000         0.55
2  0.05263158 0.5000000     0.0000000   1.0000000   0.0000000         0.55
3  0.10526316 0.7222222     0.0000000   1.0000000   0.4444444         0.75
4  0.15789474 0.7222222     0.0000000   1.0000000   0.4444444         0.75
5  0.21052632 0.7222222     0.0000000   1.0000000   0.4444444         0.75
6  0.26315789 0.7222222     0.0000000   1.0000000   0.4444444         0.75
7  0.31578947 0.8333333     0.0000000   1.0000000   0.6666667         0.85
8  0.36842105 0.9444444     0.0000000   1.0000000   0.8888889         0.95
9  0.42105263 1.0000000     0.0000000   1.0000000   1.0000000         1.00
10 0.47368421 0.9090909     0.1818182   0.8181818   1.0000000         0.90
11 0.52631579 0.8181818     0.3636364   0.6363636   1.0000000         0.80
12 0.57894737 0.8181818     0.3636364   0.6363636   1.0000000         0.80
13 0.63157895 0.8181818     0.3636364   0.6363636   1.0000000         0.80
14 0.68421053 0.8181818     0.3636364   0.6363636   1.0000000         0.80
15 0.73684211 0.7272727     0.5454545   0.4545455   1.0000000         0.70
16 0.78947368 0.6818182     0.6363636   0.3636364   1.0000000         0.65
17 0.84210526 0.6363636     0.7272727   0.2727273   1.0000000         0.60
18 0.89473684 0.5909091     0.8181818   0.1818182   1.0000000         0.55
19 0.94736842 0.5909091     0.8181818   0.1818182   1.0000000         0.55
20 1.00000000 0.5000000     1.0000000   0.0000000   1.0000000         0.45
       Kappa
1  0.0000000
2  0.0000000
3  0.4680851
4  0.4680851
5  0.4680851
6  0.4680851
7  0.6875000
8  0.8979592
9  1.0000000
10 0.8019802
11 0.6116505
12 0.6116505
13 0.6116505
14 0.6116505
15 0.4285714
16 0.3396226
17 0.2523364
18 0.1666667
19 0.1666667
20 0.0000000
Warning message:
In accuracy(obs, pred, threshold = 20) :
  1 data points removed due to missing data

SDMTools documentation built on Jan. 11, 2020, 9:23 a.m.