Description Usage Arguments Details Value References See Also Examples

This function computes PU-performance metrics useful for model selection. NOTE: The metrics are prone of uncertainty due to the nature PU-data and the user should check the plausibility of any automatic selection.

1 |

`data` |
a data frame or matrix with columns |

`lev` |
a character vector of factors levels for the response (default is |

`model` |
a character string for the model name (as taken form the method argument of train (default is |

Calculates performance metrics based on positive and unlabeled data.

The following metrics are calculated and stored:

tpr true positive rate (from P-data)

ppp probability of positive prediction (from U-data)

puAuc The area under the ROC curve (form PU-data), see Phillips et al. (2008)

puF

`(tpr^2)/ppp`

, see Liu et al (2003)negD01

`-sqrt( (1-tpr)^2 + ppp^2 )`

, i.e. the distance between the point c(0,1) and the c(ppp, tpr)

From the labeled positive data the true positive rate (TPR), and thus the false negative rate (FNR), can be estimated directly.
Unfortunately it is not possible to estimate the true negative rate (TNR), and thus neither the false positive rate (FPR).
But we can also estimate the probability of positive prediction (PPP), i.e. the fraction of unlabeled samples predicted as positives from our model.
The TPR and the PPP are useful and intuitive quantities.
Assume that the i) TPR can be estimated accurately and ii) we need to select one of several candidate models which all exhibit the same TPR.
We can select the model with the lowest PPP because for a given TPR (and FNR), it leads to a higher TNR (and lower FPR). See `plot_PPPvsTPR`

Furthermore, the `puAuc`

A vector of performance estimates.

Phillips, Steven J. and Dud\'ik, Miroslav (2008):
Modeling of species distributions with Maxent: new extensions and a comprehensive evaluation.
Ecography, 31, 2.

Liu, Bing and Dai, Yang and Li, Xiaoli and Lee, Wee Sun and Yu, Philip S. (2003):
Building text classifiers using positive and unlabeled examples.
In: Intl. Conf. on Data Mining, 2003.

1 2 3 4 5 6 7 8 9 10 |

Embedding an R snippet on your website

Add the following code to your website.

For more information on customizing the embed code, read Embedding Snippets.