evaluate.predictions: Evaluate prediction results

In KonradZych/SIAMCAT: Statistical Inference of Associations between Microbial Communities And host phenoTypes

Description

This function takes the correct labels and predictions for all samples and evaluates the results using the

• Area Under the Receiver Operating Characteristic (ROC) Curve (AU-ROC)

• and the Precision-Recall Curve (PR)

as metric. Predictions can be supplied either for a single case or as matrix after resampling of the dataset.

Prediction results are usually produced with the function make.predictions.

Usage

evaluate.predictions(siamcat, verbose = 1)

Arguments

siamcat	object of class siamcat-class
verbose	control output: 0 for no output at all, 1 for only information about progress and success, 2 for normal level of information and 3 for full debug information, defaults to 1

Details

This functions calculates for the predictions in the pred_matrix -slot of the siamcat-class-object several metrices. The Area Under the Receiver Operating Characteristic (ROC) Curve (AU-ROC) and the Precision-Recall Curve will be evaluated and the results will be saved in the eval_data-slot of the supplied siamcat-class-object. The eval_data-slot contains a list with several entries:

• $roc.average average ROC-curve across repeats or a single ROC-curve on complete dataset;

• $auc.average AUC value for the average ROC-curve;

• $ev.list list of length(num.folds), containing for different decision thresholds the number of false positives, false negatives, true negatives, and true positives;

• $pr.list list of length(num.folds), containing the positive predictive value (precision) and true positive rate (recall) values used to plot the PR curves.

For the case of repeated cross-validation, the function will additonally return

• $roc.all list of roc objects (see roc) for every repeat;

• $aucspr vector of AUC values for the PR curves for every repeat;

• $auc.all vector of AUC values for the ROC curves for every repeat.

Value

object of class siamcat-class with the slot eval_data filled

Examples

    data(siamcat_example)
    # simple working example
    siamcat_evaluated <- evaluate.predictions(siamcat_example)

KonradZych/SIAMCAT documentation built on May 17, 2019, 6:20 p.m.