evaluate.predictions: Evaluate prediction results

In zellerlab/siamcat: Statistical Inference of Associations between Microbial Communities And host phenoTypes

Evaluate prediction results

Description

This function compares the predictions (from [make.predictions]) and true labels for all samples and evaluates the results.

Usage

evaluate.predictions(siamcat, verbose = 1)

Arguments

siamcat	object of class siamcat-class
verbose	integer, control output: 0 for no output at all, 1 for only information about progress and success, 2 for normal level of information and 3 for full debug information, defaults to 1

Value

object of class siamcat-class with the slot eval_data filled

Binary classification problems

This function calculates several metrics for the predictions in the pred_matrix-slot of the siamcat-class-object. The Area Under the Receiver Operating Characteristic (ROC) Curve (AU-ROC) and the Precision-Recall Curve will be evaluated and the results will be saved in the eval_data-slot of the supplied siamcat-class- object. The eval_data-slot contains a list with several entries:

• $roc - average ROC-curve across repeats or a single ROC-curve on complete dataset (see roc);

• $auroc - AUC value for the average ROC-curve;

• $prc - list containing the positive predictive value (precision) and true positive rate (recall) values used to plot the mean PR curve;

• $auprc - AUC value for the mean PR curve;

• $ev - list containing for different decision thresholds the number of false positives, false negatives, true negatives, and true positives.

For the case of repeated cross-validation, the function will additionally return

• $roc.all - list of roc objects (see roc) for every repeat;

• $auroc.all - vector of AUC values for the ROC curves for every repeat;

• $prc.all - list of PR curves for every repeat;

• $auprc.all - vector of AUC values for the PR curves for every repeat;

• $ev.all - list of ev lists (see above) for every repeat.

Regression problems

This function calculates several metrics for the evaluation of predictions and will store the results in the eval_data-slot of the supplied siamcat-class objects. The eval_data-slot will contain:

• r2 - the mean R squared value across repeats or a single R-squared value on the complete dataset;

• mae - them mean absolute error of the predictions;

• mse - the mean squared error of the predictions.

For the case of repeated cross-validation, the function will additionally compute all three of these measures for the individual cross-validation repeats and will store the results in the eval_data slot as r2.all, mae.all, and mse.all.

Examples

data(siamcat_example)

siamcat_evaluated <- evaluate.predictions(siamcat_example)

zellerlab/siamcat documentation built on Feb. 1, 2024, 2:21 a.m.