evalm: evalm: Evaluate Machine Learning Models in R

In MLeval: Machine Learning Model Evaluation

Description

evalm is for machine learning model evaluation in R. The function can accept the Caret 'train' function results to evaluate machine learning predictions or a data frame of probabilities and ground truth labels can be passed in to evaluate. Probability data must be column1: probability group1 (column named as your group name 1), column2: probability group2 (column named as your group name 2), column3: observation labels (column named 'obs'), column4: Group, e.g. different models (column named 'Group'), optional to include if different models are combined horizontally.

Usage

evalm(list1, gnames = NULL, title = "", cols = NULL,
  silent = FALSE, rlinethick = 1.25, fsize = 12.5,
  dlinecol = "grey", dlinethick = 0.75, bins = 6, optimise = "INF",
  percent = 95, showplots = TRUE, positive = NULL, plots = c("prg",
  "pr", "r", "cc"))

Arguments

list1	List or data frame: List of Caret results objects from train, or a single train results object, or a data frame of probabilities and observed labels
gnames	Character vector: A vector of group names for the fit objects
title	Character string: A title for the ROC plot
cols	Character vector: A vector of colours for the group or groups
silent	Logical flag: whether to hide messages (default=FALSE)
rlinethick	Numerical value: Thickness of the ROC curve line
fsize	Numerical value: Font size for the ROC curve plots
dlinecol	Character string: Colour of the diagonal line
dlinethick	Numerical value: Thickness of the diagonal line
bins	Numerical value: Number of bins for calibration curve
optimise	Character string: Metric by which to select the operating point (INF, MCC, or F1)
percent	Numerical value: percentage for the confidence intervals (default = 95)
showplots	Logical flag: whether to show plots or not
positive	Character string: Name of the positive group (will effect PR metrics)
plots	Character vector: which plots to show: r = roc, pr = proc, prg = precision recall gain, cc = calibration curve

Value

List containing: 1) A ggplot2 ROC curve object for printing 2) A ggplot2 PROC object for printing 3) A ggplot2 PRG curve for printing 4) Optimised results according to defined metric 5) P cut-off of 0.5 standard results

Examples

r <- evalm(fit)

Example output

***MLeval: Machine Learning Model Evaluation***
Input: caret train function object
Not averaging probs.
Group 1 type: cv
Observations: 105
Number of groups: 1
Observations per group: 105
Positive: R
Negative: M
Group: Group 1
Positive: 49
Negative: 56
***Performance Metrics***
Group 1 Optimal Informedness = 0.737244897959184
Group 1 AUC-ROC = 0.92

MLeval documentation built on Feb. 12, 2020, 9:07 a.m.