View source: R/cv_performance.R
performance | R Documentation |
Calculate performance measures from the predictions results.
performance(object, prediction_results, y_ground_truth, type = NULL, ...)
object |
An R6 object of class |
prediction_results |
An object of class |
y_ground_truth |
A vector with the ground truth of the predicted data. |
type |
A character to select a pre-defined set of metrics for "binary"
and "regression" tasks. If not specified (default: |
... |
A list. Further arguments required to compute the performance metrics. |
The performance metric has to be specified in the object
that is used to
carry out the experiment, i.e., MLCrossValidation or
MLNestedCV.
Please note that the option return_models = TRUE
must be set in the
experiment class in order to be able to compute the predictions, which are
required to conduct the calculation of the performance.
The function returns a data.table with the computed performance metric of each fold.
dataset <- do.call(
cbind,
c(sapply(paste0("col", 1:6), function(x) {
rnorm(n = 500)
},
USE.NAMES = TRUE,
simplify = FALSE
),
list(target = sample(0:1, 500, TRUE))
))
fold_list <- splitTools::create_folds(
y = dataset[, 7],
k = 3,
type = "stratified",
seed = 123
)
glm_optimization <- mlexperiments::MLCrossValidation$new(
learner = LearnerGlm$new(),
fold_list = fold_list,
seed = 123
)
glm_optimization$learner_args <- list(family = binomial(link = "logit"))
glm_optimization$predict_args <- list(type = "response")
glm_optimization$performance_metric_args <- list(positive = "1")
glm_optimization$performance_metric <- list(
auc = metric("auc"), sensitivity = metric("sensitivity"),
specificity = metric("specificity")
)
glm_optimization$return_models <- TRUE
# set data
glm_optimization$set_data(
x = data.matrix(dataset[, -7]),
y = dataset[, 7]
)
cv_results <- glm_optimization$execute()
# predictions
preds <- mlexperiments::predictions(
object = glm_optimization,
newdata = data.matrix(dataset[, -7]),
na.rm = FALSE,
ncores = 2L,
type = "response"
)
# performance
mlexperiments::performance(
object = glm_optimization,
prediction_results = preds,
y_ground_truth = dataset[, 7],
positive = "1"
)
# performance - binary
mlexperiments::performance(
object = glm_optimization,
prediction_results = preds,
y_ground_truth = dataset[, 7],
type = "binary",
positive = "1"
)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.