score_models: Score models by suitable metrics

In ModelOriented/forester: Quick and Simple Tools for Training and Testing of Tree-Based Models

Score models by suitable metrics

Description

Score models by suitable metrics

Usage

score_models(
  models,
  predictions,
  observed,
  data,
  type,
  time = NULL,
  status = NULL,
  metrics = "auto",
  sort_by = "auto",
  metric_function = NULL,
  metric_function_name = NULL,
  metric_function_decreasing = TRUE,
  engine = NULL,
  tuning = NULL
)

Arguments

models	A list of models trained by 'train_models()' function.
predictions	A list of predictions of every engine from the test data.
observed	A vector of true values from the test data.
data	A data for models created by 'prepare_data()' function, used for Brier score calculations.
type	A string, determines if the future task is 'binary_clf', 'regression', 'survival', or 'multiclass'.
time	A string that indicates a time column name for survival analysis task. Either y, or pair: time, status can be used.
status	A string that indicates a status column name for survival analysis task. Either y, or pair: time, status can be used.
metrics	A vector of metrics names. By default param set for 'auto', most important metrics are returned. For 'all' all metrics are returned. For 'NULL' no metrics returned but still sorted by 'sort_by'. The metrics available for the binary classification are: 'auc', 'f1', 'recall', 'precision', 'accuracy', 'sensitivity', 'specificity', 'balanced_accuracy', for the regression: 'mse', 'rmse', 'r2', 'mad', 'mae', for the survival analysis: 'Brier Score', 'Concordance Index (CIN)', and for the multiclass classification: 'accuracy', 'micro_averaged_precision', 'micro_averaged_recall', 'micro_averaged_f1', 'macro_averaged_precision', 'macro_averaged_recall', 'macro_averaged_f1', 'weighted_averaged_precision', 'weighted_averaged_recall', 'weighted_averaged_f1'.
sort_by	String with name of metric to sort by. For 'auto' models going to be sorted by 'rmse' for regression and 'accuracy' for both classification tasks.
metric_function	The self-created function. It should look like name(predictions, observed) and return the numeric value. In case of using 'metrics' param with value other than 'auto' or 'all', is needed to use value 'metric_function' in order to see given metric in report. If 'sort_by' is equal to 'auto' models are sorted by 'metric_function'.
metric_function_name	The name of the column with values of param 'metric_function'. By default 'metric_function_name' is 'metric_function'.
metric_function_decreasing	A logical value indicating how metric_function should be sorted. 'TRUE' by default.
engine	A vector of strings containing information of engine in 'models' list.
tuning	A vector of strings containing information of tuning method in 'models' list.

Value

A data.frame with 'no.' - number of model from models, 'engine' - name of the model from models, other metrics columns.

ModelOriented/forester documentation built on June 6, 2024, 7:29 a.m.