metrics: Model performance metrics

In nestedcv: Nested Cross-Validation with 'glmnet' and 'caret'

Model performance metrics

Description

Returns model metrics from nestedcv models. Extended metrics including

Usage

metrics(object, extra = FALSE, innerCV = FALSE, positive = 2)

Arguments

object	A 'nestcv.glmnet', 'nestcv.train', 'nestcv.SuperLearner' or 'outercv' object.
extra	Logical whether additional performance metrics are gathered for classification models: area under precision recall curve (PR.AUC, binary classification only), Cohen's kappa, F1 score, Matthews correlation coefficient (MCC).
innerCV	Whether to calculate metrics for inner CV folds. Only available for 'nestcv.glmnet' and 'nestcv.train' objects.
positive	For binary classification, either an integer 1 or 2 for the level of response factor considered to be 'positive' or 'relevant', or a character value for that factor. This affects the F1 score. See caret::confusionMatrix().

Details

Area under precision recall curve is estimated by trapezoidal estimation using MLmetrics::PRAUC().

For multi-class classification models, Matthews correlation coefficient is calculated using Gorodkin's method. Multi-class F1 score (macro F1) is calculated as the arithmetic mean of the class-wise F1 scores.

Value

A named numeric vector of performance metrics.

References

Gorodkin, J. (2004). Comparing two K-category assignments by a K-category correlation coefficient. Computational Biology and Chemistry. 28 (5): 367–374.

See Also

mcc()

nestedcv documentation built on April 4, 2025, 2:21 a.m.