mewtoApp: Model Evaluation with Weighted Threshold Optimization App

In alexandrumonahov/mewto: Model Evaluation with Weighted Threshold Optimization

Description

This function launches a Shiny application where the user can interactively manipulate the threshold used in binary classification and view the associated metrics, confusion matrix, ROC and PR curves. The app also allows for optimal threshold calculation according to a weighted version of Youden's J-statistic.

Usage

mewtoApp(actuals, probabilities)

Arguments

actuals	Data of factor type with two levels: "yes" for positive and "no" for negative.
probabilities	Data of numeric type which should represent the probabilities of realization of the positive category.

Details

In the calculation of the optimal threshold, a weighted version of Youden's J-statistic (Youden, 1950) is employed. The optimal cut-off is the threshold that maximizes the distance to the identity (diagonal) line. The function maximizes the metric (w * sensitivity + (1 - w) * specificity), where w is the "weight" parameter chosen through the second slider. After setting the desired value on the optimal threshold slider, the user must press the button "Set" to update the threshold slider above with the optimal value. Metrics will automatically be recalculated based on the user's selection.

Author(s)

Alexandru Monahov, [e-mail]

Examples

mewtoApp(actuals, probabilities)

alexandrumonahov/mewto documentation built on March 21, 2021, 4:36 a.m.