thresholder  R Documentation 
This function uses the resampling results from a train
object to generate performance statistics over a set of probability
thresholds for twoclass problems.
thresholder(x, threshold, final = TRUE, statistics = "all")
x 
A 
threshold 
A numeric vector of candidate probability thresholds between [0,1]. If the class probability corresponding to the first level of the outcome is greater than the threshold, the data point is classified as that level. 
final 
A logical: should only the final tuning parameters
chosen by 
statistics 
A character vector indicating which statistics to
calculate. See details below for possible choices; the default value

The argument statistics
designates the statistics to compute
for each probability threshold. One or more of the following statistics can
be selected:
Sensitivity
Specificity
Pos Pred Value
Neg Pred Value
Precision
Recall
F1
Prevalence
Detection Rate
Detection Prevalence
Balanced Accuracy
Accuracy
Kappa
J
Dist
For a description of these statistics (except the last two), see the
documentation of confusionMatrix
. The last two statistics
are Youden's J statistic and the distance to the best possible cutoff (i.e.
perfect sensitivity and specificity.
A data frame with columns for each of the tuning parameters
from the model along with an additional column called
prob_threshold
for the probability threshold. There are
also columns for summary statistics averaged over resamples with
column names corresponding to the input argument statistics
.
## Not run:
set.seed(2444)
dat < twoClassSim(500, intercept = 10)
table(dat$Class)
ctrl < trainControl(method = "cv",
classProbs = TRUE,
savePredictions = "all",
summaryFunction = twoClassSummary)
set.seed(2863)
mod < train(Class ~ ., data = dat,
method = "rda",
tuneLength = 4,
metric = "ROC",
trControl = ctrl)
resample_stats < thresholder(mod,
threshold = seq(.5, 1, by = 0.05),
final = TRUE)
ggplot(resample_stats, aes(x = prob_threshold, y = J)) +
geom_point()
ggplot(resample_stats, aes(x = prob_threshold, y = Dist)) +
geom_point()
ggplot(resample_stats, aes(x = prob_threshold, y = Sensitivity)) +
geom_point() +
geom_point(aes(y = Specificity), col = "red")
## End(Not run)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.