metrics: Performance Metrics

Description Usage Arguments See Also

Description

Compute measures of agreement between observed and predicted responses.

Usage

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
accuracy(
  observed,
  predicted = NULL,
  cutoff = MachineShop::settings("cutoff"),
  ...
)

auc(
  observed,
  predicted = NULL,
  metrics = c(MachineShop::tpr, MachineShop::fpr),
  stat = MachineShop::settings("stat.Curve"),
  ...
)

brier(observed, predicted = NULL, ...)

cindex(observed, predicted = NULL, ...)

cross_entropy(observed, predicted = NULL, ...)

f_score(
  observed,
  predicted = NULL,
  cutoff = MachineShop::settings("cutoff"),
  beta = 1,
  ...
)

fnr(observed, predicted = NULL, cutoff = MachineShop::settings("cutoff"), ...)

fpr(observed, predicted = NULL, cutoff = MachineShop::settings("cutoff"), ...)

kappa2(
  observed,
  predicted = NULL,
  cutoff = MachineShop::settings("cutoff"),
  ...
)

npv(observed, predicted = NULL, cutoff = MachineShop::settings("cutoff"), ...)

ppv(observed, predicted = NULL, cutoff = MachineShop::settings("cutoff"), ...)

pr_auc(observed, predicted = NULL, ...)

precision(
  observed,
  predicted = NULL,
  cutoff = MachineShop::settings("cutoff"),
  ...
)

recall(
  observed,
  predicted = NULL,
  cutoff = MachineShop::settings("cutoff"),
  ...
)

roc_auc(observed, predicted = NULL, ...)

roc_index(
  observed,
  predicted = NULL,
  cutoff = MachineShop::settings("cutoff"),
  f = function(sensitivity, specificity) (sensitivity + specificity)/2,
  ...
)

rpp(observed, predicted = NULL, cutoff = MachineShop::settings("cutoff"), ...)

sensitivity(
  observed,
  predicted = NULL,
  cutoff = MachineShop::settings("cutoff"),
  ...
)

specificity(
  observed,
  predicted = NULL,
  cutoff = MachineShop::settings("cutoff"),
  ...
)

tnr(observed, predicted = NULL, cutoff = MachineShop::settings("cutoff"), ...)

tpr(observed, predicted = NULL, cutoff = MachineShop::settings("cutoff"), ...)

weighted_kappa2(observed, predicted = NULL, power = 1, ...)

gini(observed, predicted = NULL, ...)

mae(observed, predicted = NULL, ...)

mse(observed, predicted = NULL, ...)

msle(observed, predicted = NULL, ...)

r2(observed, predicted = NULL, distr = NULL, ...)

rmse(observed, predicted = NULL, ...)

rmsle(observed, predicted = NULL, ...)

Arguments

observed

observed responses; or confusion, performance curve, or resample result containing observed and predicted responses.

predicted

predicted responses if not contained in observed.

cutoff

numeric (0, 1) threshold above which binary factor probabilities are classified as events and below which survival probabilities are classified.

...

arguments passed to or from other methods.

metrics

list of two performance metrics for the calculation [default: ROC metrics].

stat

function or character string naming a function to compute a summary statistic at each cutoff value of resampled metrics in performance curves, or NULL for resample-specific metrics.

beta

relative importance of recall to precision in the calculation of f_score [default: F1 score].

f

function to calculate a desired sensitivity-specificity tradeoff.

power

power to which positional distances of off-diagonals from the main diagonal in confusion matrices are raised to calculate weighted_kappa2.

distr

character string specifying a distribution with which to estimate the observed survival mean in the total sum of square component of r2. Possible values are "empirical" for the Kaplan-Meier estimator, "exponential", "extreme", "gaussian", "loggaussian", "logistic", "loglogistic", "lognormal", "rayleigh", "t", or "weibull". Defaults to the distribution that was used in predicting mean survival times.

See Also

metricinfo, performance


MachineShop documentation built on June 18, 2021, 9:06 a.m.