comp_accu_freq | R Documentation |
comp_accu_freq
computes a list of current accuracy metrics
from the 4 essential frequencies (hi
,
mi
, fa
, cr
)
that constitute the current confusion matrix and
are contained in freq
.
comp_accu_freq(hi = freq$hi, mi = freq$mi, fa = freq$fa, cr = freq$cr, w = 0.5)
hi |
The number of hits |
mi |
The number of misses |
fa |
The number of false alarms |
cr |
The number of correct rejections |
w |
The weighting parameter |
Currently computed accuracy metrics include:
acc
: Overall accuracy as the proportion (or probability)
of correctly classifying cases or of dec_cor
cases:
acc = dec_cor/N = (hi + cr)/(hi + mi + fa + cr)
Values range from 0 (no correct prediction) to 1 (perfect prediction).
wacc
: Weighted accuracy, as a weighted average of the
sensitivity sens
(aka. hit rate HR
, TPR
,
power
or recall
)
and the the specificity spec
(aka. TNR
)
in which sens
is multiplied by a weighting parameter w
(ranging from 0 to 1) and spec
is multiplied by
w
's complement (1 - w)
:
wacc = (w * sens) + ((1 - w) * spec)
If w = .50
, wacc
becomes balanced accuracy bacc
.
mcc
: The Matthews correlation coefficient (with values ranging from -1 to +1):
mcc = ((hi * cr) - (fa * mi)) / sqrt((hi + fa) * (hi + mi) * (cr + fa) * (cr + mi))
A value of mcc = 0
implies random performance; mcc = 1
implies perfect performance.
See Wikipedia: Matthews correlation coefficient for additional information.
f1s
: The harmonic mean of the positive predictive value PPV
(aka. precision
)
and the sensitivity sens
(aka. hit rate HR
,
TPR
, power
or recall
):
f1s = 2 * (PPV * sens) / (PPV + sens)
See Wikipedia: F1 score for additional information.
Notes:
Accuracy metrics describe the correspondence of decisions (or predictions) to actual conditions (or truth).
There are several possible interpretations of accuracy:
as probabilities (i.e., acc
being the proportion of correct classifications,
or the ratio dec_cor
/N
),
as frequencies (e.g., as classifying a population of N
individuals into cases of dec_cor
vs. dec_err
),
as correlations (e.g., see mcc
in accu
).
Computing exact accuracy values based on probabilities (by comp_accu_prob
) may differ from
accuracy values computed from (possibly rounded) frequencies (by comp_accu_freq
).
When frequencies are rounded to integers (see the default of round = TRUE
in comp_freq
and comp_freq_prob
) the accuracy metrics computed by
comp_accu_freq
correspond to these rounded values.
Use comp_accu_prob
to obtain exact accuracy metrics from probabilities.
A list accu
containing current accuracy metrics.
Consult Wikipedia: Confusion matrix for additional information.
accu
for all accuracy metrics;
comp_accu_prob
computes exact accuracy metrics from probabilities;
num
for basic numeric parameters;
freq
for current frequency information;
txt
for current text settings;
pal
for current color settings;
popu
for a table of the current population.
Other metrics:
accu
,
acc
,
comp_accu_prob()
,
comp_acc()
,
comp_err()
,
err
Other functions computing probabilities:
comp_FDR()
,
comp_FOR()
,
comp_NPV()
,
comp_PPV()
,
comp_accu_prob()
,
comp_acc()
,
comp_comp_pair()
,
comp_complement()
,
comp_complete_prob_set()
,
comp_err()
,
comp_fart()
,
comp_mirt()
,
comp_ppod()
,
comp_prob_freq()
,
comp_prob()
,
comp_sens()
,
comp_spec()
comp_accu_freq() # => accuracy metrics for freq of current scenario comp_accu_freq(hi = 1, mi = 2, fa = 3, cr = 4) # medium accuracy, but cr > hi # Extreme cases: comp_accu_freq(hi = 1, mi = 1, fa = 1, cr = 1) # random performance comp_accu_freq(hi = 0, mi = 0, fa = 1, cr = 1) # random performance: wacc and f1s are NaN comp_accu_freq(hi = 1, mi = 0, fa = 0, cr = 1) # perfect accuracy/optimal performance comp_accu_freq(hi = 0, mi = 1, fa = 1, cr = 0) # zero accuracy/worst performance, but see f1s comp_accu_freq(hi = 1, mi = 0, fa = 0, cr = 0) # perfect accuracy, but see wacc and mcc # Effects of w: comp_accu_freq(hi = 3, mi = 2, fa = 1, cr = 4, w = 1/2) # equal weights to sens and spec comp_accu_freq(hi = 3, mi = 2, fa = 1, cr = 4, w = 2/3) # more weight to sens comp_accu_freq(hi = 3, mi = 2, fa = 1, cr = 4, w = 1/3) # more weight to spec ## Contrasting comp_accu_freq and comp_accu_prob: # (a) comp_accu_freq (based on rounded frequencies): freq1 <- comp_freq(N = 10, prev = 1/3, sens = 2/3, spec = 3/4) # => hi = 2, mi = 1, fa = 2, cr = 5 accu1 <- comp_accu_freq(freq1$hi, freq1$mi, freq1$fa, freq1$cr) # => accu1 (based on rounded freq). # accu1 # # (b) comp_accu_prob (based on probabilities): accu2 <- comp_accu_prob(prev = 1/3, sens = 2/3, spec = 3/4) # => exact accu (based on prob). # accu2 all.equal(accu1, accu2) # => 4 differences! # # (c) comp_accu_freq (exact values, i.e., without rounding): freq3 <- comp_freq(N = 10, prev = 1/3, sens = 2/3, spec = 3/4, round = FALSE) accu3 <- comp_accu_freq(freq3$hi, freq3$mi, freq3$fa, freq3$cr) # => accu3 (based on EXACT freq). # accu3 all.equal(accu2, accu3) # => TRUE (qed).
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.