comp_accu_freq: Compute accuracy metrics of current classification results.
In riskyr: Rendering Risk Literacy more Transparent

Description Usage Arguments Details Value References See Also Examples

comp_accu_freq computes a list of current accuracy metrics from the 4 essential frequencies (hi, mi, fa, cr) that constitute the current confusion matrix and are contained in freq.

1 2	comp_accu_freq(hi = freq$hi, mi = freq$mi, fa = freq$fa, cr = freq$cr, w = 0.5)

`hi`	The number of hits `hi` (or true positives).
`mi`	The number of misses `mi` (or false negatives).
`fa`	The number of false alarms `fa` (or false positives).
`cr`	The number of correct rejections `cr` (or true negatives).
`w`	The weighting parameter `w` (from 0 to 1) for computing weighted accuracy `wacc`. Default: `w = .50` (i.e., yielding balanced accuracy `bacc`).

Currently computed accuracy metrics include:

acc: Overall accuracy as the proportion (or probability) of correctly classifying cases or of dec_cor cases:

acc = dec_cor/N = (hi + cr)/(hi + mi + fa + cr)

Values range from 0 (no correct prediction) to 1 (perfect prediction).
wacc: Weighted accuracy, as a weighted average of the sensitivity sens (aka. hit rate HR, TPR, power or recall) and the the specificity spec (aka. TNR) in which sens is multiplied by a weighting parameter w (ranging from 0 to 1) and spec is multiplied by w's complement (1 - w):

wacc = (w * sens) + ((1 - w) * spec)

If w = .50, wacc becomes balanced accuracy bacc.
mcc: The Matthews correlation coefficient (with values ranging from -1 to +1):

mcc = ((hi * cr) - (fa * mi)) / sqrt((hi + fa) * (hi + mi) * (cr + fa) * (cr + mi))

A value of mcc = 0 implies random performance; mcc = 1 implies perfect performance.

See Wikipedia: Matthews correlation coefficient for additional information.
f1s: The harmonic mean of the positive predictive value PPV (aka. precision) and the sensitivity sens (aka. hit rate HR, TPR, power or recall):

f1s = 2 * (PPV * sens) / (PPV + sens)

See Wikipedia: F1 score for additional information.

Notes:

Accuracy metrics describe the correspondence of decisions (or predictions) to actual conditions (or truth).

There are several possible interpretations of accuracy:
1. as probabilities (i.e., acc being the proportion of correct classifications, or the ratio dec_cor/N),
2. as frequencies (e.g., as classifying a population of N individuals into cases of dec_cor vs. dec_err),
3. as correlations (e.g., see mcc in accu).
Computing exact accuracy values based on probabilities (by comp_accu_prob) may differ from accuracy values computed from (possibly rounded) frequencies (by comp_accu_freq).

When frequencies are rounded to integers (see the default of round = TRUE in comp_freq and comp_freq_prob) the accuracy metrics computed by comp_accu_freq correspond to these rounded values. Use comp_accu_prob to obtain exact accuracy metrics from probabilities.

A list accu containing current accuracy metrics.

Consult Wikipedia: Confusion matrix for additional information.

accu for all accuracy metrics; comp_accu_prob computes exact accuracy metrics from probabilities; num for basic numeric parameters; freq for current frequency information; txt for current text settings; pal for current color settings; popu for a table of the current population.

Other metrics: accu, acc, comp_accu_prob, comp_acc, comp_err, err

Other functions computing probabilities: comp_FDR, comp_FOR, comp_NPV, comp_PPV, comp_accu_prob, comp_acc, comp_comp_pair, comp_complement, comp_complete_prob_set, comp_err, comp_fart, comp_mirt, comp_ppod, comp_prob_freq, comp_prob, comp_sens, comp_spec

comp_accu_freq()  # => accuracy metrics for freq of current scenario
comp_accu_freq(hi = 1, mi = 2, fa = 3, cr = 4)  # medium accuracy, but cr > hi

# Extreme cases:
comp_accu_freq(hi = 1, mi = 1, fa = 1, cr = 1)  # random performance
comp_accu_freq(hi = 0, mi = 0, fa = 1, cr = 1)  # random performance: wacc and f1s are NaN
comp_accu_freq(hi = 1, mi = 0, fa = 0, cr = 1)  # perfect accuracy/optimal performance
comp_accu_freq(hi = 0, mi = 1, fa = 1, cr = 0)  # zero accuracy/worst performance, but see f1s
comp_accu_freq(hi = 1, mi = 0, fa = 0, cr = 0)  # perfect accuracy, but see wacc and mcc

# Effects of w:
comp_accu_freq(hi = 3, mi = 2, fa = 1, cr = 4, w = 1/2)  # equal weights to sens and spec
comp_accu_freq(hi = 3, mi = 2, fa = 1, cr = 4, w = 2/3)  # more weight to sens
comp_accu_freq(hi = 3, mi = 2, fa = 1, cr = 4, w = 1/3)  # more weight to spec

## Contrasting comp_accu_freq and comp_accu_prob:
# (a) comp_accu_freq (based on rounded frequencies):
freq1 <- comp_freq(N = 10, prev = 1/3, sens = 2/3, spec = 3/4)   # => hi = 2, mi = 1, fa = 2, cr = 5
accu1 <- comp_accu_freq(freq1$hi, freq1$mi, freq1$fa, freq1$cr)  # => accu1 (based on rounded freq).
# accu1
#
# (b) comp_accu_prob (based on probabilities):
accu2 <- comp_accu_prob(prev = 1/3, sens = 2/3, spec = 3/4)      # => exact accu (based on prob).
# accu2
all.equal(accu1, accu2)  # => 4 differences!
#
# (c) comp_accu_freq (exact values, i.e., without rounding):
freq3 <- comp_freq(N = 10, prev = 1/3, sens = 2/3, spec = 3/4, round = FALSE)
accu3 <- comp_accu_freq(freq3$hi, freq3$mi, freq3$fa, freq3$cr)  # => accu3 (based on EXACT freq).
# accu3
all.equal(accu2, accu3)  # => TRUE (qed).