comp_accu_freq: Compute accuracy metrics of current classification results.

Description Usage Arguments Details Value References See Also Examples

View source: R/comp_accu.R

Description

comp_accu_freq computes a list of current accuracy metrics from the 4 essential frequencies (hi, mi, fa, cr) that constitute the current confusion matrix and are contained in freq.

Usage

1
2
comp_accu_freq(hi = freq$hi, mi = freq$mi, fa = freq$fa,
  cr = freq$cr, w = 0.5)

Arguments

hi

The number of hits hi (or true positives).

mi

The number of misses mi (or false negatives).

fa

The number of false alarms fa (or false positives).

cr

The number of correct rejections cr (or true negatives).

w

The weighting parameter w (from 0 to 1) for computing weighted accuracy wacc. Default: w = .50 (i.e., yielding balanced accuracy bacc).

Details

Currently computed accuracy metrics include:

  1. acc: Overall accuracy as the proportion (or probability) of correctly classifying cases or of dec_cor cases:

    acc = dec_cor/N = (hi + cr)/(hi + mi + fa + cr)

    Values range from 0 (no correct prediction) to 1 (perfect prediction).

  2. wacc: Weighted accuracy, as a weighted average of the sensitivity sens (aka. hit rate HR, TPR, power or recall) and the the specificity spec (aka. TNR) in which sens is multiplied by a weighting parameter w (ranging from 0 to 1) and spec is multiplied by w's complement (1 - w):

    wacc = (w * sens) + ((1 - w) * spec)

    If w = .50, wacc becomes balanced accuracy bacc.

  3. mcc: The Matthews correlation coefficient (with values ranging from -1 to +1):

    mcc = ((hi * cr) - (fa * mi)) / sqrt((hi + fa) * (hi + mi) * (cr + fa) * (cr + mi))

    A value of mcc = 0 implies random performance; mcc = 1 implies perfect performance.

    See Wikipedia: Matthews correlation coefficient for additional information.

  4. f1s: The harmonic mean of the positive predictive value PPV (aka. precision) and the sensitivity sens (aka. hit rate HR, TPR, power or recall):

    f1s = 2 * (PPV * sens) / (PPV + sens)

    See Wikipedia: F1 score for additional information.

Notes:

Value

A list accu containing current accuracy metrics.

References

Consult Wikipedia: Confusion matrix for additional information.

See Also

accu for all accuracy metrics; comp_accu_prob computes exact accuracy metrics from probabilities; num for basic numeric parameters; freq for current frequency information; txt for current text settings; pal for current color settings; popu for a table of the current population.

Other metrics: accu, acc, comp_accu_prob, comp_acc, comp_err, err

Other functions computing probabilities: comp_FDR, comp_FOR, comp_NPV, comp_PPV, comp_accu_prob, comp_acc, comp_comp_pair, comp_complement, comp_complete_prob_set, comp_err, comp_fart, comp_mirt, comp_ppod, comp_prob_freq, comp_prob, comp_sens, comp_spec

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
comp_accu_freq()  # => accuracy metrics for freq of current scenario
comp_accu_freq(hi = 1, mi = 2, fa = 3, cr = 4)  # medium accuracy, but cr > hi

# Extreme cases:
comp_accu_freq(hi = 1, mi = 1, fa = 1, cr = 1)  # random performance
comp_accu_freq(hi = 0, mi = 0, fa = 1, cr = 1)  # random performance: wacc and f1s are NaN
comp_accu_freq(hi = 1, mi = 0, fa = 0, cr = 1)  # perfect accuracy/optimal performance
comp_accu_freq(hi = 0, mi = 1, fa = 1, cr = 0)  # zero accuracy/worst performance, but see f1s
comp_accu_freq(hi = 1, mi = 0, fa = 0, cr = 0)  # perfect accuracy, but see wacc and mcc

# Effects of w:
comp_accu_freq(hi = 3, mi = 2, fa = 1, cr = 4, w = 1/2)  # equal weights to sens and spec
comp_accu_freq(hi = 3, mi = 2, fa = 1, cr = 4, w = 2/3)  # more weight to sens
comp_accu_freq(hi = 3, mi = 2, fa = 1, cr = 4, w = 1/3)  # more weight to spec

## Contrasting comp_accu_freq and comp_accu_prob:
# (a) comp_accu_freq (based on rounded frequencies):
freq1 <- comp_freq(N = 10, prev = 1/3, sens = 2/3, spec = 3/4)   # => hi = 2, mi = 1, fa = 2, cr = 5
accu1 <- comp_accu_freq(freq1$hi, freq1$mi, freq1$fa, freq1$cr)  # => accu1 (based on rounded freq).
# accu1
#
# (b) comp_accu_prob (based on probabilities):
accu2 <- comp_accu_prob(prev = 1/3, sens = 2/3, spec = 3/4)      # => exact accu (based on prob).
# accu2
all.equal(accu1, accu2)  # => 4 differences!
#
# (c) comp_accu_freq (exact values, i.e., without rounding):
freq3 <- comp_freq(N = 10, prev = 1/3, sens = 2/3, spec = 3/4, round = FALSE)
accu3 <- comp_accu_freq(freq3$hi, freq3$mi, freq3$fa, freq3$cr)  # => accu3 (based on EXACT freq).
# accu3
all.equal(accu2, accu3)  # => TRUE (qed).

riskyr documentation built on Jan. 3, 2019, 1:06 a.m.