p4: P4-metric

In metrica: Prediction Performance Metrics

P4-metric

Description

It estimates the P4-metric for a nominal/categorical predicted-observed dataset.

Usage

p4(
  data = NULL,
  obs,
  pred,
  pos_level = 2,
  tidy = FALSE,
  na.rm = TRUE,
  atom = FALSE
)

Arguments

data	(Optional) argument to call an existing data frame containing the data.
obs	Vector with observed values (character | factor).
pred	Vector with predicted values (character | factor).
pos_level	Integer, for binary cases, indicating the order (1|2) of the level corresponding to the positive. Generally, the positive level is the second (2) since following an alpha-numeric order, the most common pairs are (Negative | Positive), (0 | 1), (FALSE | TRUE). Default : 2.
tidy	Logical operator (TRUE/FALSE) to decide the type of return. TRUE returns a data.frame, FALSE returns a list; Default : FALSE.
na.rm	Logic argument to remove rows with missing values (NA). Default is na.rm = TRUE.
atom	Logical operator (TRUE/FALSE) to decide if the estimate is made for each class (atom = TRUE) or at a global level (atom = FALSE); Default : FALSE. When dataset is "binomial" atom does not apply.

Details

The P4-metric it is a metric designed for binary classifiers. It is estimated from precision, recall, specificity, and npv (negative predictive value). The P4 it was designed to address criticism against the F-score, so it may be perceived as its extension. Unfortunately, it has not been generalized yet for multinomial cases.

For binomial/binary cases,

p4 = \frac{(4 x TP x TN)} {(4 x TP x TN + (TP + TN) x FP + FN)}

Or:

p4 = \frac{4} {\frac{1}{precision} + \frac{1}{recall} + \frac{1}{specificity} + \frac{1}{npv} }

The P4 metric has not been generalized for multinomial cases. The P4 metric is bounded between 0 and 1. The closer to 1 the better, which will require precision, recall, specificity and npv being close to 1. Values towards zero indicate low performance, which could be the product of only one of the four conditional probabilities being close to 0.

For the formula and more details, see online-documentation

Value

an object of class numeric within a list (if tidy = FALSE) or within a ⁠data frame⁠ (if tidy = TRUE).

References

Sitarz, M. (2023). Extending F1 metric, probabilistic approach. _Adv. Artif. Intell. Mach. Learn., 3 (2):1025-1038._\Sexpr[results=rd]{tools:::Rd_expr_doi("10.54364/AAIML.2023.1161")}

Examples

set.seed(123)
# Two-class
binomial_case <- data.frame(labels = sample(c("True","False"), 100, replace = TRUE), 
predictions = sample(c("True","False"), 100, replace = TRUE))
# Multi-class
multinomial_case <- data.frame(labels = sample(c("Red","Blue", "Green"), 100, replace = TRUE),
predictions = sample(c("Red","Blue", "Green"), 100, replace = TRUE)    )

# Get P4-metric estimate for two-class case
p4(data = binomial_case, obs = labels, pred = predictions, tidy = TRUE)

metrica documentation built on June 30, 2024, 5:07 p.m.