pre_metrics: Compute a confusion matrix recall, precision and the amount...

In wanjarast/accelerateR: Process acceleration data

Compute a confusion matrix recall, precision and the amount of unknown behaviours for a model

Description

Convenient function to calculate a confisuion matrix, recall and precision for a machine learning model

Usage

pre_metrics(predicted = ... , expected = ... , uncertain = NULL)

Arguments

predicted	Vector with predicted behaviours (output of the prediction from a trained model used on new data)
expected	Vector with expected behaviours which are already known (e.g. from observation)
uncertain	if the prediction process is designed to put all unceartain predictions in the same class, the class name can be specified here in "..."

Details

None of the provided column names should have a "-" in the name otherwise the function will not work.

Takes a vector of predicted behaviours and compares it to a vector of known behaviours. A confusion matrix is computed. The recall and precision are computed after Powers et al. 2011. In cases were unknown behaviours are considered it computes the proportion of unknown behaviours in contrast to the total number of behaviours recoreded. In cases where the model performes badly and one or more classes are not predicted, the function will only output the confusion matrix.

Value

The output is a list:

confusion_matrix	A confusion matrix from the predicted and expected behaviours of class "table"
metrics	Recall, precision and accuracy for every behaviour class in a matrix
kappa	Cohen's kappa value
uncertain_predictions	proportion of uncertain predictions in relation to to total number of redictions as a numeric value

Author(s)

Wanja Rast

References

Powers, David Martin. "Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation." (2011).

Examples

predicted <- c(rep("A",7),"B","C","C",rep("B",8),"A","A",rep("C",9),"B")
expected <- c(rep("A",10),rep("B",10),rep("C",10))

pre_metrics(predicted = predicted , expected = expected)

In case the predictions contain unidentifiable behaviours:

predicted <- c(rep("A",7),"B","C","other",rep("B",8),"A","other",rep("C",9),"B")
expected <- c(rep("A",10),rep("B",10),rep("C",10))

pre_metrics(predicted = predicted , expected = expected , uncertain = "other")

wanjarast/accelerateR documentation built on June 21, 2022, 3:29 p.m.