accuracy: Performance measures for regression and classification models

In fda.usc: Functional Data Analysis and Utilities for Statistical Computing

Performance measures for regression and classification models

Description

cat2meas and tab2meas calculate the measures for a multiclass classification model.
pred2meas calculates the measures for a regression model.

Usage

cat2meas(yobs, ypred, measure = "accuracy", cost = rep(1, nlevels(yobs)))

tab2meas(tab, measure = "accuracy", cost = rep(1, nrow(tab)))

pred.MSE(yobs, ypred)

pred.RMSE(yobs, ypred)

pred.MAE(yobs, ypred)

pred2meas(yobs, ypred, measure = "RMSE")

Arguments

yobs	A vector of the labels, true class or observed response. Can be numeric, character, or factor.
ypred	A vector of the predicted labels, predicted class or predicted response. Can be numeric, character, or factor.
measure	Type of measure, see details section.
cost	Cost value by class (only for input factors).
tab	Confusion matrix (Contingency table: observed class by rows, predicted class by columns).

Details

• cat2meas compute tab=table(yobs,ypred) and calls tab2meas function.

• tab2meas function computes the following measures (see measure argument) for a binary classification model:

  • accuracy: Proportion of correct predictions. \frac{TP + TN}{TP + TN + FP + FN}

  • sensitivity, TPrate, recall: True Positive Rate or recall. \frac{TP}{TP + FN}

  • precision: Positive Predictive Value. \frac{TP}{TP + FP}

  • specificity, TNrate: True Negative Rate. \frac{TN}{TN + FP}

  • FPrate: False Positive Rate. \frac{FP}{TN + FP}

  • FNrate: False Negative Rate. \frac{FN}{TP + FN}

  • Fmeasure: Harmonic mean of precision and recall. \frac{2}{\frac{1}{\text{recall}} + \frac{1}{\text{precision}}}

  • Gmean: Geometric Mean of recall and specificity. \sqrt{\left(\frac{TP}{TP + FN}\right) \cdot \left(\frac{TN}{TN + FP}\right)}

  • kappa: Cohen's Kappa index. Kappa = \frac{P_o - P_e}{1 - P_e} where P_o is the proportion of observed agreement, \frac{TP + TN}{TP + TN + FP + FN}, and P_e is the proportion of agreement expected by chance, \frac{(TP + FP)(TP + FN) + (TN + FN)(TN + FP)}{(TP + TN + FP + FN)^2}.

  • cost: Weighted accuracy, calculated as \frac{\sum (\text{diag(tab)} / \text{rowSums(tab)} \cdot \text{cost})}{\sum(\text{cost})}

  • IOU: Mean Intersection over Union. \frac{TP}{TP + FN + FP}

  • IOU4class: Intersection over Union by class level. \frac{TP}{TP + FN + FP}#'

• pred2meas function computes the following measures of error, usign the measure argument, for observed and predicted vectors:

  • MSE: Mean squared error, \frac{\sum{(ypred- yobs)^2}}{n} .

  • RMSE: Root mean squared error \sqrt{\frac{\sum{(ypred- yobs)^2}}{n} }.

  • MAE: Mean Absolute Error, \frac{\sum |yobs - ypred|}{n}.

See Also

Other performance: weights4class()

fda.usc documentation built on April 4, 2025, 4:35 a.m.