ht_auc_table: Hand-Till multiclass AUC with pairwise results tables

In multiUS: Functions for the Courses Multivariate Analysis and Computer Intensive Methods

Hand-Till multiclass AUC with pairwise results tables

Description

Computes the Hand-Till multiclass AUC for nominal outcomes by averaging symmetric pairwise AUCs across all category pairs. For each pair '(i, j)', the function computes:

A(i|j)

using the predicted probability for class 'i', and

A(j|i)

using the predicted probability for class 'j'. The pairwise Hand-Till AUC is

(A(i|j) + A(j|i))/2.

Usage

ht_auc_table(outcome, probs, categories = NULL, na_rm = TRUE)

Arguments

outcome	A vector containing the empirical/observed values of nominal dependent variable. May be a factor, character, numeric, or integer vector.
probs	A numeric matrix or data frame of predicted probabilities with one row per observation and one column per outcome category.
categories	Optional vector giving the category values in the same order as the columns of 'probs'. If 'NULL', the factor levels of 'outcome' are used when 'outcome' is a factor; otherwise sorted unique values are used.
na_rm	If 'TRUE', observations with missing values in 'outcome' or 'probs' are removed listwise.

Details

A weighted version is also reported using weights n_i n_j. This weighted version is a practical extension and not the original Hand-Till definition.

Value

A list with class '"ht_auc_result"' containing:

basic_info

Data frame with outcome variable summary information.

pairwise

Data frame with one row per category pair and columns 'category_i', 'category_j', 'n_i', 'n_j', 'A_i_given_j', 'A_j_given_i', 'pairwise_auc', 'weight'.

summary

Data frame with unweighted and weighted Hand-Till AUC.

ht_unweighted

Numeric scalar, mean of pairwise Hand-Till AUCs.

ht_weighted

Numeric scalar, weighted mean using 'n_i * n_j'.

categories

Category vector actually used.

n_valid

Number of valid cases used.

References

Hand, D. J., & Till, R. J. (2001). A simple generalisation of the area under the ROC curve for multiple class classification problems. Machine Learning, 45(2), 171-186.

Examples

# Example with artificial probabilities:
# outcome <- c("A", "A", "B", "B", "C", "C")
# probs <- rbind(
#   c(.65, .20, .15),
#   c(.55, .25, .20),
#   c(.20, .60, .20),
#   c(.15, .55, .30),
#   c(.10, .25, .65),
#   c(.10, .20, .70)
# )
# colnames(probs) <- c("A", "B", "C")
# res <- ht_auc_table(outcome, probs)
# res$pairwise
# res$summary

multiUS documentation built on April 27, 2026, 3:04 a.m.