get_roc: Calculate receiver operating characteristic (ROC) scores for...

In XSLmodels: Run and Optimize Cross-Situational Word Learning Models

Calculate receiver operating characteristic (ROC) scores for a model association Matrix

Description

This function computes receiver operating characteristic (ROC) scores for a given model association matrix. It evaluates the performance of the model at various thresholds, providing metrics like f-scores, precision, and recall. The function can operate with a range of thresholds and optionally consider a gold lexicon for calculating true and false positives and negatives. The result is a comprehensive assessment of model performance over a continuum of classification thresholds.

This function computes the maximum F-score from the receiver operating characteristic (ROC) scores of a model. It leverages the get_roc() function to calculate the ROC scores and then extracts the highest F-score, providing a concise metric for the best classification performance of the model.

Usage

get_roc(m, thresholds = seq(0, 1, 0.01), gold_lexicon = NULL)

plot_roc(m, thresholds = seq(0, 1, 0.01), gold_lexicon = NULL)

get_roc_max(m, thresholds = seq(0, 1, 0.01), gold_lexicon = NULL)

Arguments

m	A matrix representing the knowledge matrix with words as rows and referents as columns.
thresholds	Vector of thresholds to use.
gold_lexicon	Optional; a data frame or list where each row/element represents a word-object pair in the gold lexicon. If provided, it is used to calculate true positives, false positives, and false negatives.

Value

A tibble with columns for the threshold, f-score, precision, and recall.

A single numeric value representing the maximum F-score obtained from the ROC scores of the model.

Examples

dat <- xsl_datasets[[1]]
x <- xsl_run(baseline(), dat)
mat <- x$fits[[1]]$matrix
lex <- list(words = rep(1:18), objects = rep(1:18))
get_roc(mat, gold_lexicon = lex)
plot_roc(mat, gold_lexicon = lex)
get_roc_max(mat, gold_lexicon = lex)

XSLmodels documentation built on Oct. 18, 2026, 5:08 p.m.