run_benchmark: Run microbenchmark with specified tools and test sets

In prcbench: Testing Workbench for Precision-Recall Curves

Run microbenchmark with specified tools and test sets

Description

The run_benchmark function runs microbenchmark for specified tools and test datasets

Usage

run_benchmark(testset, toolset, times = 5, unit = "ms", use_sys_time = FALSE)

Arguments

testset	A character vector to specify a test set generated by create_testset.
toolset	A character vector to specify a tool set generated by create_toolset.
times	The number of iteration used in microbenchmark.
unit	A single string to specify the unit used in summary.microbenchmark.
use_sys_time	A Boolean value to specify system.time is used instead of summary.microbenchmark.

Details

The timing of the sklearn tool is not comparable with the timings of the tools written in R. Every call crosses the R/Python boundary and converts the input and output vectors, and that overhead is counted as part of the measurement. On a small test set it often dominates the curve calculation itself, so the sklearn row measures the cost of the round trip to Python rather than the speed of the scikit-learn algorithm. See Toolsklearn for the tool itself, and run_evalcurve for an evaluation that this does not affect.

Value

A data frame of microbenchmark results with additional columns.

See Also

create_testset to generate a test dataset. create_toolset to generate a tool set. microbenchmark for benchmarking details.

Examples

## Not run: 
## Benchmarking for b10 and i10 test sets and crv5, auc5, and def5 tool sets
testset <- create_testset("bench", c("b10", "i10"))
toolset <- create_toolset(set_names = "def5")
res1 <- run_benchmark(testset, toolset)
res1

## End(Not run)

prcbench documentation built on Sept. 26, 2026, 5:06 p.m.