R/prcbench.R

In prcbench: Testing Workbench for Precision-Recall Curves

#' prcbench: A package to provide a testing workbench for
#'   precision-recall curves
#'
#' The prcbench package provides four categories of important functions:
#' tool interface, test data interface, benchmarking, and curve evaluation.
#'
#' @section Tool interface:
#' The \code{\link{create_toolset}} function creates a common interface for
#'   seven different tools that calculate Precision-Recall curves. These tools
#'   are \href{https://ipa-tys.github.io/ROCR/}{ROCR},
#'   \href{http://mark.goadrich.com/programs/AUC/}{AUCCalculator},
#'   \href{https://cran.r-project.org/package=PerfMeas}{PerfMeas},
#'   \href{https://cran.r-project.org/package=PRROC}{PRROC},
#'   \href{https://cran.r-project.org/package=precrec}{precrec},
#'   \href{https://yardstick.tidymodels.org/}{yardstick}, and
#'   \href{https://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_recall_curve.html}{scikit-learn}.
#'
#' The \code{sklearn} tool is calculated by a standalone Python module that is
#'   bundled with \code{prcbench} and derived from the scikit-learn source
#'   code. It requires \code{reticulate}, a working Python installation and
#'   \code{numpy}. Without them it returns a flat dummy curve rather than
#'   raising an error, so the predefined tool sets that contain it stay
#'   usable.
#'
#' The \code{\link{create_usrtool}} function helps users to make the same
#'  interface of the predefined ones for their own tools.

#' @section Test data interface:
#' The \code{\link{create_testset}} function creates two different types of test
#'   data sets. The first type is for benchmarking, and the second type is for
#'   curve evaluation.
#'
#' The \code{\link{create_usrdata}} function helps users to make their own test
#'   data sets.
#'
#' @section Benchmarking:
#' The \code{\link{run_benchmark}} function takes a tool set and a test data set
#'   and run \code{\link[microbenchmark]{microbenchmark}} for them.
#'
#' The timing of the \code{sklearn} tool includes the cost of crossing the
#'   R/Python boundary, so it is not comparable with the timings of the tools
#'   written in R.
#'
#' @section Curve evaluation:
#' The \code{\link{run_evalcurve}} function takes a tool set and a test data set
#'   and evaluates the accuracy of Precision-Recall curves for them.
#'
#' @docType package
#' @name prcbench
#'
#' @useDynLib prcbench, .registration = TRUE
#' @importFrom Rcpp sourceCpp
#' @importFrom R6 R6Class
#' @importFrom ggplot2 autoplot
#' @importFrom stats runif
#' @importFrom methods slot
#' @importFrom stats aggregate
#' @importFrom stats approx
#' @importFrom methods is
#' @importFrom memoise memoise
#'
"_PACKAGE"

#' C1: Pre-calculated Precision-Recall curve
#'
#' A list contains scores, labels, and pre-calculated recall and precision
#' values as x and y.
#'
#' @format A list with 5 items.
#' \describe{
#'   \item{scores}{input scores}
#'   \item{labels}{input labels}
#'   \item{bp_x}{pre-calculated recall values for curve evaluation}
#'   \item{bp_y}{pre-calculated precision values for curve evaluation}
#'   \item{tp_x}{x position for displaying the test result in a plot}
#'   \item{tp_y}{y position for displaying the test result in a plot}
#' }
#'
#' @docType data
#' @keywords datasets
#' @name C1DATA
#' @usage data(C1DATA)
NULL

#' C2: Pre-calculated Precision-Recall curve
#'
#' A list contains scores, labels, and pre-calculated recall and precision
#' values as x and y.
#'
#' @format See \code{\link{C1DATA}}.
#'
#' @docType data
#' @keywords datasets
#' @name C2DATA
#' @usage data(C2DATA)
NULL

#' C3: Pre-calculated Precision-Recall curve
#'
#' A list contains scores, labels, and pre-calculated recall and precision
#' values as x and y.
#'
#' @format See \code{\link{C1DATA}}.
#'
#' @docType data
#' @keywords datasets
#' @name C3DATA
#' @usage data(C3DATA)
NULL

#' C4: Pre-calculated Precision-Recall curve
#'
#' A list contains scores, labels, and pre-calculated recall and precision
#' values as x and y.
#'
#' @format See \code{\link{C1DATA}}.
#'
#' @docType data
#' @keywords datasets
#' @name C4DATA
#' @usage data(C4DATA)
NULL