repeatedCV_test: Repeated k-Fold Cross-Validation with Custom Test Metrics

In powerPLS: Power Analysis for PLS Classification

Repeated k-Fold Cross-Validation with Custom Test Metrics

Description

This function performs repeated k-fold cross-validation and computes a selected performance metric across all repetitions and folds. It allows for different types of performance tests, such as MCC, sensitivity, specificity, R2, F1, and more.

Usage

repeatedCV_test(
  data,
  labels,
  k_folds = 5,
  repeats = 3,
  A = 1,
  test_type = "mccTest",
  seed = 1234
)

Arguments

data	A data frame or matrix of features (predictor variables).
labels	A vector of class labels corresponding to the rows of data.
k_folds	An integer specifying the number of cross-validation folds (default = 5).
repeats	An integer specifying the number of times the cross-validation is repeated (default = 3).
A	number of score components
test_type	A character string specifying the type of test to use. Options include: 'mccTest' for Matthews Correlation Coefficient (MCC), 'sensitivityTest' for Sensitivity, 'specificityTest' for Specificity, 'R2Test' for R-squared, 'scoreTest' for Score, 'F1Test' for F1 Score, 'FMTest' for Fowlkes-Mallows Index (FM), 'AUCTest' for Area Under the Curve (AUC), 'dQ2Test' for dQ2. Default is 'mccTest'.
seed	An integer for setting the random seed to ensure reproducibility (default = 1234).

Value

A numeric value representing the average performance metric across the outer folds.

Examples

datas <- simulatePilotData(nvar = 30, clus.size = c(15,15),m = 6,nvar_rel = 5,A = 1)
data <- datas$X
labels <- datas$Y
mean_mcc <- repeatedCV_test(data, labels, A = 1, test_type = 'mccTest')
cat('Mean MCC:', mean_mcc, '\n')

mean_score <- repeatedCV_test(data, labels, A = 1, test_type = 'scoreTest')
cat('Mean Sensitivity:', mean_score, '\n')

powerPLS documentation built on April 4, 2025, 1:57 a.m.