opdisDownsampling: Optimal Distribution Preserving Down-Sampling of Bio-Medical...

In opdisDownsampling: Optimal Distribution Preserving Down-Sampling of Bio-Medical Data

Optimal Distribution Preserving Down-Sampling of Bio-Medical Data

Description

The package provides functions for optimal distribution-preserving down-sampling of large (bio-medical) data sets. It draws statistically representative subsets of data while preserving the class proportions and original data distribution.

Usage

opdisDownsampling(Data, Cls, Size, Seed = "simple", nTrials = 1000,
  TestStat = "ad", MaxCores = getOption("mc.cores", 2L),
  PCAimportance = FALSE, JobSize = 0, verbose = FALSE)

Arguments

Data	Numeric data as a vector, matrix, or data frame. Each row represents an instance, each column a variable.
Cls	Optional vector with class labels for each instance in Data. If missing, all instances are treated as belonging to the same class.
Size	The number (integer) or proportion (0<Size<1) of instances to draw from the dataset. The reduction is class proportional and aims to preserve the variable distributions.
Seed	Seed value. Options: "auto" for seed recovery, "simple" to generate and report a seed using the current RNG state, or an integer for exact reproducibility. Use integers for systematic testing and fully reproducible analyses.
nTrials	Number of random sampling trials used to find the optimal subset (default: 1000).
TestStat	Character string defining the statistical test used to assess distribution similarity. Available options are: "ad": Anderson–Darling statistic "kuiper": Kuiper statistic "cvm": Cramér–von Mises statistic "wass": Wasserstein distance "dts": Distributional Transform Statistic "ks": Kolmogorov–Smirnov statistic "kld": Kullback–Leibler divergence (via KullbLeiblKLD2()) "amrdd": Average Mean Root of Distributional Differences (via amrdd()) "euc": Euclidean distance (via EucDist()) "nent": Absolute normalized entropy difference (via abs_norm_entropy_diff())
MaxCores	Maximum number of CPU cores to use for parallel computing (default is value stored in getOption("mc.cores"), or 2 if missing).
PCAimportance	Logical; if TRUE, only variables deemed important by principal component analysis are used in computing similarity statistics.
JobSize	Integer specifying the number of trials to process in each chunk. If 0 (default), no chunking is applied. If NULL, an automatic chunk size is calculated based on data dimensions, number of trials, available system memory, and number of processor cores. A positive integer manually sets the chunk size.
verbose	Logical; if TRUE, prints diagnostic information about chunk-size selection, including data dimensions, estimated memory usage, and the chosen chunking strategy. Useful for understanding memory usage patterns and debugging performance issues.

Details

Chunked processing can be used to reduce memory usage when dealing with large datasets or high numbers of trials. Set JobSize = NULL to enable automatic memory-aware chunk-size calculation. The automatic chunking strategy considers:

• Data size, defined by number of rows and columns

• Available system memory, detected on Linux systems

• Number of processor cores

• Number of trials to perform

Set JobSize = 0 to process all trials in a single batch. Set JobSize to a positive integer to manually define the number of trials processed per chunk.

Variable Selection Method:

If PCAimportance = TRUE, PCA-based variable selection is used. Variables are ranked by their loadings in the first principal components, and variables with higher importance scores are used for distribution comparisons.

Value

Returns a list with the following elements:

ReducedData	Down-sampled data set (as data frame or matrix) including only the selected instances.
RemovedData	Data not included in the sample.
ReducedInstances	Row indices (or names) of the selected instances from the original data set.
RemovedInstances	Row indices (or names) of the unselected instances from the original data set.

Author(s)

Jorn Lotsch

References

Lotsch, J., Malkusch, S., Ultsch, A. (2021):\ Optimal distribution-preserving downsampling of large biomedical data sets.\ PLoS ONE 16(8): e0255838. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1371/journal.pone.0255838")}

Examples

## Example: Down-sample the Iris dataset to 50 points
data(iris)
Iris50percent <- opdisDownsampling(Data = iris[,1:4], Cls = as.integer(iris$Species),
  Size = 50, Seed = 42, MaxCores = 1)

## Example: Down-sample with custom chunk size and verbose output
data(iris)
Iris50percent <- opdisDownsampling(Data = iris[,1:4], Cls = as.integer(iris$Species),
  Size = 50, Seed = 42, MaxCores = 1, JobSize = 25, verbose = TRUE)

## Example: Use PCA-based variable selection
data(iris)
Iris_pca <- opdisDownsampling(Data = iris[,1:4], Cls = as.integer(iris$Species),
  Size = 50, Seed = 42, PCAimportance = TRUE, MaxCores = 1)

## Example: Memory-efficient processing of large dataset with many trials
## Not run: 
# For large datasets, automatic chunking can reduce memory usage
LargeDataSample <- opdisDownsampling(Data = large_dataset,
  Size = 0.1, Seed = 42, nTrials = 5000, JobSize = NULL, verbose = TRUE)

## End(Not run)

opdisDownsampling documentation built on June 25, 2026, 9:06 a.m.