NSTF Testing Frameworks

In test.assessr: Assessing Package Test Reliability and Quality

knitr::opts_chunk$set(echo = TRUE)

library(test.assessr)

The test.assessr package checks package structure and test drivers to detect and classify testing configurations.

If it detects a non-standard testing framework (NSTF) (i.e. RUnit‑based, BiocGenerics, data.table, tinytest, and custom test layouts), it runs non-standard testing workflows depending on the framework type:

The following example is the workflow for BiocGenerics which involves Bioconductor RUnit framework

BiocGeneric workflow

• Builds a source-to-test mapping.
• Removes any run_unitTests.R wrapper from the mapping.
• Prepares a test environment before running RUnit tests:
• Detaches commonly attached BioC infrastructure packages (IRanges, S4Vectors, BiocGenerics) to avoid namespace/shim conflicts.
• Calls pkgload::load_all() to load the package under test.
• Clears R_TESTS to avoid R CMD check harness interference when running RUnit programmatically.
• Identifies problem/skip-able tests (which should use RUnit::runTestFile() + RUnit::getErrors() per file).
• Builds a list of test files under inst/unitTests excluding any that should be skipped.

• Exposes namespace objects + datasets to a dedicated environment and runs coverage with covr::environment_coverage().

• If BiocGeneric workflow is successful, then a unit test data coverage object like this is produced:

BiocGeneric workflow unit test data coverage object

| R object | field | element 1 | element 2 | value | | -------- | ----- | --------- | --------- | ------| | test_package |pkg_name | | | "BiocGenerics" | | | pkg_ver | | | "0.56.0" | | | date_time | | | "2026-03-09 14:27:36.673915" | | | executor | | | "xxxxxxx" | | | sysname | | | "Linux" | | | version | | | "#1 SMP PREEMPT_DYNAMIC Mon Dec 1 05:36:50 UTC 2025" | | | release | | | "6.1.158-180.294.amzn2023.x86_64" | | | machine | | | "x86_64" | | | r_version | | |"4.4.1" | | | test_framework_type | | | non-standard testing framework - Bioconductor | | | total_cov | | | 15.5 | | | res_cov | name| |"BiocGenerics" | | | | coverage | filecoverage | num [1:57(1d)] 0 0 0 0 0 0 0 0 0 0 ... | | | | | attr(*, "dimnames")=List of 1 [1:57] "/tmp/RtmpB1gcqI/R.INSTALL8ff8254343d/S4Vectors/R/DataFrame-class.R" "/tmp/RtmpB1gcqI/R.INSTALL8ff8254343d/S4Vectors/R/DataFrame-combine.R" "/tmp/RtmpB1gcqI/R.INSTALL8ff8254343d/S4Vectors/R/DataFrame-comparison.R" "/tmp/RtmpB1gcqI/R.INSTALL8ff8254343d/S4Vectors/R/DataFrame-utils.R" ... | | | | | totalcoverage | 15.5 | | | errors | | |NA | | | notes | | |NA | | functions_no_tests | source_file | | | [1:67] "annotation.R" "aperm.R" "append.R" "as.data.frame.R" ... | | | test_file | | | [1:67] NA NA NA NA ... | | | evidence | | | [1:67] NA NA NA NA ... | | | has_tests | | | [1:67] FALSE FALSE FALSE FALSE FALSE FALSE ... | | | comment | | | [1:67] "No tests found" "No tests found" "No tests found" "No tests found" ... | | tests_skipped | | | | chr(0) | | tests_passing | | | | chr [1:7] "/tmp/RtmpneA8hi/temp_file_53d27b76020b/BiocGenerics/inst/unitTests/test_combine.R" "/tmp/RtmpneA8hi/temp_file_53d27b76020b/BiocGenerics/inst/unitTests/test_Extremes.R" "/tmp/RtmpneA8hi/temp_file_53d27b76020b/BiocGenerics/inst/unitTests/test_format.R" "/tmp/RtmpneA8hi/temp_file_53d27b76020b/BiocGenerics/inst/unitTests/test_mapply.R" ... | | n_functions_tests_passing | | | | int 7 | | n_functions_no_tests| | | | int 67 | | n_functions_tests_skipped | | | | int 0 |

The NSTF unit test data coverage objects are not able to give details about test block start and end lines and the expectation types.

test.assessr documentation built on April 30, 2026, 5:06 p.m.