covr.record_tests: Record Test Traces During Coverage Execution

In covr: Test Coverage for Packages

Record Test Traces During Coverage Execution

Description

By setting options(covr.record_tests = TRUE), the result of covr coverage collection functions will include additional data pertaining to the tests which are executed and an index of which tests, at what stack depth, trigger the execution of each trace.

Details

This functionality requires that the package code and tests are installed and sourced with the source. For more details, refer to R options, keep.source, keep.source.pkgs and keep.parse.data.pkgs.

Additional fields

Within the covr result, you can explore this information in two places:

• attr(,"tests"): A list of call stacks, which results in target code execution.

• ⁠$<srcref>$tests⁠: For each srcref count in the coverage object, a ⁠$tests⁠ field is now included which contains a matrix with three columns, "test", "depth" and "i" which specify the test number (corresponding to the index of the test in attr(,"tests"), the stack depth into the target code where the trace was executed, and the order of execution for each test.

Test traces

The content of test traces are dependent on the unit testing framework that is used by the target package. The behavior is contingent on the available information in the sources kept for the testing files.

Test traces are extracted by the following criteria:

1. If any srcref files are are provided by a file within covr's temporary library, all calls from those files are kept as a test trace. This will collect traces from tests run with common testing frameworks such as testthat and RUnit.

2. Otherwise, as a conservative fallback in situations where no source references are found, or when none are from within the temporary directory, the entire call stack is collected.

These calls are subsequently subset for only those up until the call to covr's internal count function, and will always include the last call in the call stack prior to a call to count.

Examples

fcode <- '
f <- function(x) {
  if (x)
    f(!x)
  else
    FALSE
}'

options(covr.record_tests = TRUE)
cov <- code_coverage(fcode, "f(TRUE)")

# extract executed test code for the first test
tail(attr(cov, "tests")[[1L]], 1L)
# [[1]]
# f(TRUE)

# extract test itemization per trace
cov[[3]][c("srcref", "tests")]
# $srcref
# f(!x)
#
# $tests
#      test depth i
# [1,]    1     2 4

# reconstruct the code path of a test by ordering test traces by [,"i"]
lapply(cov, `[[`, "tests")
# $`source.Ref2326138c55:4:6:4:10:6:10:4:4`
#      test depth i
# [1,]    1     1 2
#
# $`source.Ref2326138c55:3:8:3:8:8:8:3:3`
#      test depth i
# [1,]    1     1 1
# [2,]    1     2 3
#
# $`source.Ref2326138c55:6:6:6:10:6:10:6:6`
#      test depth i
# [1,]    1     2 4

covr documentation built on Nov. 9, 2023, 9:07 a.m.