knitr::opts_chunk$set ( collapse = TRUE, warning = TRUE, message = TRUE, width = 120, comment = "#>", fig.retina = 2, fig.path = "README-" )
Automatic mutation testing of R packages. Mutation in the sense of mutating
inputs (parameters) to function calls. autotest
primarily works by scraping
documented examples for all functions, and mutating the parameters input to
those functions.
The easiest way to install this package is via the associated
r-universe
. As
shown there, simply enable the universe with
options (repos = c ( ropenscireviewtools = "https://ropensci-review-tools.r-universe.dev", CRAN = "https://cloud.r-project.org" ))
And then install the usual way with,
install.packages ("autotest")
Alternatively, the package can be installed by running one of the following lines:
# install.packages("remotes") remotes::install_git ("https://git.sr.ht/~mpadge/autotest") remotes::install_bitbucket ("mpadge/autotest") remotes::install_gitlab ("mpadge/autotest") remotes::install_github ("ropensci-review-tools/autotest")
The package can then be loaded the usual way:
library (autotest)
devtools::load_all (".", export_all = FALSE)
The simply way to use the package is
x <- autotest_package ("<package>")
The main argument to the autotest_package()
function
can either be the name of an installed package, or a path to a local directory
containing the source for a package. The result is a data.frame
of errors,
warnings, and other diagnostic messages issued during package auotest
-ing.
The function has an additional parameter, functions
, to restrict tests to
specified functions only.
By default,
autotest_package()
returns a list of all tests applied to a package without actually running them.
To implement those tests, set the parameter test
to TRUE
. Results are only returned for tests
in which functions do not behave as expected, whether through triggering
errors, warnings, or other behaviour as described below. The ideal behaviour of
autotest_package()
is to return nothing (or strictly, NULL
), indicating
that all tests passed successfully. See the main package
vignette for an
introductory tour of the package.
The package includes a function which lists all tests currently implemented.
autotest_types ()
That functions returns a tibble
describing
r nrow(autotest_types())
unique tests. The default behaviour of
autotest_package()
with test = FALSE
uses these test types to identify which tests will be
applied to each parameter and function. The table returned from
autotest_types()
can be used to selectively switch tests off by setting values in the test
column to FALSE
, as demonstrated below.
The package works by scraping documented examples from all .Rd
help files,
and using those to identify the types of all parameters to all functions. Usage
therefore first requires that the usage of all parameters be demonstrated in
example code.
As described above, tests can also be selectively applied to particular
functions through the parameters functions
, used to nominate functions to
include in tests, or exclude
, used to nominate functions to exclude from
tests. The following code illustrates.
x <- autotest_package (package = "stats", functions = "var", test = FALSE) print (x)
Testing the var
function also tests cor
and cov
, because these are all
documented within a single .Rd
help file. Typing ?var
shows that the help
topic is cor
, and that the examples include the three functions, var
,
cor
, and cov
. That result details the r nrow (x)
tests which would be
applied to the var
function from the stats
package. These r nrow (x)
tests yield the following results when actually applied:
y <- autotest_package (package = "stats", functions = "var", test = TRUE) print (y)
And only r nrow (y)
of the original r nrow (x)
tests produced unexpected
behaviour. There were in fact only r length (unique (y$operation))
kinds of
tests which produced these r nrow (y)
results:
unique (y$operation)
One of these involves conversion of a vector to a list-column representation
(via I(as.list(<vec>))
). Relatively few packages accept this kind of input,
even though doing so is relatively straightforward. The following lines
demonstrate how these tests can be switched off when autotest
-ing a package.
The autotest_types()
function, used above to extract information on all types
of tests, also accepts a single argument listing the test_name
entries of any
tests which are to be switched off.
types <- autotest_types (notest = "vector_to_list_col") y <- autotest_package ( package = "stats", functions = "var", test = TRUE, test_data = types ) print (y)
Those tests are still returned from autotest_package()
, but with test =
FALSE
to indicate they were not run, and a type
of "no_test" rather than the
previous "diagnostic".
autotest
automatically create tests in my tests
directory?Not yet, but that should be possible soon. In the meantime, there are
testthat
expectations, listed in the main
package
functions,
which enable autotest
to be used in a package's test suite.
great-expectations
framework for python, described in this medium
article.QuickCheck
for Haskellmutate
for rubymutant
for mutation of R code itselfPlease note that this package is released with a Contributor Code of Conduct. By contributing to this project, you agree to abide by its terms.
All contributions to this project are gratefully acknowledged using the allcontributors
package following the all-contributors specification. Contributions of any kind are welcome!
mpadge |
helske |
maelle |
simpar1471 |
noamross |
njtierney |
JeffreyRStevens |
bbolker |
mattfidler |
kieranjmartin |
statnmap |
vgherard |
christophsax |
joelnitta |
santikka |
gilbertocamara |
schneiderpy |
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.