Experiment with changes in an R project
Inspired by the Ruby gem scientist - but instead of targeted at web apps, this project targets researchers/etc. that want to compare changes in their code in a rigorous fashion.
You have some code. You want to make a change to the code, and you have a few different ideas about what you'd like to do. For example, you want to pre-allocate the size of the data.frame to see if that saves time.
scientist
can help you sort out changes by comparing how long
each version takes, and visually diff results.
Using scientist
you can compare these two functions like:
a <- Experiment$new(name = "compare_code")
a$control(v1 = {
out <- data.frame(letter = NA_character_, LETTER = NA_character_,
both = NA_character_, stringsAsFactors = FALSE)
for (i in 1:26)
out[i,] <- c(letters[i], LETTERS[i], paste0(letters[i], LETTERS[i]))
out
})
a$candidate(v2 = {
out <- data.frame(letter = rep(NA_character_, times = 26),
LETTER = rep(NA_character_, times = 26),
both = rep(NA_character_, times = 26),
stringsAsFactors = FALSE)
for (i in 1:26)
out[i,] <- c(letters[i], LETTERS[i], paste0(sample(letters, 1), LETTERS[i]))
out
})
a
Then we can run the "experiment"
a$run()
Then compare results
a$diff()
Scripts
You have an R script, let's call it code.R
. Just as above with the code
example, you want to make a change to the script. Instead of using
code blocks as input as above, you can use file names. (NOTE: file names
not supported yet, see issue #7)
Using scientist
you can compare these two scripts with:
b <- Experiment$new(name = "compare_scripts")
b$control(file = "code.R")
b$candidate(file = "code_new.R")
Packagesnote: above code doesn't work yet
You have a package, let's call it foobar
. You want to change a function
in foobar
called stuff()
. You make a new version of that function
called stuff_new()
. (NOTE: functions not supported yet per se, see
issue #8; although
you can call functions just like code blocks)
Using scientist
you can compare these two functions with:
res <- Experiment$new(name = "compare_stuff")
res$control(foobar::stuff(x = 5))
res$candidate(foobar::stuff_new(x = 5))
note: above is pseudocode, as foobar is not a real package; though you can try functions from a real package
remotes::install_github("sckott/scientist")
library(scientist)
Initialize an experiment
res <- Experiment$new(name = "jane")
Set your control code block
res$control({
x = 5
x^2
})
Set your candidate code block. You can have 1 or more candidates, which are compared against the control.
res$candidate({
y = 5
y^3
})
Now you can see some control and candidate details
res
#> <Experiment> jane
#> error on mismatch?: FALSE
#> waiting?: TRUE
#> progress?: FALSE
#> control: <unnamed>
#> candidate: <unnamed>
Run the experiment
res$run()
Get the results
res$control_result
#> [[1]]
#> [1] 25
res$candidate_results
#> [[1]]
#> [1] 125
Get all results plus timing data
res$result()
#> $name
#> [1] "jane"
#>
#> $control
#> $control$result
#> $control$result[[1]]
#> [1] 25
#>
#>
#> $control$time
#> $control$time$start
#> [1] "2020-09-18 01:05:12 GMT"
#>
#> $control$time$end
#> [1] "2020-09-18 01:05:13 GMT"
#>
#> $control$time$duration
#> [1] 0.680455
#>
#>
#>
#> $candidates
#> $candidates[[1]]
#> $candidates[[1]]$result
#> [1] 125
#>
#> $candidates[[1]]$time
#> $candidates[[1]]$time$start
#> [1] "2020-09-18 01:05:12 GMT"
#>
#> $candidates[[1]]$time$end
#> [1] "2020-09-18 01:05:13 GMT"
#>
#> $candidates[[1]]$time$duration
#> [1] 0.7593031
#>
#>
#> $candidates[[1]]$name
#> [1] NA
#>
#>
#>
#> $comparison
#> [1] FALSE
Publish results - opens a page in your default browser
res$publish()
scientist
in R doing citation(package = 'scientist')
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.