Custom Reports
In reproducibleRchunks: Automated Reproducibility Checks for R Markdown Documents

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)

library(reproducibleRchunks)

Here, I demonstrate how custom reports about the success of reproduction of a document can be generated within a markdown document. To illustrate, we first create three chunks using the code chunk type reproducibleR of which the first two reproduce just fine while not all variables of the third chunk reproduce. Finally, we generate custom reports.

Styling of chunk-level reports

The following code defines a new format for the chunk-level reports based on HTML/CSS. We draw a thin dashed line as border and fill the box with a light gray color. Last, we change the title.

options(reproducibleRchunks.templates = list(
html="<div style='border: 1px dashed gray;
padding: 15px 15px 5px 15px;
background-color: #F7F7F7;'>
<small><b>Reproducibility Checks</b><br>

${content}</small></div><br><br>"))

Next, we define some reproducibleR chunks of which some contain reproducible and some non-reproducible parts.

Cereals

In this first example we work with the UScereal data set from the MASS package. The chunk loads the data, runs an analysis of variance on calorie content by manufacturer and prints the summary of the model.

cereal_data <- MASS::UScereal
model <- anova(lm(calories~mfr, cereal_data))
summary(model)

Linear Model

The next chunk demonstrates a simple linear regression. We use the Animals data from MASS to predict brain weight from body weight and print the model summary.

animals <- MASS::Animals
model2 <- lm(brain~body, animals)
summary(model2)

Random Numbers

The following chunk contains some fixed numbers and numbers that are randomly drawn without setting a seed. Because the random numbers change from run to run, the chunk mixes reproducible and non-reproducible behaviour and serves as an example of how such situations are reported.

fixed_numbers <- 1:10
numbers <- rnorm(10, mean = 0, sd = 1)

Report

After the chunks above have been executed, reproducibleRchunks stores metadata about all created variables. The following code demonstrates how to extract and summarise this information. We start by counting the total number of variables that did not reproduce correctly:

num_errors <- reproducibleRchunks::get_num_reproducibility_errors()
 ifelse(num_errors > 0,"There were some results that did not reproduce!","The entire document reproduced succesfully")

The second function get_reproducibility_summary() generates a table that can either be used to inspect details about the reproduction status of each variable in each chunk, or to compute further summaries from. First, we obtain the table:

repro_table <- reproducibleRchunks::get_reproducibility_summary()
knitr::kable(repro_table)

Next, we summarise the table by chunk and create a bar plot that visualises the proportion of successful reproductions.

if (nrow(repro_table) > 0) {
  prop_success <- aggregate(Success ~ Chunk, data = repro_table, FUN = mean)

  barplot(prop_success$Success,
          names.arg = prop_success$Chunk,
          ylim = c(0, 1),
          ylab = "Relative Proportion",
          xlab = "Chunk",
          main = "Proportion of Successful Reproductions per Chunk")
} else {
  print("No reproducibility information available yet.")
}