expect_snapshot_file: Snapshot testing for whole files

In r-lib/testthat: Unit Testing for R

Snapshot testing for whole files

Description

Whole file snapshot testing is designed for testing objects that don't have a convenient textual representation, with initial support for images (.png, .jpg, .svg), data frames (.csv), and text files (.R, .txt, .json, ...).

The first time expect_snapshot_file() is run, it will create ⁠_snaps/{test}/{name}.{ext}⁠ containing reference output. Future runs will be compared to this reference: if different, the test will fail and the new results will be saved in ⁠_snaps/{test}/{name}.new.{ext}⁠. To review failures, call snapshot_review().

We generally expect this function to be used via a wrapper that takes care of ensuring that output is as reproducible as possible, e.g. automatically skipping tests where it's known that images can't be reproduced exactly.

Usage

expect_snapshot_file(
  path,
  name = basename(path),
  binary = lifecycle::deprecated(),
  cran = FALSE,
  compare = NULL,
  transform = NULL,
  variant = NULL
)

announce_snapshot_file(path, name = basename(path))

compare_file_binary(old, new)

compare_file_text(old, new)

Arguments

path	Path to file to snapshot. Optional for announce_snapshot_file() if name is supplied.
name	Snapshot name, taken from path by default.
binary	Please use the compare argument instead.
cran	Should these expectations be verified on CRAN? By default, they are not, because snapshot tests tend to be fragile because they often rely on minor details of dependencies.
compare	A function used to compare the snapshot files. It should take two inputs, the paths to the old and new snapshot, and return either TRUE or FALSE. This defaults to compare_file_text if name has extension .r, .R, .Rmd, .md, or .txt, and otherwise uses compare_file_binary. compare_file_binary() compares byte-by-byte and compare_file_text() compares lines-by-line, ignoring the difference between Windows and Mac/Linux line endings.
transform	Optionally, a function to scrub sensitive or stochastic text from the output. Should take a character vector of lines as input and return a modified character vector as output.
variant	If not-NULL, results will be saved in ⁠_snaps/{variant}/{test}/{name}.{ext}⁠. This allows you to create different snapshots for different scenarios, like different operating systems or different R versions.
old, new	Paths to old and new snapshot files.

Announcing snapshots

testthat automatically detects dangling snapshots that have been written to the ⁠_snaps⁠ directory but which no longer have corresponding R code to generate them. These dangling files are automatically deleted so they don't clutter the snapshot directory. However we want to preserve snapshot files when the R code wasn't executed because of an unexpected error or because of a skip(). Let testthat know about these files by calling announce_snapshot_file() before expect_snapshot_file().

Examples

# To use expect_snapshot_file() you'll typically need to start by writing
# a helper function that creates a file from your code, returning a path
save_png <- function(code, width = 400, height = 400) {
  path <- tempfile(fileext = ".png")
  png(path, width = width, height = height)
  on.exit(dev.off())
  code

  path
}
path <- save_png(plot(1:5))
path

## Not run: 
expect_snapshot_file(save_png(hist(mtcars$mpg)), "plot.png")

## End(Not run)

# You'd then also provide a helper that skips tests where you can't
# be sure of producing exactly the same output
expect_snapshot_plot <- function(name, code) {
  # Other packages might affect results
  skip_if_not_installed("ggplot2", "2.0.0")
  # Or maybe the output is different on some operation systems
  skip_on_os("windows")
  # You'll need to carefully think about and experiment with these skips

  name <- paste0(name, ".png")

  # Announce the file before touching `code`. This way, if `code`
  # unexpectedly fails or skips, testthat will not auto-delete the
  # corresponding snapshot file.
  announce_snapshot_file(name = name)

  path <- save_png(code)
  expect_snapshot_file(path, name)
}

r-lib/testthat documentation built on Feb. 18, 2024, 8:59 p.m.