NEWS.md

testthat 3.1.2

testthat 3.1.1

Once activated, snapshots will now use rlang to print error and warning messages, including the Error: and Warning: prefixes. This means the call field of conditions is now displayed in snapshots if present. Parent error messages are also displayed. Following this change, all snapshots including error and warning messages need to be revalidated.

We will enable the new rlang 1.0 output unconditionally in a future release.

The default is currently unchanged so that condition classes keep being included in snapshots. However, we plan to change the default to FALSE in an upcoming release to prevent distracting snapshot diffing as upstream packages add error classes. For instance, the development version of R is currently adding classes to basic errors, which causes spurious snapshot changes when testing against R-devel on CI.

If you depend on rlang 1.0 (see above), the default is already set to FALSE.

testthat 3.1.0

Snapshot tests

Breaking changes

This is a breaking change to the 3rd edition. Where you could previously do:

expect_equal(expect_warning(f(), "warning"), "value")

You must now use condition expectations on the outside:

``` expect_warning(expect_equal(f(), "value"), "warning")

# Equivalently, save the value before inspection expect_warning(value <- f(), "warning") expect_equal(value, "value") ```

This breaking change makes testthat more consistent. It also makes it possible to inspect both the value and the warning, which would otherwise require additional tools.

Minor improvements and bug fixes

testthat 3.0.4

testthat 3.0.3

The functions compare_file_binary() and compare_file_text() are now exported from testthat to be supplied as compare argument. These implement the same behaviour as the old binary argument which is now deprecated.

testthat 3.0.2

testthat 3.0.1

testthat 3.0.0

3rd edition

testhat 3.0.0 brings with it a 3rd edition that makes a number of breaking changes in order to clean up the interface and help you use our latest recommendations. To opt-in to the 3rd edition for your package, set Config/testthat/edition: 3 in your DESCRIPTION or use local_edition(3) in individual tests. You can retrieve the active edition with edition_get(). Learn more in vignette("third-edition").

Snapshot testing

New family of snapshot expectations (expect_snapshot(), expect_snapshot_output(), expect_snapshot_error(), and expect_snapshot_value()) provide "snapshot" tests, where the expected results are stored in separate files in test/testthat/_snaps. They're useful whenever it's painful to store expected results directly in the test files.

expect_snapshot_file() along with snapshot_review() help snapshot more complex data, with initial support for text files, images, and data frames (#1050).

See vignette("snapshotting") for more details.

Reporters

Fixures

Skips

Test running

Minor improvements and bug fixes

testthat 2.3.2

testthat 2.3.1

testthat 2.3.0

Conditions

This release mostly focusses on an overhaul of how testthat works with conditions (i.e. errors, warnings and messages). There are relatively few user-facing changes, although you should now see more informative backtraces from errors and failures.

Expectations

Other minor improvements and bug fixes

testthat 2.2.1

testthat 2.2.0

New features

Minor improvements and bug fixes

testthat 2.1.1

testthat 2.1.0

New expectations

Improvements to existing expectations

Reporters

Skips

Other new features

Other minor improvements and bug fixes

testthat 2.0.1

testthat 2.0.0

Breaking API changes

Expectations

New and improved expectations

New and improved skips

Known good values

We have identified a useful family of expectations that compares the results of an expression to a known good value stored in a file. They are designed to be use in conjunction with git so that you can see what precisely has changed, and revert it if needed.

Quasiquotation support

All expectations can now use unquoting (#626). This makes it much easier to generate informative failure messages when running tests in a for loop.

For example take this test:

f <- function(i) if (i > 3) i * 9 else i * 10

for (i in 1:5) {
  expect_equal(f(i), i * 10)
}

When it fails, you'll see the message Error: `f(i)` not equal to `i * 10`. That's hard to diagnose because you don't know which iteration caused the problem!

for (i in 1:5) {
  expect_equal(f(!!i), !!(i * 10))
}

If you unquote the values using !!, you get the failure message `f(4L)` not equal to 40.. This is much easier to diagnose! See ?quasi_label() for more details.

(Note that this is not tidy evaluation per se, but is closely related. At this time you can not unquote quosures.)

New features

Setup and teardown

Other new features

New default reporter

A new default reporter, ReporterProgress, produces more aesthetically pleasing output and makes the most important information available upfront (#529). You can return to the previous default by setting options(testthat.default_reporter = "summary").

Reporters

Deprecated functions

Minor improvements and bug fixes

testthat 1.0.2

testthat 1.0.1

testthat 1.0.0

Breaking changes

The expectation() function now expects an expectation type (one of "success", "failure", "error", "skip", "warning") as first argument. If you're creating your own expectations, you'll need to use expect() instead (#437).

New expectations

The expectation system got a thorough overhaul (#217). This primarily makes it easier to add new expectations in the future, but also included a thorough review of the documentation, ensuring that related expectations are documented together, and have evocative names.

One useful change is that most expectations invisibly return the input object. This makes it possible to chain together expectations with magrittr:

factor("a") %>% 
  expect_type("integer") %>% 
  expect_s3_class("factor") %>% 
  expect_length(1)

(And to make this style even easier, testthat now re-exports the pipe, #412).

The exception to this rule are the expectations that evaluate (i.e. for messages, warnings, errors, output etc), which invisibly return NULL. These functions are now more consistent: using NA will cause a failure if there is a errors/warnings/mesages/output (i.e. they're not missing), and will NULL fail if there aren't any errors/warnings/mesages/output. This previously didn't work for expect_output() (#323), and the error messages were confusing with expect_error(..., NA) (#342, @nealrichardson + @krlmlr, #317).

Another change is that expect_output() now requires you to explicitly print the output if you want to test a print method: expect_output("a", "a") will fail, expect_output(print("a"), "a") will succeed.

There are six new expectations:

A number of older features have been deprecated:

Expectations are conditions

Now all expectations are also conditions, and R's condition system is used to signal failures and successes (#360, @krlmlr). All known conditions (currently, "error", "warning", "message", "failure", and "success") are converted to expectations using the new as.expectation(). This allows third-party test packages (such as assertthat, testit, ensurer, checkmate, assertive) to seamlessly establish testthat compatibility by issuing custom error conditions (e.g., structure(list(message = "Error message"), class = c("customError", "error", "condition"))) and then implementing as.expectation.customError(). The assertthat package contains an example.

Reporters

The reporters system class has been considerably refactored to make existing reporters simpler and to make it easier to write new reporters. There are two main changes:

Other

testthat 0.11.0

testthat 0.10.0

testthat 0.9.1

testthat 0.9

New features

Minor improvements and bug fixes

Deprecated functions

testthat 0.8.1

testthat 0.8

testthat 0.8 comes with a new recommended structure for storing your tests. To better meet CRAN recommended practices, testthat now recommend that you to put your tests in tests/testthat, instead of inst/tests (this makes it possible for users to choose whether or not to install tests). With this new structure, you'll need to use test_check() instead of test_packages() in the test file (usually tests/testthat.R) that runs all testthat unit tests.

The other big improvement to usability comes from @kforner, who contributed code to allow the default results (i.e. those produced by SummaryReporter) to include source references so you can see exactly where failures occured.

New reporters

New expectations

Minor improvements and bug fixes

testthat 0.7.1

testthat 0.7

testthat 0.6

testthat 0.5

testthat 0.4

testthat 0.3

testthat 0.2



Try the testthat package in your browser

Any scripts or data that you put into this service are public.

testthat documentation built on Jan. 21, 2022, 1:10 a.m.