In rich-iannone/pointblank: Data Validation and Organization of Metadata for Local and Remote Tables

knitr::opts_chunk$set(echo = TRUE)

library(pointblank)

The above message indicates a dire reality: validation checks did not pass (4 failing validations!). Turns out we can validate our data right inside of an R Markdown document. The pointblank package makes that possible by providing the validate_rmd() function (put it inside the setup code chunk). The pointblank package ships with a small dataset called small_table. It really is small, only a few rows:

small_table

We can perform validation checks on that table with pointblank step functions. Be sure to use validate = TRUE as a chunk option. The results will be initially hidden in the rendered HTML document but can be revealed by pressing the status button. Testing like this would normally stop the rendering of the R Markdown document but here any errors captured and reported in the rendered doc.

col_exists(small_table, columns = vars(a, b, c, d, e, f))
rows_distinct(small_table, vars(d, e))
col_vals_gt(small_table, vars(d), 1000)

We could also use pointblank's stop_if_not() function to generate some predicate-based validation statements. This works really well on all sorts of data tables and these predicates are really easy to whip up.

stop_if_not(nrow(small_table) > 10)
stop_if_not("time" %in% colnames(small_table))

Note that with multiple pointblank step functions chained together, only the first error encountered will be reported. So, use multiple statements if necessary to check which may result in failed validations.

small_table %>% 
  col_exists(columns = vars(a, b, c, d, e, f)) %>% # this passes validation
  rows_distinct() %>%                              # this step fails (showing us the error message)
  col_vals_gt(vars(d), 5000)                       # this also fails (we don't see its message)

When validation doesn't fail us, we can still inspect the validation code and the results.

small_table %>% 
  col_is_date("date") %>%
  col_vals_gt(vars(d), vars(c), na_pass = TRUE)

That's about it for this example .Rmd. Always be sure to test that there data!

rich-iannone/pointblank documentation built on May 10, 2024, 12:46 p.m.