README.md
In poissonconsulting/deckr: Data Checking
datacheckr
Introduction
datacheckr is an R package to check the classes and values of scalars and vectors and the column names, classes, values, keys and joins in data frames. It provides an informative error message if a user-defined condition fails to be met otherwise it returns the object (so it can be used in pipes).
Demonstration
Consider the data frame my_data
library(tibble)
my_data <- data_frame(
Count = c(0L, 3L, 3L, 0L, NA),
Longitude = c(0, 0, 90, 90, 180),
Latitude = c(0, 90, 90.2, 100, -180),
Type = factor(c("Good", "Bad", "Bad", "Bad", "Bad"), levels = c("Good", "Bad")),
Extra = TRUE,
Comments = c("In Greenwich", "Somewhere else", "I'm lost", "I didn't see any", "Help"))
my_data
#> # A tibble: 5 x 6
#> Count Longitude Latitude Type Extra Comments
#> <int> <dbl> <dbl> <fctr> <lgl> <chr>
#> 1 0 0 0.0 Good TRUE In Greenwich
#> 2 3 0 90.0 Bad TRUE Somewhere else
#> 3 3 90 90.2 Bad TRUE I'm lost
#> 4 0 90 100.0 Bad TRUE I didn't see any
#> 5 NA 180 -180.0 Bad TRUE Help
Integers
To specify that my_data must contain a column called col1 of class integer use the check_data2 function
library(datacheckr)
check_data2(my_data, values = list(col1 = integer()))
#> Error: my_data must have column 'col1'
Missing Values
To specify that a column cannot include missing values pass a single non-missing value (of the correct class)
check_data2(my_data, list(Count = 1L))
#> Error: column Count in my_data cannot include missing values
To specify that it can include missing values include an NA in the vector
check_data2(my_data, list(Count = c(1L, NA)))
and to specify that it can only include missing values pass an NA (of the correct class)
check_data2(my_data, list(Count = NA_integer_))
#> Error: column Count in my_data can only include missing values
Value Ranges
To indicate that the values must fall within a range use two non-missing values
check_data2(my_data, list(Count = c(0L, 2L)))
#> Error: column Count in my_data cannot include missing values
Specific Values
If particular values are required then specify them as a vector of three or more non-missing values
check_data2(my_data, list(Count = c(1L, 2L, 2L)))
#> Error: column Count in my_data cannot include missing values
The order of the values in an element is unimportant.
Numeric, Date and POSIXct
Numeric, Date and POSIXct columns have exactly the same behaviour regarding ranges and specific values as illustrated above using integers.
Logical
With logical values two non-missing values produce the same behaviour as three or more non-missing values.
For example to test for only FALSE values use
check_data2(my_data, list(Extra = c(FALSE, FALSE)))
#> Error: column Extra in my_data can only include FALSE values
Characters
The following requires that the values of Comments match both character elements which are treated as regular expressions
check_data2(my_data, list(Comments = c("e", "o")))
#> Error: column Comments in my_data contains strings that do not match both regular expressions 'e' and 'o'
with three or more non-missing character elements each value must match at least one of the elements which are treated as regular expressions.
check_data2(my_data, list(Comments = c("e", "o", "o")))
Regular expressions are matched using grepl with perl=TRUE.
Factors
To specify that Type should be a factor that includes "Bad1" and "Good" among its levels
check_data2(my_data, list(Type = factor(c("Bad1", "Good"))))
#> Error: column Type in my_data lacks factor levels 'Bad1' and 'Good'
And to specify the actual factor levels, pass three or more non-missing values
check_data2(my_data, list(Type = factor(c("Bad", "Good", "Good"))))
#> Error: column Type in my_data must be a factor with the levels 'Bad' and 'Good'
Column Names
Whereas check_data2() ignores unnamed columns and doesn't care about the order, check_data3() requires that column names match the names in values.
check_data3(my_data, list(Comments = character()))
#> Error: column names in my_data must be identical to 'Comments'
Missing Columns
In contrast, check_data1() can be used to test that specific columns are missing or that a column satisfies one of multiple conditions.
check_data1(my_data, list(Comments = NULL))
#> Error: my_data must not include column Comments
check_data1(my_data, list(Comments = integer(),
Comments = numeric()))
#> Error: column Comments in my_data must be of class 'integer' or 'numeric'
To specify that my_data can contain a column col1 that can be integer or numeric values the call would be
check_data1(my_data, list(
col1 = integer(),
col1 = NULL,
col1 = numeric()))
Naming Objects
By default, datacheckr determines the name of an object based on the call. This results in uninformative error messages when used in a pipe
library(magrittr)
my_data %<>% check_data2(values = list(col1 = integer()))
#> Error: . must have column 'col1'
The argument data_name can be used to define the name
library(magrittr)
my_data %<>% check_data2(values = list(col1 = integer()), data_name = "d8r")
#> Error: d8r must have column 'col1'
Relational Data
Consider the relational data in the nycflights13 package.
Keys
The following code uses the check_data3 function to confirm that airlines has just two columns carrier and name, in that order, which are both character vectors and that carrier is unique (a key).
library(nycflights13)
check_data3(airlines, list(carrier = "",
name = ""),
key = "carrier")
The next code checks that airports has the listed columns in that order and that faa is a unique character vector of three 'word characters', lat is a number between 0 and 90, alt is an integer between -100 and 10,000, and dst is a character vector with the possible values A, N or U.
check_data3(airports, list(faa = rep("^\\w{3,3}$",2),
name = "",
lat = c(0, 90),
lon = c(-180, 180),
alt = as.integer(c(-100, 10^5L)),
tz = c(-11, 11),
dst = rep("A|N|U", 2),
tzone = ""),
key = "faa")
This checks that planes includes tailnum, engines and year (as using less strict check_data2) and that engines is 1, 2, 3 or 4, that year is an integer between 1956 and 2013 that can include missing values and tailnum (which consists of strings of 5 to 6 letter 'word characters') is the unique key.
check_data2(planes, list(tailnum = rep("^\\w{5,6}$",2),
engines = 1:4,
year = c(1956L, 2013L, NA)),
key = "tailnum")
Selecting Columns
Weather has lots of columns. by setting select = TRUE in check_data3 we drop non-named columns and order to match values. The checks indicate that year is only 2013, and like month is a number but day and hour are integers (as expected)
weather %<>% check_data3(list(year = c(2013,2013),
month = c(1, 12),
day = c(1L, 31L),
hour = c(0L, 23L),
origin = rep("^\\w{3,3}$",2)),
select = TRUE)
weather
#> # A tibble: 26,130 x 5
#> year month day hour origin
#> <dbl> <dbl> <int> <int> <chr>
#> 1 2013 1 1 0 EWR
#> 2 2013 1 1 1 EWR
#> 3 2013 1 1 2 EWR
#> 4 2013 1 1 3 EWR
#> 5 2013 1 1 4 EWR
#> 6 2013 1 1 6 EWR
#> 7 2013 1 1 7 EWR
#> 8 2013 1 1 8 EWR
#> 9 2013 1 1 9 EWR
#> 10 2013 1 1 10 EWR
#> # ... with 26,120 more rows
Joins
Checking the referential integrity of the (many-to-one) join between flights and airlines is easy.
check_join(flights, airlines, join = "carrier")
In addition to tailnum, flights and planes have additional column with the same name.
check_join(flights, planes, join = "tailnum")
#> Error: flights and planes must not have additional matching columns
We can deal with this by setting extra = TRUE but the data fail referential integrity because we have planes without flights.
check_join(flights, planes, extra = TRUE, join = "tailnum")
#> Error: many-to-one join between flights and planes violates referential integrity
Installation
To install the most recent release from CRAN
install.packages("datacheckr")
To install the development version from GitHub
# install.packages("devtools")
devtools::install_github("datacheckr")
Contribution
Please report any issues.
Pull requests are always welcome.
Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.
poissonconsulting/deckr documentation built on March 18, 2021, 10:37 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
datacheckr
Introduction
datacheckr is an R package to check the classes and values of scalars and vectors and the column names, classes, values, keys and joins in data frames. It provides an informative error message if a user-defined condition fails to be met otherwise it returns the object (so it can be used in pipes).
Demonstration
Consider the data frame my_data
library(tibble)
my_data <- data_frame(
Count = c(0L, 3L, 3L, 0L, NA),
Longitude = c(0, 0, 90, 90, 180),
Latitude = c(0, 90, 90.2, 100, -180),
Type = factor(c("Good", "Bad", "Bad", "Bad", "Bad"), levels = c("Good", "Bad")),
Extra = TRUE,
Comments = c("In Greenwich", "Somewhere else", "I'm lost", "I didn't see any", "Help"))
my_data
#> # A tibble: 5 x 6
#> Count Longitude Latitude Type Extra Comments
#> <int> <dbl> <dbl> <fctr> <lgl> <chr>
#> 1 0 0 0.0 Good TRUE In Greenwich
#> 2 3 0 90.0 Bad TRUE Somewhere else
#> 3 3 90 90.2 Bad TRUE I'm lost
#> 4 0 90 100.0 Bad TRUE I didn't see any
#> 5 NA 180 -180.0 Bad TRUE Help
Integers
To specify that my_data must contain a column called col1 of class integer use the check_data2 function
library(datacheckr)
check_data2(my_data, values = list(col1 = integer()))
#> Error: my_data must have column 'col1'
Missing Values
To specify that a column cannot include missing values pass a single non-missing value (of the correct class)
check_data2(my_data, list(Count = 1L))
#> Error: column Count in my_data cannot include missing values
To specify that it can include missing values include an NA in the vector
check_data2(my_data, list(Count = c(1L, NA)))
and to specify that it can only include missing values pass an NA (of the correct class)
check_data2(my_data, list(Count = NA_integer_))
#> Error: column Count in my_data can only include missing values
Value Ranges
To indicate that the values must fall within a range use two non-missing values
check_data2(my_data, list(Count = c(0L, 2L)))
#> Error: column Count in my_data cannot include missing values
Specific Values
If particular values are required then specify them as a vector of three or more non-missing values
check_data2(my_data, list(Count = c(1L, 2L, 2L)))
#> Error: column Count in my_data cannot include missing values
The order of the values in an element is unimportant.
Numeric, Date and POSIXct
Numeric, Date and POSIXct columns have exactly the same behaviour regarding ranges and specific values as illustrated above using integers.
Logical
With logical values two non-missing values produce the same behaviour as three or more non-missing values.
For example to test for only FALSE values use
check_data2(my_data, list(Extra = c(FALSE, FALSE)))
#> Error: column Extra in my_data can only include FALSE values
Characters
The following requires that the values of Comments match both character elements which are treated as regular expressions
check_data2(my_data, list(Comments = c("e", "o")))
#> Error: column Comments in my_data contains strings that do not match both regular expressions 'e' and 'o'
with three or more non-missing character elements each value must match at least one of the elements which are treated as regular expressions.
check_data2(my_data, list(Comments = c("e", "o", "o")))
Regular expressions are matched using grepl with perl=TRUE.
Factors
To specify that Type should be a factor that includes "Bad1" and "Good" among its levels
check_data2(my_data, list(Type = factor(c("Bad1", "Good"))))
#> Error: column Type in my_data lacks factor levels 'Bad1' and 'Good'
And to specify the actual factor levels, pass three or more non-missing values
check_data2(my_data, list(Type = factor(c("Bad", "Good", "Good"))))
#> Error: column Type in my_data must be a factor with the levels 'Bad' and 'Good'
Column Names
Whereas check_data2() ignores unnamed columns and doesn't care about the order, check_data3() requires that column names match the names in values.
check_data3(my_data, list(Comments = character()))
#> Error: column names in my_data must be identical to 'Comments'
Missing Columns
In contrast, check_data1() can be used to test that specific columns are missing or that a column satisfies one of multiple conditions.
check_data1(my_data, list(Comments = NULL))
#> Error: my_data must not include column Comments
check_data1(my_data, list(Comments = integer(),
Comments = numeric()))
#> Error: column Comments in my_data must be of class 'integer' or 'numeric'
To specify that my_data can contain a column col1 that can be integer or numeric values the call would be
check_data1(my_data, list(
col1 = integer(),
col1 = NULL,
col1 = numeric()))
Naming Objects
By default, datacheckr determines the name of an object based on the call. This results in uninformative error messages when used in a pipe
library(magrittr)
my_data %<>% check_data2(values = list(col1 = integer()))
#> Error: . must have column 'col1'
The argument data_name can be used to define the name
library(magrittr)
my_data %<>% check_data2(values = list(col1 = integer()), data_name = "d8r")
#> Error: d8r must have column 'col1'
Relational Data
Consider the relational data in the nycflights13 package.
Keys
The following code uses the check_data3 function to confirm that airlines has just two columns carrier and name, in that order, which are both character vectors and that carrier is unique (a key).
library(nycflights13)
check_data3(airlines, list(carrier = "",
name = ""),
key = "carrier")
The next code checks that airports has the listed columns in that order and that faa is a unique character vector of three 'word characters', lat is a number between 0 and 90, alt is an integer between -100 and 10,000, and dst is a character vector with the possible values A, N or U.
check_data3(airports, list(faa = rep("^\\w{3,3}$",2),
name = "",
lat = c(0, 90),
lon = c(-180, 180),
alt = as.integer(c(-100, 10^5L)),
tz = c(-11, 11),
dst = rep("A|N|U", 2),
tzone = ""),
key = "faa")
This checks that planes includes tailnum, engines and year (as using less strict check_data2) and that engines is 1, 2, 3 or 4, that year is an integer between 1956 and 2013 that can include missing values and tailnum (which consists of strings of 5 to 6 letter 'word characters') is the unique key.
check_data2(planes, list(tailnum = rep("^\\w{5,6}$",2),
engines = 1:4,
year = c(1956L, 2013L, NA)),
key = "tailnum")
Selecting Columns
Weather has lots of columns. by setting select = TRUE in check_data3 we drop non-named columns and order to match values. The checks indicate that year is only 2013, and like month is a number but day and hour are integers (as expected)
weather %<>% check_data3(list(year = c(2013,2013),
month = c(1, 12),
day = c(1L, 31L),
hour = c(0L, 23L),
origin = rep("^\\w{3,3}$",2)),
select = TRUE)
weather
#> # A tibble: 26,130 x 5
#> year month day hour origin
#> <dbl> <dbl> <int> <int> <chr>
#> 1 2013 1 1 0 EWR
#> 2 2013 1 1 1 EWR
#> 3 2013 1 1 2 EWR
#> 4 2013 1 1 3 EWR
#> 5 2013 1 1 4 EWR
#> 6 2013 1 1 6 EWR
#> 7 2013 1 1 7 EWR
#> 8 2013 1 1 8 EWR
#> 9 2013 1 1 9 EWR
#> 10 2013 1 1 10 EWR
#> # ... with 26,120 more rows
Joins
Checking the referential integrity of the (many-to-one) join between flights and airlines is easy.
check_join(flights, airlines, join = "carrier")
In addition to tailnum, flights and planes have additional column with the same name.
check_join(flights, planes, join = "tailnum")
#> Error: flights and planes must not have additional matching columns
We can deal with this by setting extra = TRUE but the data fail referential integrity because we have planes without flights.
check_join(flights, planes, extra = TRUE, join = "tailnum")
#> Error: many-to-one join between flights and planes violates referential integrity
Installation
To install the most recent release from CRAN
install.packages("datacheckr")
To install the development version from GitHub
# install.packages("devtools")
devtools::install_github("datacheckr")
Contribution
Please report any issues.
Pull requests are always welcome.
Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.