col_is_date | R Documentation |
Date
objects?The col_is_date()
validation function, the expect_col_is_date()
expectation function, and the test_col_is_date()
test function all check
whether one or more columns in a table is of the R Date
type. Like many
of the col_is_*()
-type functions in pointblank, the only requirement is
a specification of the column names. The validation function can be used
directly on a data table or with an agent object (technically, a
ptblank_agent
object) whereas the expectation and test functions can only
be used with a data table. Each validation step or expectation will operate
over a single test unit, which is whether the column is a Date
-type column
or not.
col_is_date(
x,
columns,
actions = NULL,
step_id = NULL,
label = NULL,
brief = NULL,
active = TRUE
)
expect_col_is_date(object, columns, threshold = 1)
test_col_is_date(object, columns, threshold = 1)
x |
A data frame, tibble ( |
columns |
The column (or a set of columns, provided as a character vector) to which this validation should be applied. |
actions |
A list containing threshold levels so that the validation step
can react accordingly when exceeding the set levels. This is to be created
with the |
step_id |
One or more optional identifiers for the single or multiple
validation steps generated from calling a validation function. The use of
step IDs serves to distinguish validation steps from each other and provide
an opportunity for supplying a more meaningful label compared to the step
index. By default this is |
label |
An optional label for the validation step. This label appears in the agent report and for the best appearance it should be kept short. |
brief |
An optional, text-based description for the validation step. If
nothing is provided here then an autobrief is generated by the agent,
using the language provided in |
active |
A logical value indicating whether the validation step should
be active. If the validation function is working with an agent, |
object |
A data frame, tibble ( |
threshold |
A simple failure threshold value for use with the
expectation ( |
For the validation function, the return value is either a
ptblank_agent
object or a table object (depending on whether an agent
object or a table was passed to x
). The expectation function invisibly
returns its input but, in the context of testing data, the function is
called primarily for its potential side-effects (e.g., signaling failure).
The test function returns a logical value.
The types of data tables that are officially supported are:
data frames (data.frame
) and tibbles (tbl_df
)
Spark DataFrames (tbl_spark
)
the following database tables (tbl_dbi
):
PostgreSQL tables (using the RPostgres::Postgres()
as driver)
MySQL tables (with RMySQL::MySQL()
)
Microsoft SQL Server tables (via odbc)
BigQuery tables (using bigrquery::bigquery()
)
DuckDB tables (through duckdb::duckdb()
)
SQLite (with RSQLite::SQLite()
)
Other database tables may work to varying degrees but they haven't been formally tested (so be mindful of this when using unsupported backends with pointblank).
If providing multiple column names, the result will be an expansion of
validation steps to that number of column names (e.g., vars(col_a, col_b)
will result in the entry of two validation steps). Aside from column names in
quotes and in vars()
, tidyselect helper functions are available for
specifying columns. They are: starts_with()
, ends_with()
, contains()
,
matches()
, and everything()
.
Often, we will want to specify actions
for the validation. This argument,
present in every validation function, takes a specially-crafted list object
that is best produced by the action_levels()
function. Read that function's
documentation for the lowdown on how to create reactions to above-threshold
failure levels in validation. The basic gist is that you'll want at least a
single threshold level (specified as either the fraction of test units
failed, or, an absolute value), often using the warn_at
argument. This is
especially true when x
is a table object because, otherwise, nothing
happens. For the col_is_*()
-type functions, using action_levels(warn_at = 1)
or action_levels(stop_at = 1)
are good choices depending on the
situation (the first produces a warning, the other will stop()
).
Want to describe this validation step in some detail? Keep in mind that this
is only useful if x
is an agent. If that's the case, brief
the agent
with some text that fits. Don't worry if you don't want to do it. The
autobrief protocol is kicked in when brief = NULL
and a simple brief will
then be automatically generated.
A pointblank agent can be written to YAML with yaml_write()
and the
resulting YAML can be used to regenerate an agent (with yaml_read_agent()
)
or interrogate the target table (via yaml_agent_interrogate()
). When
col_is_date()
is represented in YAML (under the top-level steps
key as a
list member), the syntax closely follows the signature of the validation
function. Here is an example of how a complex call of col_is_date()
as a
validation step is expressed in R code and in the corresponding YAML
representation.
R statement:
agent %>% col_is_date( columns = vars(a), actions = action_levels(warn_at = 0.1, stop_at = 0.2), label = "The `col_is_date()` step.", active = FALSE )
YAML representation:
steps: - col_is_date: columns: vars(a) actions: warn_fraction: 0.1 stop_fraction: 0.2 label: The `col_is_date()` step. active: false
In practice, both of these will often be shorter as only the columns
argument requires a value. Arguments with default values won't be written to
YAML when using yaml_write()
(though it is acceptable to include them with
their default when generating the YAML by other means). It is also possible
to preview the transformation of an agent to YAML without any writing to disk
by using the yaml_agent_string()
function.
The small_table
dataset in the package has a date
column. The following
examples will validate that that column is of the Date
class.
small_table #> # A tibble: 13 x 8 #> date_time date a b c d e f #> <dttm> <date> <int> <chr> <dbl> <dbl> <lgl> <chr> #> 1 2016-01-04 11:00:00 2016-01-04 2 1-bcd-345 3 3423. TRUE high #> 2 2016-01-04 00:32:00 2016-01-04 3 5-egh-163 8 10000. TRUE low #> 3 2016-01-05 13:32:00 2016-01-05 6 8-kdg-938 3 2343. TRUE high #> 4 2016-01-06 17:23:00 2016-01-06 2 5-jdo-903 NA 3892. FALSE mid #> 5 2016-01-09 12:36:00 2016-01-09 8 3-ldm-038 7 284. TRUE low #> 6 2016-01-11 06:15:00 2016-01-11 4 2-dhe-923 4 3291. TRUE mid #> 7 2016-01-15 18:46:00 2016-01-15 7 1-knw-093 3 843. TRUE high #> 8 2016-01-17 11:27:00 2016-01-17 4 5-boe-639 2 1036. FALSE low #> 9 2016-01-20 04:30:00 2016-01-20 3 5-bce-642 9 838. FALSE high #> 10 2016-01-20 04:30:00 2016-01-20 3 5-bce-642 9 838. FALSE high #> 11 2016-01-26 20:07:00 2016-01-26 4 2-dmx-010 7 834. TRUE low #> 12 2016-01-28 02:51:00 2016-01-28 2 7-dmx-010 8 108. FALSE low #> 13 2016-01-30 11:23:00 2016-01-30 1 3-dka-303 NA 2230. TRUE high
agent
with validation functions and then interrogate()
Validate that the column date
has the Date
class.
agent <- create_agent(tbl = small_table) %>% col_is_date(columns = vars(date)) %>% interrogate()
Printing the agent
in the console shows the validation report in the
Viewer. Here is an excerpt of validation report, showing the single entry
that corresponds to the validation step demonstrated here.
agent
)This way of using validation functions acts as a data filter. Data is passed
through but should stop()
if there is a single test unit failing. The
behavior of side effects can be customized with the actions
option.
small_table %>% col_is_date(columns = vars(date)) %>% dplyr::slice(1:5) #> # A tibble: 5 x 8 #> date_time date a b c d e f #> <dttm> <date> <int> <chr> <dbl> <dbl> <lgl> <chr> #> 1 2016-01-04 11:00:00 2016-01-04 2 1-bcd-345 3 3423. TRUE high #> 2 2016-01-04 00:32:00 2016-01-04 3 5-egh-163 8 10000. TRUE low #> 3 2016-01-05 13:32:00 2016-01-05 6 8-kdg-938 3 2343. TRUE high #> 4 2016-01-06 17:23:00 2016-01-06 2 5-jdo-903 NA 3892. FALSE mid #> 5 2016-01-09 12:36:00 2016-01-09 8 3-ldm-038 7 284. TRUE low
With the expect_*()
form, we would typically perform one validation at a
time. This is primarily used in testthat tests.
expect_col_is_date(small_table, columns = vars(date))
With the test_*()
form, we should get a single logical value returned to
us.
small_table %>% test_col_is_date(columns = vars(date)) #> [1] TRUE
2-26
Other validation functions:
col_count_match()
,
col_exists()
,
col_is_character()
,
col_is_factor()
,
col_is_integer()
,
col_is_logical()
,
col_is_numeric()
,
col_is_posix()
,
col_schema_match()
,
col_vals_between()
,
col_vals_decreasing()
,
col_vals_equal()
,
col_vals_expr()
,
col_vals_gte()
,
col_vals_gt()
,
col_vals_in_set()
,
col_vals_increasing()
,
col_vals_lte()
,
col_vals_lt()
,
col_vals_make_set()
,
col_vals_make_subset()
,
col_vals_not_between()
,
col_vals_not_equal()
,
col_vals_not_in_set()
,
col_vals_not_null()
,
col_vals_null()
,
col_vals_regex()
,
col_vals_within_spec()
,
conjointly()
,
row_count_match()
,
rows_complete()
,
rows_distinct()
,
serially()
,
specially()
,
tbl_match()
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.