identify_inconsistency: Identify inconsistencies in a dataset

In eHDPrep: Quality Control and Semantic Enrichment of Datasets

Identify inconsistencies in a dataset

Description

Tests variables for consistency between their values according to a table of rules or 'consistency table'.

Usage

identify_inconsistency(data = NULL, consis_tbl = NULL, id_var = NULL)

Arguments

data	data frame which will be checked for internal consistency
consis_tbl	data frame or tibble containing information on internal consistency rules (see "Consistency Table Requirements" section)
id_var	An unquoted expression which corresponds to a variable in data which identifies each row.

Details

Multiple types of checks for inconsistency are supported:

1. Comparing by logical operators (<, <=, ==, !=, >=, >)

2. Comparing permitted categories (e.g. cat1 in varA only if cat2 in varB)

3. Comparing permitted numeric ranges (e.g. 20-25 in varC only if 10-20 in varD)

4. Mixtures of 2 and 3 (e.g. cat1 in varA only if 20-25 in varC)

5. Constraining a single variable against fixed values (e.g. varA must be >= 70, or must lie in 0-120), by leaving the second column NA

The consistency tests rely on such rules being specified in a separate data frame (consis_tbl; see section "Consistency Table Requirements").

Variable A is given higher priority than Variable B when A is a category. If A (as char) is not equal to the value in col 4, the check is not made. This is to account for one way dependencies (i.e. VarA is fruit, VarB is apple)

Value

tibble detailing any identified internal inconsistencies in data, if any are found. If no inconsistencies are found, data is returned invisibly.

Consistency Table Requirements

Table must have exactly five character columns. The columns should be ordered according to the list below which describes the values of each column:

1. First column name of data values that will be subject to consistency checking. String. Required.

2. Second column name of data values that will be subject to consistency checking. String. Required for a two-variable check; NA for a single-variable check (see below).

3. Logical test to compare columns one and two. One of: ">",">=", "<","<=","==", "!=". String. Optional if columns 4 and 5 have non-NA values.

4. Either a single character string or a colon-separated range of numbers which should only appear in column A. Optional if column 3 has a non-NA value.

5. Either a single character string or a colon-separated range of numbers which should only appear in column B given the value/range specified in column 4. Optional if column 3 has a non-NA value.

Each row should detail one test to make. Therefore, either column 3 or columns 4 and 5 must contain non-NA values.

Single-variable checks. A rule may also constrain one variable against fixed values, rather than comparing two variables. Such a rule has NA in column 2, the boundary in column 4, and NA in column 5. Two forms are available:

• Comparison against a value: column 3 holds the logical test and column 4 the value to compare against. For example "age", NA, ">=", "70", NA requires every value of age to be at least 70.

• Permitted values: column 3 is NA and column 4 holds either a colon-separated numeric range or a single category. For example "age", NA, NA, "0:120", NA requires every value of age to lie between 0 and 120, and "sex", NA, NA, "Male", NA requires every value of sex to be "Male".

Unlike two-variable rules, single-variable rules may use columns 3 and 4 together. Missing (NA) values are never reported as inconsistent by a single-variable check, as they cannot be assessed against a boundary.

See Also

Other internal consistency functions: validate_consistency_tbl()

Examples

require(tibble)
# example with synthetic dataset on number of bean counts
# there is a lot going on in the function so a simple dataset aids this example
#
# creating `data`:
beans <- tibble::tibble(red_beans = 1:15,
blue_beans = 1:15,
total_beans = 1:15*2,
red_bean_summary = c(rep("few_beans",9), rep("many_beans",6)))
#
# creating `consis_tbl`
bean_rules <- tibble::tribble(~varA, ~varB, ~lgl_test, ~varA_boundaries, ~varB_boundaries,
"red_beans", "blue_beans", "==", NA, NA,
"red_beans", "total_beans", "<=", NA,NA,
"red_beans", "red_bean_summary", NA, "1:9", "few_beans",
"red_beans", "red_bean_summary", NA, "10:15", "many_beans")

identify_inconsistency(beans, bean_rules)

# creating some inconsistencies as examples
beans[1, "red_bean_summary"] <- "many_beans"
beans[1, "red_beans"] <- 10

identify_inconsistency(beans, bean_rules)

# single-variable rules: the second column is NA and the boundary is in the
# fourth column. Here no more than 10 red beans are permitted, so the rows
# holding 11 to 15 are reported:
bean_limits <- tibble::tribble(~varA, ~varB, ~lgl_test, ~varA_boundaries, ~varB_boundaries,
"red_beans", NA, "<=", "10", NA)

identify_inconsistency(beans, bean_limits)

eHDPrep documentation built on Oct. 21, 2026, 5:10 p.m.