hmatch: Match sets of hierarchical variables between a raw and...

In epicentre-msf/hmatch: Tools for Cleaning and Matching Hierarchically-Structured Data

Match sets of hierarchical variables between a raw and reference dataset

Description

Match sets of hierarchical values (e.g. province, county, township) in a raw, messy dataset to corresponding values within a reference dataset, optionally accounting for discrepancies between the datasets such as:

• variation in character case, use of accents, or spelling

• variation in hierarchical resolution (e.g. some entries specified to municipality but others only to region)

• missing values at one or more hierarchical levels

Usage

hmatch(
  raw,
  ref,
  pattern,
  pattern_ref = pattern,
  by,
  by_ref = by,
  type = "left",
  allow_gaps = TRUE,
  fuzzy = FALSE,
  fuzzy_method = "osa",
  fuzzy_dist = 1L,
  dict = NULL,
  ref_prefix = "ref_",
  std_fn = string_std,
  ...
)

Arguments

raw	data frame containing hierarchical columns with raw data
ref	data frame containing hierarchical columns with reference data
pattern	regex pattern to match the hierarchical columns in raw Note: hierarchical column names can be matched using either the pattern or by arguments. Or, if neither pattern or by are specified, the hierarchical columns are assumed to be all column names that are common to both raw and ref. See specifying_columns.
pattern_ref	regex pattern to match the hierarchical columns in ref. Defaults to pattern, so only need to specify if the hierarchical columns have different names in raw and ref.
by	vector giving the names of the hierarchical columns in raw
by_ref	vector giving the names of the hierarchical columns in ref. Defaults to by, so only need to specify if the hierarchical columns have different names in raw and ref.
type	type of join ("left", "inner", "anti", "resolve_left", "resolve_inner", or "resolve_anti"). Defaults to "left". See join_types.
allow_gaps	logical indicating whether to allow missing values below the match level, where 'match level' is the highest level with a non-missing value within a given row of raw. Defaults to TRUE.
fuzzy	logical indicating whether to use fuzzy-matching (based on the stringdist package). Defaults to FALSE.
fuzzy_method	if fuzzy = TRUE, the method to use for string distance calculation (see stringdist-metrics). Defaults to "osa".
fuzzy_dist	if fuzzy = TRUE, the maximum string distance to use to classify matches (i.e. a string distance less than or equal to fuzzy_dist will be considered matching). Defaults to 1L.
dict	optional dictionary for recoding values within the hierarchical columns of raw (see dictionary_recoding)
ref_prefix	prefix to add to names of returned columns from ref if they are otherwise identical to names within raw. Defaults to "ref_".
std_fn	function to standardize strings during matching. Defaults to string_std. Set to NULL to omit standardization. See also string_standardization.
...	additional arguments passed to std_fn()

Value

a data frame obtained by matching the hierarchical columns in raw and ref, using the join type specified by argument type (see join_types for more details)

Resolve joins

In hmatch, if argument type corresponds to a resolve join, rows of raw with multiple matches to ref are always resolved to 'no match'. This is because hmatch does not accept matches below the highest non-missing level within a given row of raw. E.g.

raw:
⁠1. | United States | <NA> | Jefferson |⁠

Relevant rows from ref:
⁠1. | United States | New York | Jefferson |⁠
⁠2. | United States | Pennsylvania | Jefferson |⁠

In a regular join with hmatch, the single row from raw (above) will match both rows of ref. However, in a resolve join the multiple conflicting matches (i.e. conflicting values at the 2nd hierarchical level) will result in the row from raw being treated as non-matching to ref.

Examples

data(ne_raw)
data(ne_ref)

hmatch(ne_raw, ne_ref, pattern = "adm", type = "inner")

epicentre-msf/hmatch documentation built on Nov. 15, 2023, 1:47 a.m.