View source: R/conform_table.R
| conform_table | R Documentation |
clean_table() makes the fixes that are unambiguously safe: it legalises
column names and trims whitespace, and it reports near-duplicate labels
without merging them. conform_table() is the next step and makes the two
decisions that need judgement: deciding that
"NSW" and "Nsw" are one category, and deciding that a character column
holding six labels is a factor.
conform_table(
df,
merge_labels = c("report", "auto", "off"),
types = c("report", "auto", "off"),
max_levels = 20L,
clean = c("auto", "report", "off"),
quiet = FALSE
)
df |
A data frame. |
merge_labels |
One of |
types |
One of |
max_levels |
Cardinality at or below which a character column is
treated as categorical. Default |
clean |
Passed to |
quiet |
Logical. When |
Both are off by default: the default "report" mode lists every change
it would make, in plain words, and applies none. Set a mode to "auto" to
apply them; the assumption behind each one is recorded in the returned
object.
Storage is decided before categories, so a column of numbers or dates
held as text is typed first and "4.2" is never merged into "4.4". Only
columns that remain categorical are considered for merging, and a
one-character edit between labels shorter than four characters is ignored
("a" and "b" are one edit apart).
Two labels that differ by an edit are merged only when one spelling is
clearly more common; a column is converted only when every value parses;
numeric columns keep their storage. Anything not changed for these reasons
is reported as a not applied row with the reason.
Merges follow chains: if "Nsw" merges onto "nsw" and "nsw" onto
"NSW", both end on "NSW". A label whose merges end on two different
labels ("colur" beside "color" and "colour" that occur equally
often) is not merged; it is reported, and a masque_conform_ambiguous_merge
warning is raised under merge_labels = "auto". Codes that differ only
inside a run of digits ("G01" and "G02") are never treated as
near-duplicates. Which columns hold categories is decided the same way
whatever types is, so types = "off" still merges labels and leaves
storage as it is.
Pass the result to mask() (or as a table of mask_set()): the recipe
records the merges and type changes, and apply_recipe() replays them on
the original so a pipeline built on the conformed clone sees the same
labels and storage.
An object of class masque_conformance: a list with
data - the conformed data.table;
cleaning - the masque_cleaning record from clean_table();
merges - one row per near-duplicate pair considered
(col, from, to, n_from, n_to, applied, reason);
merge_map - the merges as applied to the data (col, from,
to), one row per merged-away label, each to a label left in
the column. This is what mask() records and apply_recipe()
replays;
types - one row per column whose storage was considered
(col, from, to, applied, reason);
assumptions - every decision as a plain sentence
(col, assumption, applied), which is the thing to read;
modes - the modes applied;
max_levels - the cardinality threshold used.
clean_table() for the safe fixes, propose_roles() for what a
column is, mask() for the round-trip.
df <- data.frame(
state = c("NSW", "Nsw", "NSW", "VIC", "VIC"),
yield = c("3.1", "2.9", "5.0", "4.2", "3.8"),
sown = c("2024-05-01", "2024-05-03", "2024-05-01", "2024-05-08",
"2024-05-08"),
stringsAsFactors = FALSE
)
# report only: nothing is changed
cf <- conform_table(df, quiet = TRUE)
cf$assumptions
# apply the decisions
cf2 <- conform_table(df, merge_labels = "auto", types = "auto", quiet = TRUE)
str(cf2$data)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.