#| include: false knitr::opts_chunk$set(warning = FALSE) library(basetable) .old_opts <- options(width = 90, digits = 4)
basetable is for people who already know base R and want fast table
operations without learning data.table's [i, j, by] syntax or dplyr's tidy
evaluation. It is aimed specifically at two audiences: teaching table
manipulation to people learning base R (without the extra cognitive load of
non-standard evaluation or terse [i, j, by] syntax), and migrating a
codebase already built on base-R table semantics (subset(), merge(),
aggregate(), and friends) onto a faster engine without a rewrite.
It is not trying to replace dplyr or data.table for a new project that's free
to pick any tool; both are more established choices with a larger ecosystem.
Design goals:
subset(),
transform(), aggregate(), merge(), and split(), the base R verbs
most R users already know, rather than inventing a new grammar.data.table's [i, j, by] syntax and without a
data.table dependency.subset()'s and transform()'s
expression arguments, which mirror base R's own subset()/transform()
behavior).basetable deliberately does not ship the dplyr-coined verbs
(filter(), select(), mutate(), arrange(), summarise(),
distinct(), glimpse(), slice(), relocate(), bind_rows(),
bind_cols()), so it can be attached alongside dplyr without shadowing
its grammar. The two names it does share with dplyr are count() and
pick(), kept because they read as base-style verbs. It also reuses true
base-R names like subset(), merge(), transform() and split(), which
data.table also defines. Whenever two attached packages export the same
name the one attached last wins and silently shadows the other: call
basetable::fn() explicitly (or use the
conflicted package) if you need both
loaded at once.
This document walks through every function basetable exports, grouped
into sections that build on each other: start with the core verbs, move
through aggregation, joining, reshaping, and exploration, then into the
lower-level data-cleaning, string, date, and numeric utility functions. Read
it top to bottom to learn the package, or jump to a section as a reference.
Every example below is real, runnable code: the output shown is the actual
output basetable produces.
These are the everyday functions: subsetting rows, picking columns, and
creating new columns. If you know mtcars, you know enough to follow along.
subset()subset() mirrors base::subset() exactly: a logical condition for rows, and
an optional select for columns.
subset(mtcars, cyl == 6, select = c("mpg", "hp", "wt"))
Pass by to evaluate the condition within each group, so an aggregate
inside it is per group. This is basetable's grouped filter; there is no
stateful group_by(), the grouping is named at the call.
subset(mtcars, mpg > mean(mpg), by = "cyl")
pick(), drop()pick() keeps only the named columns; drop() removes them.
pick(mtcars, c("mpg", "hp")) drop(mtcars, c("vs", "am"))
transform(), within()transform() adds or replaces columns using named expressions, exactly like
base::transform(). Later expressions in the same call can refer to columns
created earlier in that call.
transform(mtcars, power = hp / wt, power_sq = power^2)
within() instead evaluates a whole block of code against the data's columns
and returns whatever new/changed variables come out: this is basetable's
version of base::within().
within(mtcars, { power <- hp / wt heavy <- wt > 3.5 })
renamecols()renamecols() renames columns using new = old pairs. Old column names may
be supplied as bare names or character strings.
mtcars |> subset(cyl == 6 & mpg > 18, select = c("mpg", "cyl", "hp")) |> transform(ratio = hp / mpg) |> orderrows(by = "ratio", decreasing = TRUE)
transform(mtcars, ratio = hp / mpg, .keep = FALSE) renamecols(mtcars, horsepower = hp) |> pick(c("horsepower", "mpg"))
aggregate()aggregate() computes one summary function over one or more value columns,
per group, a faster analogue of stats::aggregate(). Note it returns a
bt_table directly, since it's meant to be fast and composable with further
basetable operations.
aggregate(mtcars, by = "cyl", value = c("mpg", "hp"), fun = mean)
count(), propcount()count() counts rows per group; propcount() adds proportions, optionally
computed within a margin of columns rather than over the whole table.
count(iris, by = "Species") propcount(mtcars, by = c("cyl", "am"), margin = "cyl")
summaries()summaries() takes named summary expressions directly, with by as an
argument, useful when you're calling it programmatically:
summaries(mtcars, "cyl", mean_hp = mean(hp), sd_hp = sd(hp))
split() and applyby()split() breaks a table into a named list of bt_tables, one per group, using
the native grouping engine internally:
pieces <- split(iris, by = "Species") names(pieces) pieces$setosa |> headtail(2)
applyby() applies a function to each group through the same grouping
split path and, if bind = TRUE, stitches the results back into one table:
applyby(mtcars, by = "cyl", fun = function(d) data.frame(mean_mpg = mean(d$mpg)), bind = TRUE, id = "cyl_group")
firstby(), lastby()df <- data.frame(id = c(1, 1, 2, 2, 2), visit = c(1, 2, 1, 2, 3), value = c(10, 12, 5, 6, 7)) firstby(df, by = "id") lastby(df, by = "id")
basetable provides the full family of table joins, all engine-backed.
merge()merge() mirrors base::merge()'s interface exactly (by, all, all.x,
all.y, suffixes) while running on the native join engine internally.
orders <- data.frame(id = c(1, 2, 3), customer = c("a", "b", "c")) payments <- data.frame(id = c(2, 3, 4), amount = c(50, 75, 20)) merge(orders, payments, by = "id") merge(orders, payments, by = "id", all.x = TRUE)
semimerge(), antimerge()semimerge() keeps rows of x whose key exists in y (like merge() but
without adding y's columns); antimerge() keeps rows of x whose key does
not exist in y.
semimerge(orders, payments, by = "id") antimerge(orders, payments, by = "id")
matchedkeys(), unmatchedkeys(), joinrelationship()matchedkeys()/unmatchedkeys() return x's distinct rows whose key is (or
isn't) present in y. joinrelationship() classifies the cardinality of a
join key between two tables.
matchedkeys(orders, payments, by = "id") unmatchedkeys(orders, payments, by = "id") joinrelationship(orders, payments, by = "id")
crossmerge()crossmerge() returns the full Cartesian product of two tables.
crossmerge(data.frame(size = c("S", "M")), data.frame(color = c("red", "blue")))
rollingmerge(), nearestmerge()rollingmerge() matches each row of x to the nearest row of y at or
before it (direction = "backward"), at or after it ("forward"), or
whichever is numerically closest ("nearest"), the classic "as-of" join used
for time series and event logs. nearestmerge() is a convenience wrapper for
direction = "nearest".
trades <- data.frame(id = 1L, time = c(5, 10, 15)) quotes <- data.frame(id = 1L, time = c(3, 8, 14), price = c(100, 101, 99)) rollingmerge(trades, quotes, by = c("id", "time"), direction = "backward") nearestmerge(trades, quotes, by = c("id", "time"))
overlapmerge()overlapmerge() matches rows whose [startx, endx] interval overlaps a
[starty, endy] interval in y (optionally within an exact by key), for
example, matching events to the time windows they fall inside.
events <- data.frame(id = 1L, start = c(1, 10), end = c(5, 15)) windows <- data.frame(id = 1L, start = c(0, 8), end = c(6, 20), label = c("early", "late")) overlapmerge(events, windows, startx = "start", endx = "end", starty = "start", endy = "end", by = "id")
updatemerge()updatemerge() overwrites x's values with y's wherever the join key
matches (an in-place "upsert" of column values, not a column-adding merge).
current <- data.frame(id = c(1, 2, 3), status = c("pending", "pending", "pending")) updates <- data.frame(id = c(2, 3), status = c("shipped", "cancelled")) updatemerge(current, updates, by = "id")
nonequimerge(), rangemerge()nonequimerge() supports genuine inequality conditions in by, written as
"xcol<op>ycol",
alongside any plain exact-match columns:
events <- data.frame(id = 1, date = as.Date("2024-01-15")) periods <- data.frame( id = 1, start_date = as.Date(c("2024-01-01", "2024-02-01")), end_date = as.Date(c("2024-01-31", "2024-02-28")), period = c("Jan", "Feb") ) nonequimerge(events, periods, by = c("id", "date>=start_date", "date<=end_date"))
rangemerge() keeps every row of x (a table of ranges), attaching the row(s)
of y whose value column falls inside that range, matched within by. An
x row with no y row in range still appears once, with NA for y's
columns.
ranges <- data.frame(id = c(1, 1, 2), lower = c(0, 0, 0), upper = c(10, 10, 20)) points <- data.frame(id = c(1, 1, 2), val = c(5, 15, 5), label = c("a", "b", "c")) rangemerge(ranges, points, by = "id", lower = "lower", upper = "upper", value = "val")
unionrows(), intersectrows(), diffrows()a <- data.frame(id = c(1, 2, 3)) b <- data.frame(id = c(2, 3, 4)) unionrows(a, b) intersectrows(a, b, by = "id") diffrows(a, b, by = "id")
equalrows(), equaldata(), sameschema(), compareschema(), changedcols()equalrows() aligns two tables by a key and compares every column
(order-insensitive, duplicate-sensitive), not just the key itself.
equaldata() compares full table content directly. sameschema() and
compareschema() (aliased for this purpose by changedcols()) compare column
names and types.
equalrows(a, data.frame(id = c(3, 2, 1)), by = "id") equaldata(a, a[nrow(a):1, , drop = FALSE], ignoreorder = TRUE) sameschema(mtcars, mtcars) compareschema(mtcars, iris)
addedrows(), removedrows(), changedrows()changedrows() returns only the rows whose key exists in both snapshots
and whose non-key values actually differ: a row whose key matches but
whose values are identical is not "changed".
old <- data.frame(id = c(1, 2, 3), status = c("a", "b", "c")) new <- data.frame(id = c(2, 3, 4), status = c("b", "c2", "d")) addedrows(old, new, by = "id") removedrows(old, new, by = "id") changedrows(old, new, by = "id")
rbindfill(), cbind()rbindfill() stacks tables, filling missing columns with NA. For binding
tables side by side, use base R's own cbind().
rbindfill(data.frame(x = 1), data.frame(x = 2, y = 3)) cbind(data.frame(a = 1:2), data.frame(b = 3:4))
tolong(), towide()wide <- data.frame(id = 1:2, jan = c(10, 20), feb = c(11, 22)) long <- tolong(wide, cols = c("jan", "feb"), names = "month", values = "sales") long towide(long, names = "month", values = "sales", idcols = "id", fun = sum)
reshape(), stack(), unstack()reshape(), stack(), and unstack() are basetable-native wrappers around
stats::reshape(), utils::stack(), and utils::unstack(), kept for
base-R compatibility rather than reimplemented on the engine.
wide2 <- data.frame(id = 1:2, t1 = c(10, 20), t2 = c(11, 22)) reshape(wide2, varying = c("t1", "t2"), v.names = "value", timevar = "time", idvar = "id", direction = "long") stack(data.frame(a = 1:2, b = 3:4))
separate(), unite()codes <- data.frame(code = c("A-1-red", "B-2-blue")) parts <- separate(codes, column = "code", into = c("letter", "num", "color"), sep = "-") parts unite(parts, column = "code", cols = c("letter", "num", "color"), sep = "_")
transpose()transpose(data.frame(a = 1:2, b = 3:4))
dims(), types(), nrows(), ncols()dims(iris) types(iris)
headtail(), preview()headtail(iris, 2) preview(iris)
describe(), profile()describe() gives a per-column summary (mean, sd, quantiles for numeric
columns; top values for others). profile() is an alias.
describe(iris)
freq()freq(iris, column = "Species") freq(mtcars, column = "cyl", by = "am", prop = TRUE)
summarytab()dat <- transform(mtcars, am = factor(am, labels = c("Automatic", "Manual"))) summarytab(dat, vars = c("mpg", "cyl"), by = "am", p_value = TRUE)
missingness()with_na <- transform(iris, Sepal.Length = ifelse(Sepal.Length > 7, NA, Sepal.Length)) missingness(with_na)
compare()compare(iris, with_na, by = "Species")
assert*()Each assert*() function returns its input invisibly if the check passes, and
throws an informative error otherwise, useful as a pipeline guard.
mtcars |> assertnames(c("mpg", "cyl")) |> assertrows(mpg > 0) |> assertrange("mpg", lower = 0, upper = 60) |> asserttype("cyl", "numeric") |> assertcomplete() |> headtail(2)
tryCatch(assertkey(mtcars, "cyl"), error = function(e) conditionMessage(e)) tryCatch(assertunique(mtcars, "cyl"), error = function(e) conditionMessage(e)) tryCatch(assertvalues(mtcars, "am", allowed = c(0, 1)), error = function(e) "passes")
assertcols()/assertkey() are convenience aliases for the lower-level
assert_cols()/assert_key(), which do the same checks and are what you'd
call from code that doesn't need the assert* naming convention:
assert_cols(mtcars, c("mpg", "cyl")) |> invisible() tryCatch(assert_key(mtcars, "cyl"), error = function(e) conditionMessage(e))
invalidrows(), invalidvalues(), outofrange()These are the non-throwing counterparts of the assert*() functions above:
instead of stopping, they return the offending rows.
invalidrows(mtcars, mpg > 15) outofrange(mtcars, "mpg", lower = 15, upper = 30)
missingrows(), keepmissing(), omitmissing(), missingindicator()d <- data.frame(a = c(1, NA, 3), b = c(NA, NA, 3)) missingrows(d, mode = "any") keepmissing(d) omitmissing(d, mode = "any") missingindicator(d)
completegrid(), expandrows()completegrid() fills in every combination of the named columns that's
missing from the data (useful before time-series analysis). expandrows()
repeats each row a given number of times.
sparse <- data.frame(site = c("A", "A", "B"), year = c(2020, 2021, 2020), value = c(1, 2, 3)) completegrid(sparse, cols = c("site", "year"), fill = list(value = 0)) expandrows(data.frame(x = c(1, 2)), times = c(2, 1))
naif(), nato(), blanktona(), natoblank(), replacevalues(), replacewhere(), replacecols()naif(c(1, 99, 2), 99) nato(c(1, NA, 2), 0) blanktona(c("a", "", "b")) replacevalues(c("a", "b", "c"), old = c("a", "b"), new = c("A", "B")) replacewhere(mtcars, cyl == 4, cols = "mpg", value = NA) |> headtail(2)
uniquerows(), removeduplicates(), duplicaterows(), duplicatekeys(), duplicatenames()dup_df <- data.frame(id = c(1, 1, 2), v = c("a", "b", "c")) uniquerows(dup_df, cols = "id") removeduplicates(dup_df, by = "id", keep = "first") duplicaterows(data.frame(x = c(1, 1, 2))) duplicatekeys(dup_df, "id")
duplicatekeys() is a convenience alias for the lower-level duplicated_keys():
duplicated_keys(dup_df, "id")
cleannames(), repairnames(), renamewith(), commonnames()messy <- data.frame(`First Name` = 1, `2nd_col` = 2, check.names = FALSE) cleannames(messy) commonnames(mtcars, iris)
commonnames() is a convenience alias for the lower-level common_names():
common_names(mtcars, iris)
filldown(), fillup(), fillboth()filldown() carries the last non-missing value forward within each group
(last observation carried forward); fillup() carries the next non-missing
value backward; fillboth() does both, filling any remaining gaps from
either direction.
visits <- data.frame( id = c(1, 1, 1, 2, 2), visit = c(1, 2, 3, 1, 2), treatment = c("A", NA, NA, NA, "B") ) filldown(visits, cols = "treatment", by = "id") fillup(visits, cols = "treatment", by = "id") fillboth(visits, cols = "treatment", by = "id")
colnames(), rownames(), classes(), uniques(), cardinality(), constants(), emptycols()colnames(iris) classes(iris) uniques(iris) cardinality(iris, cols = "Species") constants(data.frame(a = 1, b = 1:2)) emptycols(data.frame(a = c(NA, NA), b = 1:2))
emptyrows()emptyrows(data.frame(a = c(NA, "x", ""), b = c(NA, "y", NA)))
move(), firstcols(), lastcols()move(mtcars, "hp", before = "mpg") |> colnames() firstcols(mtcars, "wt") |> colnames() lastcols(mtcars, "mpg") |> colnames()
firstrows(), lastrows(), samplerows(), samplefrac(), reverse()firstrows(mtcars, 3) lastrows(mtcars, 2) set.seed(1) samplerows(mtcars, 2)
samplerows() and samplefrac() take by to sample within each group
(n is capped at the group size); sampled rows keep their original order.
set.seed(1) samplerows(mtcars, 2, by = "cyl")
For an arbitrary set of row positions, index the table directly, the same way you would any base R data frame:
firstrows(mtcars, c(1, 3, 5))
orderrows()orderrows(mtcars, by = "mpg", decreasing = TRUE) |> headtail(2)
rownumber()rownumber(c("a", "b", "c"))
rowmin(), rowmax(), rowany(), rowall(), rowcount(), rowfirst(), rowlast(), rowapply()d <- data.frame(x = c(1, NA, 3), y = c(4, 5, NA), z = c(TRUE, FALSE, TRUE)) rowmin(d, cols = c("x", "y"), na.rm = TRUE) rowmax(d, cols = c("x", "y"), na.rm = TRUE) rowfirst(d, cols = c("x", "y"), na.rm = TRUE) rowlast(d, cols = c("x", "y"), na.rm = TRUE) rowapply(d, cols = c("x", "y"), fun = function(row) sum(row, na.rm = TRUE))
applycols(), convertcols()applycols(mtcars, cols = "mpg", fun = round) |> headtail(2) convertcols(mtcars, cols = "cyl", fun = as.character) |> types()
trim(), squish(), lower(), upper(), titlecase(), sentencecase(), textlen()trim(" hello ") squish(" too many spaces ") lower("HELLO"); upper("hello") titlecase("hello world"); sentencecase("HELLO WORLD") textlen(c("abc", NA))
left(), right(), middle(), truncate(), padleft(), padright(), padcenter()left("hello", 3); right("hello", 3); middle("hello", 2, 4) truncate("a long sentence", 10) padleft("7", 3, pad = "0"); padright("7", 3, pad = "0"); padcenter("hi", 6, pad = "*")
containstext(), matchestext(), startswith(), endswith(), countmatch(), locate(), locateall()containstext(c("apple", "banana"), "an") matchestext(c("cat", "cats"), "cat") startswith(c("apple", "banana"), "a") endswith(c("apple", "banana"), "a") countmatch("banana", "a") locate("banana", "an")
extract(), extractall(), extractnum(), extractint(), extractbetween()extract("order #4231", "[0-9]+") extractall("a1b2c3", "[0-9]") extractnum("total: -12.5 kg") extractbetween("[important]", "\\[", "\\]")
replacetext(), removetext(), removeall(), replaceall()replacetext("a-b-c", "-", "_") removetext("a-b-c", "-") replaceall(c("a", "b"), old = "a", new = "A")
splittext(), splitfirst(), splitlast(), jointext(), collapsetext()splittext("a-b-c", "-") splitfirst("a-b-c", "-"); splitlast("a-b-c", "-") jointext("a", "b", "c") collapsetext(c("a", "b", "c"), sep = ", ")
isblank(), removeaccents(), normalizeunicode(), normalizeencoding(), transliterate()isblank(c("", " ", NA, "x")) removeaccents("café")
textdist(), nearesttext(), similartext()textdist("cat", c("cat", "bat", "dog")) nearesttext("kat", c("cat", "dog", "bird"))
isalpha(), isalphanumeric(), isnumerictext(), isintegertext(), isemail(), isurl()isalpha(c("abc", "a1c")) isnumerictext(c("12.5", "abc")) isemail(c("a@b.com", "not-an-email")) isurl(c("https://example.com", "example.com"))
recode(), collapsevalues(), casewhen(), collapselevels(), lump(), reorderlevels(), expandlevels()recode(c("a", "b", "c"), old = "a", new = "A") collapsevalues(c("cat", "dog", "bird"), groups = list(pet = c("cat", "dog"))) x <- rep(c("a", "b", "c", "d"), c(10, 5, 3, 1)) lump(x, n = 2, other = "Other") |> table() f <- factor(c("low", "high", "mid"), levels = c("low", "mid", "high")) reorderlevels(f, by = c("high", "mid", "low")) expandlevels(f, "extra")
casewhen() maps conditions to a label, taking the same named-list shape as
collapsevalues() but with logical vectors: the first element that is
TRUE at a position gives its value there, and default covers the rest.
It composes inside transform().
transform(mtcars, size = casewhen( list(light = wt < 2.5, mid = wt < 3.5), default = "heavy" )) |> pick(c("wt", "size")) |> headtail(2)
parse*() functions convert messy text into typed vectors, with an na= for
sentinel missing-value strings and a strict= to error instead of silently
returning NA on failure.
parseint(c("1", "2", "NA")) parsenum("1.234,56", decimal = ",", grouping = ".") parselogical(c("TRUE", "false", "T", "0")) parsedate(c("2024-01-15", "15/01/2024"), formats = c("%Y-%m-%d", "%d/%m/%Y")) parsedatetime("2024-01-15 08:30:00", formats = "%Y-%m-%d %H:%M:%S") parsepercent("42%") parsecurrency("$1,234.50") parsefailures(c("1", "x", "3"), parseint)
d <- as.Date("2024-03-15") year(d); month(d); day(d); weekday(d); yearday(d); week(d); quarter(d) t <- as.POSIXct("2024-03-15 08:30:45", tz = "UTC") hour(t); minute(t); second(t)
adddays(), addweeks(), addmonths(), addyears(), datediff(), dateseq(), betweendates()addmonths()/addyears() handle end-of-month overflow explicitly via
invalid = c("previous", "next", "missing", "error"): e.g. what should
January 31st plus one month become, given February doesn't have 31 days?
adddays(d, 10); addweeks(d, 2) addmonths(as.Date("2024-01-31"), 1, invalid = "previous") addmonths(as.Date("2024-01-31"), 1, invalid = "next") datediff(as.Date("2024-01-10"), as.Date("2024-01-01"), units = "days") dateseq(as.Date("2024-01-01"), as.Date("2024-01-05")) betweendates(d, "2024-01-01", "2024-12-31")
floordate(), ceilingdate(), rounddate()floordate(d, "month"); ceilingdate(d, "month"); rounddate(d, "month") floordate(d, "week")
rescale(), standardize(), center(), winsorize()x <- c(1, 2, 3, 4, 100) rescale(x); standardize(x); center(x) winsorize(x, probs = c(0.1, 0.9))
quantilegroup()quantilegroup(1:100, n = 4) |> table()
percentchange(), percentrank(), denserank()denserank() ranks values by sorted order with no gaps between ranks, tied
values sharing a rank (the same semantics as SQL's DENSE_RANK()).
percentchange(c(100, 110, 99)) percentrank(c(10, 20, 20, 30)) denserank(c(30, 10, 20, 10, 30))
cumcount(), cumedist(), cumavg()cumcount(c("a", "b", "c")) cumedist(c("a", "a", "b")) cumavg(c(1, 2, 3))
difference(), lagvalue(), leadvalue()difference(c(1, 3, 6, 10)) lagvalue(1:5, n = 1) leadvalue(1:5, n = 1)
rollmean(), rollsum(), rollmin(), rollmax(), rollmedian(), rollsd(), rollvar(), rollprod(), rollapply()All roll functions share the same interface: a width, an align
("right"/"left"/"center"), a fill for incomplete windows, partial to allow
incomplete boundary windows instead of fill, and (where numerically
meaningful) na.rm.
v <- c(1, 2, NA, 4, 5) rollmean(v, width = 3, na.rm = TRUE) rollsum(v, width = 3, na.rm = TRUE, partial = TRUE) rollmax(v, width = 2, na.rm = TRUE) rollapply(1:5, width = 3, FUN = sum, partial = TRUE)
map()/traverse()/foldr() are lightweight base-R helpers for working with
plain vectors and lists (not tables), useful for the occasional
loop-replacement inside a larger basetable pipeline.
map(1:3, function(x) x + 0.5) traverse(list(a = 1:2, b = 10:11), function(a, b) a + b) foldr(1:4, `+`)
This document covers all of basetable's exported functions. For the
performance story (how basetable compares to base R and dplyr, and where
its overhead was tracked down and fixed), see the companion benchmarking
vignette.
#| include: false options(.old_opts)
Any scripts or data that you put into this service are public.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.