multiplyr: Data Manipulation with Parallelism and Shared Memory Matrices

Provides a new form of data frame backed by shared memory matrices and a way to manipulate them. Upon creation these data frames are shared across multiple local nodes to allow for simple parallel processing.

AuthorJim Blundell [aut, cre, cph]
Date of publication2016-05-31 23:24:54
MaintainerJim Blundell <>

View on CRAN

Man pages

add_rownames: Add a new column with row names

arrange: Sort data

between: Tests whether elements of a vector lie between two values...

bm_morder: Extension of bigmemory::morder to allow decreasing parameter...

bm_mpermute: Extension of bigmemory::mpermute to allow decreasing...

cumall: Cumulative all

cumany: Cumulative any

cummean: Cumulative mean

define: Define new columns

desc: Arrange specified column in descending order

distinct: Select unique rows or unique combinations of variables

distribute: Calculations for how to distribute x items over N nodes

dotscapture: Capture ... for later evaluation

dotscombine: Combine explicit and implicit dots

dotseval: Evaluate previously captured dots

dotsname: Ensure captured dots are all named

dotsname1: Name an expression (called by dotsname)

filter: Filter data

first: Returns first value in vector

group_by: Group data

group_sizes: Return size of groups

lag: Offset x backwards by n

last: Returns last value in vector

lead: Offset x forwards by n

multiplyr: Data Manipulation with Parellelism and Shared Memory Matrices

Multiplyr-class: Parallel processing data frame

Multiplyr-methods: Data access methods for Multiplyr

mutate: Change values of existing variables (and create new ones)

n: Number of items in current group

NA_class_: Returns NA of a particular class

n_distinct: Return the number of unique values

n_groups: Return number of groups

nonunique: Returns values of x that are non-unique

nsa: No strings attached mode

nth: Return the nth item from a vector

p: Concatenate (internal)

partition_even: Partition data evenly amongst cluster nodes

partition_group: Partition data so that each group is wholly on a node

reduce: Summarise data (with local reduction)

regroup: Return to grouped data

rename: Rename variables

select: Retain only specified variables

shutdown: Shutdown running cluster

slice: Select rows by position

sm_desc_comp: Returns big.matrix descriptor offset by 1 (for row by row...

sm_desc_group: Returns a big.matrix descriptor for a particular group ID

sm_desc_subset: Returns big.matrix descriptor limited to particular start/end...

sm_desc_update: Update description of a big.matrix after a row subset...

summarise: Summarise data

test_transition: Test for grouping transition (internal)

transmute: Change variables and drop all others

undefine: Delete variables

ungroup: Return data to non-grouped

within_group: Execute code within a group

within_node: Execute code within a node


add_rownames Man page
arrange Man page
arrange_ Man page,Multiplyr-method Man page
between Man page
bm_morder Man page
bm_mpermute Man page
cumall Man page
cumany Man page
cummean Man page
define Man page
define_ Man page
desc Man page
dimnames,Multiplyr-method Man page
distinct Man page
distinct_ Man page
distribute Man page
dotscapture Man page
dotscombine Man page
dotseval Man page
dotsname Man page
dotsname1 Man page
filter Man page
filter_ Man page
first Man page
group_by Man page
group_by_ Man page
group_sizes Man page
groupwise Man page
lag Man page
last Man page
lead Man page
multiplyr Man page
Multiplyr Man page
[,Multiplyr,ANY,ANY,ANY-method Man page
[<-,Multiplyr,ANY,ANY-method Man page
Multiplyr-class Man page
Multiplyr-methods Man page
multiplyr-package Man page
mutate Man page
mutate_ Man page
n Man page
NA_class_ Man page
names,Multiplyr-method Man page
n_distinct Man page
n_groups Man page
nonunique Man page
nsa Man page
nth Man page
.p Man page
partition_even Man page
partition_group Man page
partition_group_ Man page
reduce Man page
reduce_ Man page
regroup Man page
rename Man page
rename_ Man page
row.names,Multiplyr-method Man page
rowwise Man page
select Man page
select_ Man page
shutdown Man page
slice Man page
sm_desc_comp Man page
sm_desc_group Man page
sm_desc_subset Man page
sm_desc_update Man page
summarise Man page
summarise_ Man page
test_transition Man page
transmute Man page
transmute_ Man page
undefine Man page
undefine_ Man page
ungroup Man page
unselect Man page
unselect_ Man page
within_group Man page
within_node Man page


multiplyr/R/multiplyr.R multiplyr/R/internal.R multiplyr/R/nse.R multiplyr/R/ops.R multiplyr/R/util.R multiplyr/R/class.R
multiplyr/man/NA_class_.Rd multiplyr/man/group_sizes.Rd multiplyr/man/group_by.Rd multiplyr/man/n.Rd multiplyr/man/cumany.Rd multiplyr/man/sm_desc_comp.Rd multiplyr/man/within_node.Rd multiplyr/man/last.Rd multiplyr/man/transmute.Rd multiplyr/man/n_groups.Rd multiplyr/man/dotscapture.Rd multiplyr/man/nsa.Rd multiplyr/man/first.Rd multiplyr/man/mutate.Rd multiplyr/man/distinct.Rd multiplyr/man/partition_group.Rd multiplyr/man/sm_desc_update.Rd multiplyr/man/cummean.Rd multiplyr/man/dotsname.Rd multiplyr/man/desc.Rd multiplyr/man/partition_even.Rd multiplyr/man/bm_mpermute.Rd multiplyr/man/between.Rd multiplyr/man/p.Rd multiplyr/man/lead.Rd multiplyr/man/n_distinct.Rd multiplyr/man/multiplyr.Rd multiplyr/man/add_rownames.Rd multiplyr/man/regroup.Rd multiplyr/man/arrange.Rd multiplyr/man/rename.Rd multiplyr/man/cumall.Rd multiplyr/man/within_group.Rd multiplyr/man/Multiplyr-class.Rd multiplyr/man/nth.Rd multiplyr/man/dotscombine.Rd multiplyr/man/test_transition.Rd multiplyr/man/sm_desc_group.Rd multiplyr/man/shutdown.Rd multiplyr/man/dotsname1.Rd multiplyr/man/filter.Rd multiplyr/man/define.Rd multiplyr/man/bm_morder.Rd multiplyr/man/undefine.Rd multiplyr/man/ungroup.Rd multiplyr/man/sm_desc_subset.Rd multiplyr/man/slice.Rd multiplyr/man/Multiplyr-methods.Rd multiplyr/man/reduce.Rd multiplyr/man/dotseval.Rd multiplyr/man/distribute.Rd multiplyr/man/nonunique.Rd multiplyr/man/lag.Rd multiplyr/man/select.Rd multiplyr/man/summarise.Rd

Questions? Problems? Suggestions? or email at

Please suggest features or report bugs with the GitHub issue tracker.

All documentation is copyright its authors; we didn't write any of that.