multiplyr: Data Manipulation with Parallelism and Shared Memory Matrices

Provides a new form of data frame backed by shared memory matrices and a way to manipulate them. Upon creation these data frames are shared across multiple local nodes to allow for simple parallel processing.

Install the latest version of this package by entering the following in R:
AuthorJim Blundell [aut, cre, cph]
Date of publication2016-05-31 23:24:54
MaintainerJim Blundell <>

View on CRAN

Man pages

add_rownames: Add a new column with row names

arrange: Sort data

between: Tests whether elements of a vector lie between two values...

bm_morder: Extension of bigmemory::morder to allow decreasing parameter...

bm_mpermute: Extension of bigmemory::mpermute to allow decreasing...

cumall: Cumulative all

cumany: Cumulative any

cummean: Cumulative mean

define: Define new columns

desc: Arrange specified column in descending order

distinct: Select unique rows or unique combinations of variables

distribute: Calculations for how to distribute x items over N nodes

dotscapture: Capture ... for later evaluation

dotscombine: Combine explicit and implicit dots

dotseval: Evaluate previously captured dots

dotsname: Ensure captured dots are all named

dotsname1: Name an expression (called by dotsname)

filter: Filter data

first: Returns first value in vector

group_by: Group data

group_sizes: Return size of groups

lag: Offset x backwards by n

last: Returns last value in vector

lead: Offset x forwards by n

multiplyr: Data Manipulation with Parellelism and Shared Memory Matrices

Multiplyr-class: Parallel processing data frame

Multiplyr-methods: Data access methods for Multiplyr

mutate: Change values of existing variables (and create new ones)

n: Number of items in current group

NA_class_: Returns NA of a particular class

n_distinct: Return the number of unique values

n_groups: Return number of groups

nonunique: Returns values of x that are non-unique

nsa: No strings attached mode

nth: Return the nth item from a vector

p: Concatenate (internal)

partition_even: Partition data evenly amongst cluster nodes

partition_group: Partition data so that each group is wholly on a node

reduce: Summarise data (with local reduction)

regroup: Return to grouped data

rename: Rename variables

select: Retain only specified variables

shutdown: Shutdown running cluster

slice: Select rows by position

sm_desc_comp: Returns big.matrix descriptor offset by 1 (for row by row...

sm_desc_group: Returns a big.matrix descriptor for a particular group ID

sm_desc_subset: Returns big.matrix descriptor limited to particular start/end...

sm_desc_update: Update description of a big.matrix after a row subset...

summarise: Summarise data

test_transition: Test for grouping transition (internal)

transmute: Change variables and drop all others

undefine: Delete variables

ungroup: Return data to non-grouped

within_group: Execute code within a group

within_node: Execute code within a node


add_rownames Man page
arrange Man page
arrange_ Man page,Multiplyr-method Man page
between Man page
bm_morder Man page
bm_mpermute Man page
cumall Man page
cumany Man page
cummean Man page
define Man page
define_ Man page
desc Man page
dimnames,Multiplyr-method Man page
distinct Man page
distinct_ Man page
distribute Man page
dotscapture Man page
dotscombine Man page
dotseval Man page
dotsname Man page
dotsname1 Man page
filter Man page
filter_ Man page
first Man page
group_by Man page
group_by_ Man page
group_sizes Man page
groupwise Man page
lag Man page
last Man page
lead Man page
multiplyr Man page
Multiplyr Man page
[,Multiplyr,ANY,ANY,ANY-method Man page
[<-,Multiplyr,ANY,ANY-method Man page
Multiplyr-class Man page
Multiplyr-methods Man page
multiplyr-package Man page
mutate Man page
mutate_ Man page
n Man page
NA_class_ Man page
names,Multiplyr-method Man page
n_distinct Man page
n_groups Man page
nonunique Man page
nsa Man page
nth Man page
.p Man page
partition_even Man page
partition_group Man page
partition_group_ Man page
reduce Man page
reduce_ Man page
regroup Man page
rename Man page
rename_ Man page
row.names,Multiplyr-method Man page
rowwise Man page
select Man page
select_ Man page
shutdown Man page
slice Man page
sm_desc_comp Man page
sm_desc_group Man page
sm_desc_subset Man page
sm_desc_update Man page
summarise Man page
summarise_ Man page
test_transition Man page
transmute Man page
transmute_ Man page
undefine Man page
undefine_ Man page
ungroup Man page
unselect Man page
unselect_ Man page
within_group Man page
within_node Man page


tests/testthat/test-internal.R tests/testthat/test-utils.R tests/testthat/test-mutate.R tests/testthat/test-summarise.R tests/testthat/test-subset.R tests/testthat/test-filter.R tests/testthat/test-compact.R tests/testthat/test-groups.R tests/testthat/test-bm.R tests/testthat/test-partition.R tests/testthat/test-arrange.R tests/testthat/test-nsa.R tests/testthat/test-class.R tests/testthat/test-within.R
R/multiplyr.R R/internal.R R/nse.R R/ops.R R/util.R R/class.R
man/NA_class_.Rd man/group_sizes.Rd man/group_by.Rd man/n.Rd man/cumany.Rd man/sm_desc_comp.Rd man/within_node.Rd man/last.Rd man/transmute.Rd man/n_groups.Rd man/dotscapture.Rd man/nsa.Rd man/first.Rd man/mutate.Rd man/distinct.Rd man/partition_group.Rd man/sm_desc_update.Rd man/cummean.Rd man/dotsname.Rd man/desc.Rd man/partition_even.Rd man/bm_mpermute.Rd man/between.Rd man/p.Rd man/lead.Rd man/n_distinct.Rd man/multiplyr.Rd man/add_rownames.Rd man/regroup.Rd man/arrange.Rd man/rename.Rd man/cumall.Rd man/within_group.Rd man/Multiplyr-class.Rd man/nth.Rd man/dotscombine.Rd man/test_transition.Rd man/sm_desc_group.Rd man/shutdown.Rd man/dotsname1.Rd man/filter.Rd man/define.Rd man/bm_morder.Rd man/undefine.Rd man/ungroup.Rd man/sm_desc_subset.Rd man/slice.Rd man/Multiplyr-methods.Rd man/reduce.Rd man/dotseval.Rd man/distribute.Rd man/nonunique.Rd man/lag.Rd man/select.Rd man/summarise.Rd

Questions? Problems? Suggestions? or email at

Please suggest features or report bugs with the GitHub issue tracker.

All documentation is copyright its authors; we didn't write any of that.