multiplyr: Data Manipulation with Parallelism and Shared Memory Matrices

Provides a new form of data frame backed by shared memory matrices and a way to manipulate them. Upon creation these data frames are shared across multiple local nodes to allow for simple parallel processing.

Author
Jim Blundell [aut, cre, cph]
Date of publication
2016-05-31 23:24:54
Maintainer
Jim Blundell <jeblundell@doctors.org.uk>
License
GPL-3
Version
0.1.1
URLs

View on CRAN

Man pages

add_rownames
Add a new column with row names
arrange
Sort data
between
Tests whether elements of a vector lie between two values...
bm_morder
Extension of bigmemory::morder to allow decreasing parameter...
bm_mpermute
Extension of bigmemory::mpermute to allow decreasing...
cumall
Cumulative all
cumany
Cumulative any
cummean
Cumulative mean
define
Define new columns
desc
Arrange specified column in descending order
distinct
Select unique rows or unique combinations of variables
distribute
Calculations for how to distribute x items over N nodes
dotscapture
Capture ... for later evaluation
dotscombine
Combine explicit and implicit dots
dotseval
Evaluate previously captured dots
dotsname
Ensure captured dots are all named
dotsname1
Name an expression (called by dotsname)
filter
Filter data
first
Returns first value in vector
group_by
Group data
group_sizes
Return size of groups
lag
Offset x backwards by n
last
Returns last value in vector
lead
Offset x forwards by n
multiplyr
Data Manipulation with Parellelism and Shared Memory Matrices
Multiplyr-class
Parallel processing data frame
Multiplyr-methods
Data access methods for Multiplyr
mutate
Change values of existing variables (and create new ones)
n
Number of items in current group
NA_class_
Returns NA of a particular class
n_distinct
Return the number of unique values
n_groups
Return number of groups
nonunique
Returns values of x that are non-unique
nsa
No strings attached mode
nth
Return the nth item from a vector
p
Concatenate (internal)
partition_even
Partition data evenly amongst cluster nodes
partition_group
Partition data so that each group is wholly on a node
reduce
Summarise data (with local reduction)
regroup
Return to grouped data
rename
Rename variables
select
Retain only specified variables
shutdown
Shutdown running cluster
slice
Select rows by position
sm_desc_comp
Returns big.matrix descriptor offset by 1 (for row by row...
sm_desc_group
Returns a big.matrix descriptor for a particular group ID
sm_desc_subset
Returns big.matrix descriptor limited to particular start/end...
sm_desc_update
Update description of a big.matrix after a row subset...
summarise
Summarise data
test_transition
Test for grouping transition (internal)
transmute
Change variables and drop all others
undefine
Delete variables
ungroup
Return data to non-grouped
within_group
Execute code within a group
within_node
Execute code within a node

Files in this package

multiplyr
multiplyr/inst
multiplyr/inst/doc
multiplyr/inst/doc/basics.Rmd
multiplyr/inst/doc/basics.html
multiplyr/inst/doc/basics.R
multiplyr/tests
multiplyr/tests/testthat.R
multiplyr/tests/testthat
multiplyr/tests/testthat/test-internal.R
multiplyr/tests/testthat/test-utils.R
multiplyr/tests/testthat/test-mutate.R
multiplyr/tests/testthat/test-summarise.R
multiplyr/tests/testthat/test-subset.R
multiplyr/tests/testthat/test-filter.R
multiplyr/tests/testthat/test-compact.R
multiplyr/tests/testthat/test-groups.R
multiplyr/tests/testthat/test-bm.R
multiplyr/tests/testthat/test-partition.R
multiplyr/tests/testthat/test-arrange.R
multiplyr/tests/testthat/test-nsa.R
multiplyr/tests/testthat/test-class.R
multiplyr/tests/testthat/test-within.R
multiplyr/NAMESPACE
multiplyr/NEWS.md
multiplyr/R
multiplyr/R/multiplyr.R
multiplyr/R/internal.R
multiplyr/R/nse.R
multiplyr/R/ops.R
multiplyr/R/util.R
multiplyr/R/class.R
multiplyr/vignettes
multiplyr/vignettes/basics.Rmd
multiplyr/README.md
multiplyr/MD5
multiplyr/build
multiplyr/build/vignette.rds
multiplyr/DESCRIPTION
multiplyr/man
multiplyr/man/NA_class_.Rd
multiplyr/man/group_sizes.Rd
multiplyr/man/group_by.Rd
multiplyr/man/n.Rd
multiplyr/man/cumany.Rd
multiplyr/man/sm_desc_comp.Rd
multiplyr/man/within_node.Rd
multiplyr/man/last.Rd
multiplyr/man/transmute.Rd
multiplyr/man/n_groups.Rd
multiplyr/man/dotscapture.Rd
multiplyr/man/nsa.Rd
multiplyr/man/first.Rd
multiplyr/man/mutate.Rd
multiplyr/man/distinct.Rd
multiplyr/man/partition_group.Rd
multiplyr/man/sm_desc_update.Rd
multiplyr/man/cummean.Rd
multiplyr/man/dotsname.Rd
multiplyr/man/desc.Rd
multiplyr/man/partition_even.Rd
multiplyr/man/bm_mpermute.Rd
multiplyr/man/between.Rd
multiplyr/man/p.Rd
multiplyr/man/lead.Rd
multiplyr/man/n_distinct.Rd
multiplyr/man/multiplyr.Rd
multiplyr/man/add_rownames.Rd
multiplyr/man/regroup.Rd
multiplyr/man/arrange.Rd
multiplyr/man/rename.Rd
multiplyr/man/cumall.Rd
multiplyr/man/within_group.Rd
multiplyr/man/Multiplyr-class.Rd
multiplyr/man/nth.Rd
multiplyr/man/dotscombine.Rd
multiplyr/man/test_transition.Rd
multiplyr/man/sm_desc_group.Rd
multiplyr/man/shutdown.Rd
multiplyr/man/dotsname1.Rd
multiplyr/man/filter.Rd
multiplyr/man/define.Rd
multiplyr/man/bm_morder.Rd
multiplyr/man/undefine.Rd
multiplyr/man/ungroup.Rd
multiplyr/man/sm_desc_subset.Rd
multiplyr/man/slice.Rd
multiplyr/man/Multiplyr-methods.Rd
multiplyr/man/reduce.Rd
multiplyr/man/dotseval.Rd
multiplyr/man/distribute.Rd
multiplyr/man/nonunique.Rd
multiplyr/man/lag.Rd
multiplyr/man/select.Rd
multiplyr/man/summarise.Rd