multiplyr: Data Manipulation with Parallelism and Shared Memory Matrices

Provides a new form of data frame backed by shared memory matrices and a way to manipulate them. Upon creation these data frames are shared across multiple local nodes to allow for simple parallel processing.

AuthorJim Blundell [aut, cre, cph]
Date of publication2016-05-31 23:24:54
MaintainerJim Blundell <jeblundell@doctors.org.uk>
LicenseGPL-3
Version0.1.1
http://github.com/jeblundell/multiplyr/

View on CRAN

Man pages

add_rownames: Add a new column with row names

arrange: Sort data

between: Tests whether elements of a vector lie between two values...

bm_morder: Extension of bigmemory::morder to allow decreasing parameter...

bm_mpermute: Extension of bigmemory::mpermute to allow decreasing...

cumall: Cumulative all

cumany: Cumulative any

cummean: Cumulative mean

define: Define new columns

desc: Arrange specified column in descending order

distinct: Select unique rows or unique combinations of variables

distribute: Calculations for how to distribute x items over N nodes

dotscapture: Capture ... for later evaluation

dotscombine: Combine explicit and implicit dots

dotseval: Evaluate previously captured dots

dotsname: Ensure captured dots are all named

dotsname1: Name an expression (called by dotsname)

filter: Filter data

first: Returns first value in vector

group_by: Group data

group_sizes: Return size of groups

lag: Offset x backwards by n

last: Returns last value in vector

lead: Offset x forwards by n

multiplyr: Data Manipulation with Parellelism and Shared Memory Matrices

Multiplyr-class: Parallel processing data frame

Multiplyr-methods: Data access methods for Multiplyr

mutate: Change values of existing variables (and create new ones)

n: Number of items in current group

NA_class_: Returns NA of a particular class

n_distinct: Return the number of unique values

n_groups: Return number of groups

nonunique: Returns values of x that are non-unique

nsa: No strings attached mode

nth: Return the nth item from a vector

p: Concatenate (internal)

partition_even: Partition data evenly amongst cluster nodes

partition_group: Partition data so that each group is wholly on a node

reduce: Summarise data (with local reduction)

regroup: Return to grouped data

rename: Rename variables

select: Retain only specified variables

shutdown: Shutdown running cluster

slice: Select rows by position

sm_desc_comp: Returns big.matrix descriptor offset by 1 (for row by row...

sm_desc_group: Returns a big.matrix descriptor for a particular group ID

sm_desc_subset: Returns big.matrix descriptor limited to particular start/end...

sm_desc_update: Update description of a big.matrix after a row subset...

summarise: Summarise data

test_transition: Test for grouping transition (internal)

transmute: Change variables and drop all others

undefine: Delete variables

ungroup: Return data to non-grouped

within_group: Execute code within a group

within_node: Execute code within a node

Files in this package

multiplyr
multiplyr/inst
multiplyr/inst/doc
multiplyr/inst/doc/basics.Rmd
multiplyr/inst/doc/basics.html
multiplyr/inst/doc/basics.R
multiplyr/tests
multiplyr/tests/testthat.R
multiplyr/tests/testthat
multiplyr/tests/testthat/test-internal.R
multiplyr/tests/testthat/test-utils.R
multiplyr/tests/testthat/test-mutate.R
multiplyr/tests/testthat/test-summarise.R
multiplyr/tests/testthat/test-subset.R
multiplyr/tests/testthat/test-filter.R
multiplyr/tests/testthat/test-compact.R
multiplyr/tests/testthat/test-groups.R
multiplyr/tests/testthat/test-bm.R
multiplyr/tests/testthat/test-partition.R
multiplyr/tests/testthat/test-arrange.R
multiplyr/tests/testthat/test-nsa.R
multiplyr/tests/testthat/test-class.R
multiplyr/tests/testthat/test-within.R
multiplyr/NAMESPACE
multiplyr/NEWS.md
multiplyr/R
multiplyr/R/multiplyr.R multiplyr/R/internal.R multiplyr/R/nse.R multiplyr/R/ops.R multiplyr/R/util.R multiplyr/R/class.R
multiplyr/vignettes
multiplyr/vignettes/basics.Rmd
multiplyr/README.md
multiplyr/MD5
multiplyr/build
multiplyr/build/vignette.rds
multiplyr/DESCRIPTION
multiplyr/man
multiplyr/man/NA_class_.Rd multiplyr/man/group_sizes.Rd multiplyr/man/group_by.Rd multiplyr/man/n.Rd multiplyr/man/cumany.Rd multiplyr/man/sm_desc_comp.Rd multiplyr/man/within_node.Rd multiplyr/man/last.Rd multiplyr/man/transmute.Rd multiplyr/man/n_groups.Rd multiplyr/man/dotscapture.Rd multiplyr/man/nsa.Rd multiplyr/man/first.Rd multiplyr/man/mutate.Rd multiplyr/man/distinct.Rd multiplyr/man/partition_group.Rd multiplyr/man/sm_desc_update.Rd multiplyr/man/cummean.Rd multiplyr/man/dotsname.Rd multiplyr/man/desc.Rd multiplyr/man/partition_even.Rd multiplyr/man/bm_mpermute.Rd multiplyr/man/between.Rd multiplyr/man/p.Rd multiplyr/man/lead.Rd multiplyr/man/n_distinct.Rd multiplyr/man/multiplyr.Rd multiplyr/man/add_rownames.Rd multiplyr/man/regroup.Rd multiplyr/man/arrange.Rd multiplyr/man/rename.Rd multiplyr/man/cumall.Rd multiplyr/man/within_group.Rd multiplyr/man/Multiplyr-class.Rd multiplyr/man/nth.Rd multiplyr/man/dotscombine.Rd multiplyr/man/test_transition.Rd multiplyr/man/sm_desc_group.Rd multiplyr/man/shutdown.Rd multiplyr/man/dotsname1.Rd multiplyr/man/filter.Rd multiplyr/man/define.Rd multiplyr/man/bm_morder.Rd multiplyr/man/undefine.Rd multiplyr/man/ungroup.Rd multiplyr/man/sm_desc_subset.Rd multiplyr/man/slice.Rd multiplyr/man/Multiplyr-methods.Rd multiplyr/man/reduce.Rd multiplyr/man/dotseval.Rd multiplyr/man/distribute.Rd multiplyr/man/nonunique.Rd multiplyr/man/lag.Rd multiplyr/man/select.Rd multiplyr/man/summarise.Rd

Questions? Problems? Suggestions? or email at ian@mutexlabs.com.

All documentation is copyright its authors; we didn't write any of that.