Description Usage Arguments Details Value Author(s) References See Also Examples

Divide a ddo/ddf object into subsets based on different criteria

1 2 3 4 |

`data` |
an object of class "ddf" or "ddo" - in the latter case, need to specify |

`by` |
specification of how to divide the data - conditional (factor-level or shingles), random replicate, or near-exact replicate (to come) – see details |

`spill` |
integer telling the division method how many lines of data should be collected until spilling over into a new key-value pair |

`filterFn` |
a function that is applied to each candidate output key-value pair to determine whether it should be (if returns |

`bsvFn` |
a function to be applied to each subset that returns a list of between subset variables (BSVs) |

`output` |
a "kvConnection" object indicating where the output data should reside (see |

`overwrite` |
logical; should existing output location be overwritten? (also can specify |

`preTransFn` |
a transformation function (if desired) to applied to each subset prior to division - note: this is deprecated - instead use |

`postTransFn` |
a transformation function (if desired) to apply to each post-division subset |

`params` |
a named list of objects external to the input data that are needed in the distributed computing (most should be taken care of automatically such that this is rarely necessary to specify) |

`packages` |
a vector of R package names that contain functions used in |

`control` |
parameters specifying how the backend should handle things (most-likely parameters to |

`update` |
should a MapReduce job be run to obtain additional attributes for the result data prior to returning? |

`verbose` |
logical - print messages about what is being done |

The division methods this function will support include conditioning variable division for factors (implemented – see `condDiv`

), conditioning variable division for numerical variables through shingles, random replicate (implemented – see `rrDiv`

), and near-exact replicate. If `by`

is a vector of variable names, the data will be divided by these variables. Alternatively, this can be specified by e.g. `condDiv(c("var1", "var2"))`

.

an object of class "ddf" if the resulting subsets are data frames. Otherwise, an object of class "ddo".

Ryan Hafen

`recombine`

, `ddo`

, `ddf`

, `condDiv`

, `rrDiv`

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | ```
# divide iris data by Species by passing in a data frame
bySpecies <- divide(iris, by = "Species")
bySpecies
# divide iris data into random partitioning of ~30 rows per subset
irisRR <- divide(iris, by = rrDiv(30))
irisRR
# any ddf can be passed into divide:
irisRR2 <- divide(bySpecies, by = rrDiv(30))
irisRR2
bySpecies2 <- divide(irisRR2, by = "Species")
bySpecies2
# splitting on multiple columns
byEdSex <- divide(adult, by = c("education", "sex"))
byEdSex
byEdSex[[1]]
# splitting on a numeric variable
bySL <- ddf(iris) %>%
addTransform(function(x) {
x$slCut <- cut(x$Sepal.Length, 10)
x
}) %>%
divide(by = "slCut")
bySL
bySL[[1]]
``` |

datadr documentation built on May 30, 2017, 5:46 a.m.

Embedding an R snippet on your website

Add the following code to your website.

For more information on customizing the embed code, read Embedding Snippets.