gather: Gather Distributed Data
In DSL: Distributed Storage and List

Description Usage Arguments Details Value Examples

Retrieves "DList" data distributed as chunks.

1	DGather( x, keys = FALSE, n = -1L, names = TRUE )

`x`	a `"DList"` object.
`keys`	logical; should only keys be retrieved from chunks? Default: `FALSE`, i.e., only values are retrieved.
`n`	an integer specifying the number of chunks to be read.
`names`	logical; should the return value be a named list? Default: `TRUE`.

DGather() is similar to an MPI_GATHER (see http://www.mpi-forum.org/docs/mpi-3.1/mpi31-report/node103.htm#Node103) where: “[...] each process (root process included) sends the contents of its send buffer to the root process. The root process receives the messages and stores them in rank order.” For "DList" objects DGather() will gather data contained in chunks possibly distributed on a cluster of workstations and store it in a (possibly named) list. Note that depending of the size of the data, the resulting list may not fit into memory.

A (named) list.

dl <- DList( line1 = "This is the first line.",
             line2 = "Now, the second line." )
DGather( dl )
## retrieve keys
unlist(DGather( dl, keys = TRUE, names = FALSE ))
## remove DList and garbage collect it
rm( dl )
gc()

$`DSL-20181211-141430-epqwwrwytx/chunk-2d5440b0a8c5`
$`DSL-20181211-141430-epqwwrwytx/chunk-2d5440b0a8c5`[[1]]
[1] "This is the first line."

$`DSL-20181211-141430-epqwwrwytx/chunk-2d5440b0a8c5`[[2]]
[1] "Now, the second line."


[1] "line1" "line2"
         used (Mb) gc trigger (Mb) max used (Mb)
Ncells 319672 17.1     592000 31.7   398644 21.3
Vcells 593506  4.6    1208034  9.3  1021245  7.8