registerDoRedis: Register the Redis back end for foreach.

Description Usage Arguments Details Value Note See Also Examples

View source: R/doRedis.R

Description

The doRedis package imlpements a simple but flexible parallel back end for foreach that uses Redis for inter-process communication. The work queue name specifies the base name of a small set of Redis keys that the master and worker processes use to exchange data.

Usage

1
registerDoRedis(queue, host = "localhost", port = 6379, password, ...)

Arguments

queue

A work queue name

host

The Redis server host name or IP address

port

The Redis server port number

password

An optional Redis database password

...

Optional arguments passed to redisConnect

Details

Back-end worker R processes advertise their availablility for work with the redisWorker function.

The doRedis parallel back end tolerates faults among the worker processes and automatically resubmits failed tasks. It is also portable and supports heterogeneous sets of workers, even across operative systems. The back end supports dynamic pools of worker processes. New workers may be added to work queues at any time and can be used by running foreach computations.

Value

NULL is invisibly returned.

Note

All doRedis functions require access to a Redis database server (not included with this package).

The doRedis package sets RNG streams across the worker processes using the L'Ecuyer-CMRG method from R's parallel package for reproducible pseudorandom numbers independent of the number of workers or task distribution. See the package vignette for more details and additional options.

Avoid using fork-based parallel functions within doRedis expressions. Use of mclapply and similar functions in the body of a doRedis foreach loop can result in worker faults.

See Also

foreach, doRedis-package, setChunkSize, removeQueue

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
## Not run: 
## The example assumes that a Redis server is running on the local host
## and standard port.

## 1. Open one or more 'worker' R sessions and run:
require('doRedis')
redisWorker('jobs')

## 2. Open another R session acting as a 'master' and run this simple 
##    sampling approximation of pi:
require('doRedis')
registerDoRedis('jobs')
foreach(j=1:10, .combine=sum, .multicombine=TRUE) \%dopar\%
        4 * sum((runif(1000000) ^ 2 + runif(1000000) ^ 2) < 1) / 10000000
removeQueue('jobs')

## End(Not run)

bwlewis/doRedis documentation built on Nov. 21, 2018, 1:16 p.m.