zmq.lapply: apply a function to a list.

In armstrtw/deathstar: distributed lappy and other tools

Description

zmq.lapply returns a list of the same length as X, each element of which is the result of applying FUN to the corresponding element of X.

The execution of the function is performed in parallel over zmq sockets.

Usage

zmq.cluster.lapply(cluster,X,FUN,...,exec.port=6000L,status.port=6001L,sleep.interval=0.01)

Arguments

cluster	a vector of computer names to use for the parallel lapply
X	a list or vector.
FUN	the function to be applied to each element of X
exec.port	the port to use to send jobs/receive results.
status.port	the port to use to check node capacity.
sleep.interval	the amount of time to sleep (in seconds) before checking the capacity of the compute nodes.
...	additional arguments to lapply.

Details

Successful "parallelization" of lapply requires a deathstar daemon to be running on the remote nodes.

deathstar is packaged for debian and debian derivatives here: https://github.com/downloads/armstrtw/deathstar.core/deathstar_0.0.1-1_amd64.deb

Value

a list of the same length as X

Author(s)

ZMQ was written by Martin Sustrik <[e-mail]> and Martin Lucina <[e-mail]>. rzmq was written by Whit Armstrong. segue and estimatePi were written by JD Long.

References

http://www.zeromq.org http://api.zeromq.org http://zguide.zeromq.org/page:all http://zguide.zeromq.org/page:all#Divide-and-Conquer https://gist.github.com/406824 http://www.vcasmo.com/video/drewconway/8468 http://www.r-bloggers.com/abusing-amazon http://code.google.com/p/segue

See Also

connect.socket,bind.socket,receive.socket,send.socket

Examples

## Not run: 
library(rzmq)

## run locally
## estimatePi example courtesy of JD Long / Jeffrey Breen

library(AWS.tools)
library(rzmq)

cl <- startCluster(ami="ami-4bc90522",key="kls-ec2",instance.count=5,instance.type="c1.xlarge")

estimatePi <- function(seed) {
    set.seed(seed)
    numDraws <- 1e6

    r <- .5
    x <- runif(numDraws, min=-r, max=r)
    y <- runif(numDraws, min=-r, max=r)
    inCircle <- ifelse( (x^2 + y^2)^.5 < r , 1, 0)
    
    sum(inCircle) / length(inCircle) * 4
}

run.time <- system.time(ans <- zmq.cluster.lapply(cluster=cl$instances[,"PublicDNS"],as.list(1:1000),estimatePi))

print(mean(unlist(ans)))
print(run.time)
print(attr(ans,"execution.report"))

res <- terminateCluster(cl)

pi.est <- mean(unlist(ans))
print(pi.est)


## End(Not run)

armstrtw/deathstar documentation built on May 10, 2019, 1:42 p.m.