Description Usage Arguments Details Value Author(s) See Also Examples
Farm out a procedure (an arbitrary block of R-code) to a cluster of slaves, with different values of the variables.
1 2 3 4 5 6 |
proc |
An arbitrary block of R code that will be executed for each element in |
joblist |
A list of the jobs that will be farmed out.
Each element of |
common |
An optional list of variables common to all the jobs. |
status |
An optional integer vector giving the status of the individual jobs in |
chunk |
optional integer; the chunk size to be used.
By default, |
stop.condition |
After each execution, this expression will be evaluated in the context of the return-value of |
info |
If |
checkpoint |
optional integer specifying the granularity of checkpointing.
That is, the checkpoint file will be saved once every |
checkpoint.file |
optional filename.
If |
max.backup |
positive integer; maximum number of backup checkpoint files that will be created. |
sleep |
amount of time (in seconds) that the master will sleep between consecutive polls of the slave pool. |
blocking |
If TRUE, blocking MPI calls will be used. If FALSE non-blocking MPI calls are used for greater evenness and efficiency in the scheduling. |
verbose |
logical; if |
fn |
A arbitrary block of R code, deparsed to a string. |
A higher-level code is mpi.farmer
, which calls mpi.farm
and handles checkpointing.
However, it may sometimes happen that a user wants to call mpi.farm
directly.
mpi.farm.slave
does the work on the slaves and should never need to be called by the user.
mpi.farm
will execute the code in proc
repeatedly in environments defined by the entries of joblist
and the common environment common
.
If Rmpi slaves have been spawned, the jobs will be farmed out to them according to a load-balancing algorithm;
if no slaves are running, or Rmpi has not been loaded, the jobs will be executed serially.
For many of the applications envisioned, the jobs the slaves are assigned involve stochastic simulations.
Because of the way that R initializes its pseudorandom number generators (RNGs), it is easy to make the mistake of failing to initialize the RNGs on different slaves to different states.
If one fails to do this (and doesn't use a sophisticated parallel RNG like SPRNG) then it is possible that the random numbers generated on different slaves will be correlated or even identical.
For this reason, it is a good idea to set the seed of the RNG as part of the block of code proc
.
Storing the state of the RNG before doing so is often desirable, but this can be frustrating if the RNG has not been initialized. mpi.farm
checks to see if .Random.seed
exists and, if it does not, initializes the RNG with a call to runif
. Thus, the user is guaranteed that the RNG has been initialized on each slave.
A user interrupt to mpi.farm
results in an attempt to terminate the slaves cleanly.
This may take some time, since each slave has to finish the job it is currently working on before it becomes receptive to messages from the master.
A user interrupt issued during the abort process will leave some finished jobs in the MPI queue and therefore compromise the integrity of future parallel computations.
For this reason, when it is necessary to abort mpi.farm
and not possible to allow it to terminate cleanly, it is recommended that the slaves be closed (via mpi.close.Rslaves
) and restarted before further parallel computations are attempted.
The value returned by mpi.farm
and farm
is a list with one entry for each of the elements in joblist
.
The elements in the result correspond to the elements in the joblist
argument.
Aaron A. King
mpi.farmer
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | ## Not run:
mpi.spawn.Rslaves(needlog=F)
x <- lapply(1:100,function(k)list(a=k,b=rnorm(1)))
y <- mpi.farm(a+b,x)
print(unlist(y))
x <- lapply(1:100,function(k)list(a=k,b=0,done=0))
y <- mpi.farm({b <- b+rnorm(1); list(a=a,b=b,done=done+1)},x,stop.condition=((abs(b)>2)|(done>10)))
mpi.close.Rslaves()
## End(Not run)
## run some jobs in serial mode
x <- lapply(1:100,function(k)list(a=k,b=0,done=0))
y <- mpi.farm({b <- b+rnorm(1); list(a=a,b=b,done=done+1)},x,stop.condition=((abs(b)>2)|(done>10)))
|
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.