startMPIcluster: Create and start an MPI cluster
In doMPI: Foreach Parallel Adaptor for the Rmpi Package

Description Usage Arguments Note Examples

View source: R/mpiworkers.R

The startMPIcluster function is used to start an MPI cluster.

startMPIcluster(count, verbose=FALSE, workdir=getwd(), logdir=workdir,
                maxcores=1, includemaster=TRUE, bcast=TRUE,
                comm=if (mpi.comm.size(0) > 1) 0 else 3,
                intercomm=comm + 1, mtag=10, wtag=11,
                defaultopts=list())

`count`	Number of workers to spawn. If you start your script using mpirun, then you don't really need to use the `count` argument, because `startMPIcluster` will try to do something reasonable. To be more specific, if `comm` is `0`, then it will set `count` to `mpi.comm.size(0) - 1`. In fact, it is an error to set `count` to any other value. If `comm` is greater than `0`, it will determine the number of processes to spawn by calling `mpi.universe.size()`. If that value is greater than one, then `count` is set to one less. If that value is equal to one, then `count` is arbitrarily set to `2`. Note that if you've started the script without mpirun, than `mpi.universe.size()` will always return `1`, so `count` will default to `2`.
`verbose`	Indicates if verbose messages should be enabled. Defaults to `FALSE`.
`workdir`	Working directory of the cluster workers. Defaults to the master's working directory.
`logdir`	Directory to put the worker log files. Defaults to `workdir`.
`maxcores`	Maximum number of cores for workers to use. Defaults to `1`.
`includemaster`	Indicates if the master process should be counted as a load on the CPU. This will effect how many cores will be used on the local machine by mclapply, if a worker process is started on the local machine. Defaults to `TRUE`.
`bcast`	Indicates if a true MPI broadcast should be used to send shared “job” data to the workers. If `FALSE` is specified, the data is sent by separate messages to each worker, which is sometimes faster than using a broadcast. So this option really controls whether to do a real or an emulated broadcast. Defaults to `TRUE`.
`comm`	Communicator number to use. A value of 0 means to use non-spawn mode, which means the cluster workers are started using mpirun/ortrun with more than one worker. A value of 1 or more forces spawn mode. Multiple clusters can be started by using different values for `comm` and `intercomm`. It defaults to `0` if `mpi.comm.size(0) > 1`, otherwise `3`.
`intercomm`	Inter-communicator number. Defaults to `comm + 1`.
`mtag`	Tag to use for messages sent to the master. Do not use this option unless you know what you're doing, or your program will very likely hang. Defaults to `10`.
`wtag`	Tag to use for messages sent to the workers. Do not use this option unless you know what you're doing, or your program will very likely hang. Defaults to `11`.
`defaultopts`	A list containing default values to use for some of the `.options.mpi` options. These options include: `chunkSize`, `info`, `profile`, `bcastThreshold`, `forcePiggyback`, `nocompile`, and `seed`.

The startMPIcluster function will return an MPI cluster object of different classes, depending on the bcast option. This is because broadcasting is implemented as a method on the MPI cluster object, and that method is implemented differently in the different classes.

Also note that the bcast option has no effect if the backend-specific forcePiggyback option is used with foreach, since “piggy-backing” is an alternative way to send the job data to the workers in separate messages.

So there are currently three ways that the job data can be sent to the workers: piggy-backed with the first task to each worker, broadcast, or sent in separate messages. Which method is best will presumably depend on your hardware and your MPI implementation.

## Not run: 
# start and register an MPI cluster with two workers in verbose mode:
cl <- startMPIcluster(count=2, verbose=TRUE)
registerDoMPI(cl)
# and shut it down
closeCluster(cl)

# set the working directory to /tmp:
cl <- startMPIcluster(count=2, workdir='/tmp')
registerDoMPI(cl)
# and shut it down
closeCluster(cl)

## End(Not run)