nodes_to_plan: Make a plan, given a set of nodes

In burchill/cs: Functions for working with Rochester's CS cluster

Make a plan, given a set of nodes

Description

This is relatively complicated, because I made it needlessly flexible. Basically, it will either execute future::plan in a topology that logs into the login node, has a cluster set up with the nodes specified where on each of these nodes, it uses a multiprocess. The needlessly flexible bit is where I let the user specify the number of cores used for each cluster node either by a custom function or a list corresponding to the list of cluster hostnames (which can be numerics or functions).

Usage

nodes_to_plan(nodes, login_node, core_mapper = NULL, goahead = TRUE,
  use_abbreviations = TRUE)

Arguments

login_node	a string for the gateway login, e.g., "[e-mail]" (not real)
core_mapper	if NULL, the multiprocess plan uses its default arguments. If a function, core_mapper determines the number of cores used for each machine. If a list/vector, it will be assumed to be the number of cores for each particular node, in the same order as the nodes were given. nodes_to_plan will then convert this into a function automatically.
goahead	indicates whether this function should execute the plan or just return an expression representing the plan
use_abbreviations	because the way ssh adds known hosts, if you've only sshed into a node via ssh nodeN, you'll want to set this to TRUE
df	a vector/list of node hostnames or a data frame with a column called "nodes" that has the same information

Details

The bloat in this code comes from my desire to show the user what command might be run. Unfortunately, it's too much of a pain in the butt to give users the proper amount of information about the functions they're calling while at the same time letting them customize those functions. Therefore, I represent all the arbitrary custom core-setting functions with 'function()'.

Value

Either nothing or an expression representing the structure of the plan.

burchill/cs documentation built on May 28, 2023, 1:29 p.m.