run: Run automated Pipelines

In flowr: Streamlining Design and Deployment of Complex Workflows

Description

Run complete pipelines, by wrapping several steps into one convenient function.

NOTE: please use flowr version 0.9.8.9010 or higher.

In summary, this function performs the following steps:

• the argument x defines the name of the pipeline. Say, for example sleep_pipe.

• fetch_pipes: finds the pipeline definition (sleep_pipe.R, sleep_pipe.def and sleep_pipe.conf files)

• sleep_pipe(...): Create all the required commands (flowmat)

• to_flow: Use flowmat and sleep_pipe.def to create a flow object.

• submit_flow: Submit the flow to the cluster.

Usage

run(
  x,
  platform,
  def,
  conf,
  wd = opts_flow$get("flow_run_path"),
  flow_run_path = wd,
  rerun_wd,
  start_from,
  execute = FALSE,
  ...
)

run_pipe(
  x,
  platform,
  def,
  conf,
  wd = opts_flow$get("flow_run_path"),
  flow_run_path = wd,
  rerun_wd,
  start_from,
  execute = FALSE,
  ...
)

Arguments

x	name of the pipeline to run. This is a function called to create a flow_mat.
platform	what platform to use, overrides flowdef
def	flow definition
conf	a tab-delimited configuration file with path to tools and default parameters. See fetch_pipes.
wd	an alias to flow_run_path
flow_run_path	passed onto to_flow. Default it picked up from flowr.conf. Typically this is ~/flowr/runs
rerun_wd	if you need to re-run, supply the previous working dir
start_from	the step to start a rerun from. Intuitively, this is ignored in a fresh run and only used in re-running a pipeline.
execute	TRUE/FALSE
...	passed onto the pipeline function as specified in x

Examples

## Not run: 

## Run a short pipeline (dry run)
run("sleep_pipe")

## Run a short pipeline on the local machine
run("sleep_pipe", platform = "local", execute = TRUE)

## Run a short pipeline on the a torque cluster (qsub)
run("sleep_pipe", platform = "torque", execute = TRUE)

## Run a short pipeline on the a MOAB cluster (msub)
run("sleep_pipe", platform = "moab", execute = TRUE)

## Run a short pipeline on the a IBM (LSF) cluster (bsub)
run("sleep_pipe", platform = "lsf", execute = TRUE)

## Run a short pipeline on the a MOAB cluster (msub)
run("sleep_pipe", platform = "moab", execute = TRUE)

## change parameters of the pipeline
## All extra parameters are passed on to the function function.
run("sleep_pipe", platform = "lsf", execute = TRUE, x = 5)


## End(Not run)

flowr documentation built on March 3, 2021, 1:12 a.m.