to_flow: Create flow objects
In sahilseth/flowr: Streamlining Design and Deployment of Complex Workflows

Description Usage Arguments Details Value See Also Examples

Use a set of shell commands (flow mat) and flow definition to create flow object.

to_flow(x, ...)

is.flow(x)

## S3 method for class 'character'
to_flow(x, def, grp_col, jobname_col, cmd_col, ...)

## S3 method for class 'flowmat'
to_flow(
  x,
  def,
  flowname,
  grp_col,
  jobname_col,
  cmd_col,
  submit = FALSE,
  execute = FALSE,
  containerize = TRUE,
  platform,
  flow_run_path,
  qobj,
  verbose = opts_flow$get("verbose"),
  ...
)

## S3 method for class 'data.frame'
to_flow(x, ...)

## S3 method for class 'list'
to_flow(
  x,
  def,
  flowname,
  flow_run_path,
  desc,
  qobj,
  module_cmds = opts_flow$get("module_cmds"),
  verbose = opts_flow$get("verbose"),
  ...
)

`x`	this can either to a filename, a data.frame or a list. In case it is a file name, it should be a tsv file representing a flow_mat. See to_flowmat for details
`...`	Supplied to specific functions like to_flow.data.frame
`def`	a flow definition. Basically a table with resource requirements and mapping of the jobs in this flow. See to_flowdef for details on the format.
`grp_col`	name of the grouping column in the supplied flow_mat. See to_flow for details. Default value is [samplename].
`jobname_col`	name of the job name column in flow_mat. Default value is [jobname].
`cmd_col`	name of the command column name in flow_mat. Default value is [cmd].
`flowname`	name of the flow, this is used as part of the execution foldername. A good simple identifier, which does not support any special characters. Names may use characters (a-z) and numbers (0-9), using underscore (_) as a word separator. Default value is [flowname].
`submit`	after creating a flow object, should flowr also use submit_flow to perform a dry-run OR real submission. See below for details. Default value is [FALSE]
`execute`	when calling submit_flow, should flowr execute the flow or perform a dry-run. See below for details. Default value is [FALSE].
`containerize`	if the flowmat has multiple samples, flowr creates a creates a new date-stamped folder, and includes all flows in this batch inside it. This is keeps the logs clean, and containerizes each batch. To disable this behavior set this to FALSE, default is [TRUE].
`platform`	a specifying the platform to use, possible values are local, lsf, torque, moab, sge and slurm This over-rides the platform column in the flowdef. (optional)
`flow_run_path`	base path to be used for execution of this flow. flowr would create a new time-stamped folder in this base path and use it for logs, scripts etc. The default is retrieved using `opts_flow$get("flow_run_path")`.
`qobj`	Depreciated, modify cluster templates as explained on flow-r.github.io/flowr. An object of class queue.
`verbose`	A numeric value indicating the amount of messages to produce. Values are integers varying from 0, 1, 2, 3, .... Please refer to the verbose page for more details. `opts_flow$get("verbose")`
`desc`	Advanced Use. final flow name.
`module_cmds`	A character vector of additional commands, which will be prepended to each script of the flow. Default is retrieved using `opts_flow$get("module_cmds")`.

The parameter x can be a path to a flow_mat, or a data.frame (as read by read_sheet). This is a minimum three column table with columns: samplename, jobname and cmd. See to_flowmat for details.

Returns a flow object. If execute=TRUE, fobj is rich with information about where and how the flow was executed. It would include details like jobids, path to exact scripts run etc. To use kill_flow, to kill all the jobs one would need a rich flow object, with job ids present.

Behaviour: What goes in, and what to expect in return?

submit=FALSE & execute=FALSE: Create and return a flow object
submit=TRUE & execute=FALSE: dry-run, Create a flow object then, create a structured execution folder with all the commands
submit=TRUE, execute=TRUE: Do all of the above and then, submit to cluster

to_flowmat, to_flowdef, to_flowdet, flowopts and submit_flow

## Use this link for a few elaborate examples:
## http://flow-r.github.io/flowr/flowr/tutorial.html#define_modules

ex = file.path(system.file(package = "flowr"), "pipelines")
flowmat = as.flowmat(file.path(ex, "sleep_pipe.tsv"))
flowdef = as.flowdef(file.path(ex, "sleep_pipe.def"))
fobj = to_flow(x = flowmat, def = flowdef, flowname = "sleep_pipe", platform = "lsf")


## create a vector of shell commands
cmds = c("sleep 1", "sleep 2")
## create a named list
lst = list("sleep" = cmds)
## create a flowmat
flowmat = to_flowmat(lst, samplename = "samp")

## Use flowmat to create a skeleton flowdef
flowdef = to_flowdef(flowmat)

## use both (flowmat and flowdef) to create a flow
fobj = to_flow(flowmat, flowdef)

## submit the flow to the cluster (execute=TRUE) or do a dry-run (execute=FALSE)
## Not run: 
fobj2 = submit_flow(fobj, execute=FALSE)
fobj3 = submit_flow(fobj, execute=TRUE)

## Get the status or kill all the jobs
status(fobj3)
kill(fobj3)

## End(Not run)