Flow: Flow

FlowR Documentation

Flow

Description

Instantiates a Flow object which is a purely diagnostic / visualization utility to monitor progress across a set of entities x tasks.

Note: For job monitoring, it does not do anything fancy - just checks the paths pointed to by the columns of an entities table against task inputs and outputs to determine whether a task is ready or done.

However if some other task (or manual setting) of that column points to a valid path, then that pipeline will be marked as "done".

All job control should be done at the Job level. TBD: linking this object to a set of Flow object and allowing some job control at the Flow level.

Returns Flow object which then can be plotted or subsetted via [.Flow operator.

Usage

Flow(
  flowdir = "./Flow",
  taskdir = NULL,
  entities = NULL,
  update = FALSE,
  quick = FALSE,
  mc.cores = 1,
  paths.only = TRUE
)

Arguments

flowdir

path to Flow directory containing .rds of Job files, and/or paths to those .rds files

taskdir

path to task directory or set of task paths

entities

keyed entities data.table can be optionally provided with taskdir

update

logical flag whether to update jobs if flowdir provided (FALSE)

quick

won't check existence of file paths, just will check to see if non empty

mc.cores

integer multicore param

paths.only

whether to build the DAG only using path annotations rather than all arguments (otherwise too crowded)

Value

Flow object

Author(s)

Marcin Imielinski


mskilab/Flow documentation built on Jan. 12, 2023, 8:31 a.m.