runPipeline: runPipeline

In pipeComp: pipeComp pipeline benchmarking framework

Description

This function runs a pipeline with combinations of parameter variations on nested steps. The pipeline has to be defined as a list of functions applied consecutively on their respective outputs. See 'examples' for more details.

Usage

runPipeline(
  datasets,
  alternatives,
  pipelineDef,
  comb = NULL,
  output.prefix = "",
  nthreads = 1,
  saveEndResults = TRUE,
  debug = FALSE,
  skipErrors = TRUE,
  ...
)

Arguments

datasets	A named vector of initial objects or paths to rds files.
alternatives	The (named) list of alternative values for each parameter.
pipelineDef	An object of class PipelineDefinition.
comb	An optional matrix of indexes indicating the combination to run. Each column should correspond to an element of 'alternatives', and contain indexes relative to this element. If omitted, all combinations will be performed.
output.prefix	An optional prefix for the output files.
nthreads	Number of threads, default 1. If the memory requirements are very high or the first steps very long to compute, consider setting this as the number of datasets or below.
saveEndResults	Logical; whether to save the output of the last step.
debug	Logical (default FALSE). When enabled, disables multithreading and prints extra information.
skipErrors	Logical. When enabled, 'runPipeline' will continue even when an error has been encountered, and report the list of steps/datasets in which errors were encountered.
...	passed to MulticoreParam. Can for instance be used to set 'makeCluster' arguments, or set 'threshold="TRACE"' when debugging in a multithreaded context.

Value

A SimpleList with elapsed time and the results of the evaluation functions defined by the given 'pipelineDef'.

The results are also stored in the output folder with:

• The clustering results for each dataset ('endOutputs.rds' files),

• A SimpletList of elapsed time and evaluations for each dataset ('evaluation.rds' files),

• A list of the 'pipelineDef', 'alternatives', 'sessionInfo()' and function call used to produce the results ('runPipelineInfo.rds' file),

• A copy of the SimpleList returned by the function ('aggregated.rds'file).

Examples

pip <- mockPipeline()
datasets <- list( ds1=1:3, ds2=c(5,10,15) )
tmpdir1 <- paste0(tempdir(),"/")
res <- runPipeline(datasets, pipelineDef=pip, output.prefix=tmpdir1,
                   alternatives=list() )
# See the `pipeComp_scRNA` vignette for a more complex example

pipeComp documentation built on Nov. 8, 2020, 7:35 p.m.