runPipeline: runPipeline

Description Usage Arguments Value Examples

View source: R/runPipeline.R

Description

This function runs a pipeline with combinations of parameter variations on nested steps. The pipeline has to be defined as a list of functions applied consecutively on their respective outputs. See 'examples' for more details.

Usage

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
runPipeline(
  datasets,
  alternatives,
  pipelineDef,
  comb = NULL,
  output.prefix = "",
  nthreads = 1,
  saveEndResults = TRUE,
  debug = FALSE,
  skipErrors = TRUE,
  ...
)

Arguments

datasets

A named vector of initial objects or paths to rds files.

alternatives

The (named) list of alternative values for each parameter.

pipelineDef

An object of class PipelineDefinition.

comb

An optional matrix of indexes indicating the combination to run. Each column should correspond to an element of 'alternatives', and contain indexes relative to this element. If omitted, all combinations will be performed.

output.prefix

An optional prefix for the output files.

nthreads

Number of threads, default 1. If the memory requirements are very high or the first steps very long to compute, consider setting this as the number of datasets or below.

saveEndResults

Logical; whether to save the output of the last step.

debug

Logical (default FALSE). When enabled, disables multithreading and prints extra information.

skipErrors

Logical. When enabled, 'runPipeline' will continue even when an error has been encountered, and report the list of steps/datasets in which errors were encountered.

...

passed to MulticoreParam. Can for instance be used to set 'makeCluster' arguments, or set 'threshold="TRACE"' when debugging in a multithreaded context.

Value

A SimpleList with elapsed time and the results of the evaluation functions defined by the given 'pipelineDef'.

The results are also stored in the output folder with:

Examples

1
2
3
4
5
6
pip <- mockPipeline()
datasets <- list( ds1=1:3, ds2=c(5,10,15) )
tmpdir1 <- paste0(tempdir(),"/")
res <- runPipeline(datasets, pipelineDef=pip, output.prefix=tmpdir1,
                   alternatives=list() )
# See the `pipeComp_scRNA` vignette for a more complex example

plger/pipeComp documentation built on Nov. 2, 2021, 8:40 p.m.