create_task_plan: Define a set of tasks and steps within a single job

Description Usage Arguments Value Examples

View source: R/create_task_plan.R

Description

Create a structured list that organizes tasks and their sub-tasks (steps) for a large number of near-identical tasks

Usage

1
2
3
create_task_plan(task_names, task_steps, final_steps = sapply(task_steps,
  `[[`, "step_name"), ind_dir = NULL, add_complete = TRUE,
  ind_ext = getOption("scipiper.ind_ext"))

Arguments

task_names

character vector of IDs for sets of work coordinates, with length equal to the number of tasks. You will usually use these character strings within the names for each step in each task. These names should therefore describe the aspect/aspects of a task that position it within the larger parameter space of work to be done. Examples: site IDs, model names, or character code identifying a unique row in a configuration file or data.frame.

task_steps

list of definitions steps to perform within each task. Each step definition should be created by create_task_step()

final_steps

vector of step indices (either step_names or integers) identifying those task_steps that must be completed to consider the entire task complete. If one step depends on all previous steps (as indicated by its remake depends: and command: fields) then only that one step need be included in final_steps.

ind_dir

directory path, only used within steps if add_complete==TRUE, specifying the location where task-specific indicator files should be written. Such files will always be named after their associated tasks, with '.ind' as a suffix. This path information is also passed on to create_task_makefile() so that you don't need to specify it in both places.

add_complete

logical. if TRUE, ind_dir must be supplied and a final step named 'complete' will be added to the step list. This step will depend on all final_steps and will write an indicator file with a timestamp when all final steps are complete.

ind_ext

the indicator file extension to use in creating indicator files for any complete steps (only used if add_complete==TRUE)

Value

a structured list that can be passed to create_task_makefile or create_task_table

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
task_config <- data.frame(
  id=c('AZ','CO','CA'),
  capital=c('Phoenix','Denver','Sacramento')
)
step1 <- create_task_step(
  step_name = 'prep',
  target_name = function(task_name, step_name, ...) {
    sprintf('%s_%s', task_name, step_name)
  },
  depends = c('A','B'),
  command = "process(target_name, I('C'))"
)
step2 <- create_task_step(
  step_name = 'plot',
  command = function(target_name, task_name, ...) {
    capital <- task_config[task_config$id == task_name, 'capital']
    sprintf('visualize(\'%s\', \'%s\')', task_name, capital)
  }
)
step3 <- create_task_step('report')
task_plan <- create_task_plan(c('AZ','CA','CO'), list(step1, step2, step3),
  final_steps='report', ind_dir='states/log')

USGS-R/scipiper documentation built on Aug. 14, 2018, 5:23 a.m.