create_task_makefile: Create a .yml makefile for a multi-task job

Description Usage Arguments Value Examples

View source: R/create_task_makefile.R

Description

Create a .yml makefile (for use with remake or scipiper) for a set of tasks that together form a single job. The default target will be named after makefile (specifically, indicatorfile=tools::file_path_sans_ext(basename(makefile))) and can be evoked from another remake file as make(I('indicatorfile'),remake_file='thismakefile.yml') after replacing indicatorfile and thismakefile.yml with their values.

Usage

1
2
3
4
5
6
create_task_makefile(task_plan, makefile, include = c(),
  packages = c(), sources = c(), file_extensions = c("ind"),
  ind_complete = NA, ind_dir = attr(task_plan, "ind_dir"),
  ind_ext = getOption("scipiper.ind_ext"),
  template_file = system.file("extdata/task_makefile.mustache", package =
  "scipiper"))

Arguments

task_plan

a task plan as produced by create_task_plan()

makefile

character name of the remake file to create

include

character vector of any remake .yml files to include within this one. If any files must be quoted in the remake file, quote them with inner single quotes, e.g. c("unquoted", "'quoted file name.tsv'")

packages

character vector of any packages to load before running steps

sources

character vector of any files that should be sourced before running steps. If any files must be quoted in the remake file, quote them with inner single quotes, e.g. c("unquoted", "'quoted file name.tsv'")

file_extensions

character vector of file extensions to be added to the defaults at remake::file_extensions(). Inclusion of 'ind' is recommended because this indicator file extension is commonly used by scipiper.

ind_dir

directory path specifing the location where an overall job indicator file should be written once all tasks and steps are complete. This file will always be named after the makefile, but with the indicator file extension (ind_ext) instead of '.yml' as a suffix

ind_ext

the indicator file extension to use in creating the job indicator file

template_file

character name of the mustache template to render into makefile. The default is recommended

Value

the file name of the makefile that was created (can be displayed with cat(readLines(makefile), sep="\n")).

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
task_config <- data.frame(
  id=c('AZ','CO','CA'),
  capital=c('Phoeniz','Denver','Sacramento')
)
step1 <- create_task_step(
  step_name = 'prep',
  target = function(task_name, step_name, ...) {
    sprintf('%s_%s', task_name, step_name)
  },
  depends = c('A','B'),
  command = "process(target_name, I('C'))"
)
step2 <- create_task_step(
  step_name = 'plot',
  command = function(target_name, task_name, ...) {
    capital <- task_config[task_config$id == task_name, 'capital']
    sprintf('visualize(\'%s\', \'%s\')', task_name, capital)
  }
)
step3 <- create_task_step('report')
task_plan <- create_task_plan(c('AZ','CA','CO'), list(step1, step2, step3),
  final_steps='report', ind_dir='states/log')
task_makefile <- create_task_makefile(
  task_plan, makefile=file.path(tempdir(), 'states.yml'),
  file_extensions=c('ind'), packages='mda.streams')
cat(readLines(task_makefile), sep='\n')

USGS-R/scipiper documentation built on Aug. 14, 2018, 5:23 a.m.