create_task_makefile: Create a .yml makefile for a multi-task job

In USGS-R/scipiper: Support functions for ushering data through a scientific workflow

Create a .yml makefile for a multi-task job

Description

Create a .yml makefile (for use with remake or scipiper) for a set of tasks that together form a single job. The default target will be named after makefile (specifically, indicatorfile=tools::file_path_sans_ext(basename(makefile))) and can be evoked from another remake file as make(I('indicatorfile'),remake_file='thismakefile.yml') after replacing indicatorfile and thismakefile.yml with their values.

Usage

create_task_makefile(
  task_plan,
  makefile,
  include = c(),
  packages = "scipiper",
  sources = c(),
  file_extensions = c("ind"),
  template_file = system.file("extdata/task_makefile.mustache", package = "scipiper"),
  final_targets,
  finalize_funs = "combine_to_ind",
  as_promises = TRUE,
  tickquote_combinee_objects = TRUE
)

Arguments

task_plan	a task plan as produced by create_task_plan()
makefile	character name of the remake file to create
include	character vector of any remake .yml files to include within this one. If any files must be quoted in the remake file, quote them with inner single quotes, e.g. c("unquoted", "'quoted file name.tsv'")
packages	character vector of any packages to load before running steps
sources	character vector of any files that should be sourced before running steps. If any files must be quoted in the remake file, quote them with inner single quotes, e.g. c("unquoted", "'quoted file name.tsv'")
file_extensions	character vector of file extensions to be added to the defaults at remake::file_extensions(). Inclusion of 'ind' is recommended because this indicator file extension is commonly used by scipiper.
template_file	character name of the mustache template to render into makefile. The default is recommended
final_targets	a file path or object name (or vector of either) specifying final target(s) to be created by finalize_funs
finalize_funs	a string function name (or vector of function names, of equal length to final_targets). Using NULL for finalize_funs will turn the final steps from the task plan into depends for the default/all target.
as_promises	hides the actual final_targets file/object from remake, and uses a dummy target name instead, with suffix "_promise". This allows us to avoid cyclic dependencies. Naming convention for ⁠_promise⁠ variables is to drop any dir structure from the final_targets
tickquote_combinee_objects	TRUE by default

Value

the file name of the makefile that was created, or the string output (if makefile = NULL)

Examples

task_config <- data.frame(
  id=c('AZ','CO','CA'),
  capital=c('Phoeniz','Denver','Sacramento')
)
step1 <- create_task_step(
  step_name = 'prep',
  target = function(task_name, step_name, ...) {
    sprintf('%s_%s', task_name, step_name)
  },
  depends = c('A','B'),
  command = "process(target_name, I('C'))"
)
step2 <- create_task_step(
  step_name = 'plot',
  command = function(target_name, task_name, ...) {
    capital <- task_config[task_config$id == task_name, 'capital']
    sprintf('visualize(\'%s\', \'%s\')', task_name, capital)
  }
)
step3 <- create_task_step('report')
task_plan <- create_task_plan(c('AZ','CA','CO'), list(step1, step2, step3),
  final_steps='report', ind_dir='states/log')
task_makefile <- create_task_makefile(
  task_plan, makefile=file.path(tempdir(), 'states.yml'),
  file_extensions=c('ind'), packages='mda.streams')
cat(readLines(task_makefile), sep='\n')

USGS-R/scipiper documentation built on May 25, 2023, 8:47 a.m.