job_loop: Build a bash script that loops over variables and submits SGE...

In LieberInstitute/sgejobs: Helper functions for SGE jobs at JHPCE

Build a bash script that loops over variables and submits SGE jobs

Description

This function builds a bash script that loops over a set of variables with pre-specified values to create an internal bash script that then gets submitted as a SGE job.

Usage

job_loop(
  loops,
  name,
  create_shell = FALSE,
  queue = "shared",
  memory = "10G",
  cores = 1L,
  email = "e",
  logdir = "logs",
  filesize = "100G",
  task_num = NULL,
  tc = 20
)

Arguments

loops	A named list where each of the elements are character vectors. The names of loops specify the variables used for the loops and the contents specify the options to loop through for each variable.
name	A character(1) vector with the name of the script. Any spaces will be replaced by underscores.
create_shell	A logical(1) vector specifying whether to create a shell file for the script.
queue	A character(1) vector with the name of the SGE queue. Check how busy a given queue is by running ⁠qpic -q queuename⁠.
memory	The amount of memory per core to request in SGE syntax. You can check how much a current job is utilizing using the qmem JHPCE command. For more detail on the memory options, check https://jhpce.jhu.edu/knowledge-base/how-to/#MemSpec.
cores	The number of cores to request. Note that the total memory your job will request is cores multiplied by memory.
email	The email reporting option for the email. For more information check https://jhpce.jhu.edu/knowledge-base/how-to/#Email.
logdir	The directory for the SGE log files relative to the current working directory.
filesize	The maximum file size in SGE format.
task_num	The number of tasks for your job, which will make it into an array job. If NULL this is ignored.
tc	If task_num is specified, this option controls the number of concurrent tasks.

Value

A character vector with the script contents. If create_shell was specified then it also creates the actual script in the current working directory.

Author(s)

Leonardo Collado-Torres

Examples

job_loop(
    loops = list(region = c("DLPFC", "HIPPO"), feature = c("gene", "exon", "tx", "jxn")),
    name = "bsp2_test"
)

job_loop(
    loops = list(region = c("DLPFC", "HIPPO"), feature = c("gene", "exon", "tx", "jxn")),
    cores = 5,
    task_num = 10,
    name = "bsp2_test_array"
)

LieberInstitute/sgejobs documentation built on May 12, 2023, 5:20 p.m.