q.dispatcher: job queue exector

In okux/qrmarkdown: Job Queue Management For Rmarkdown

Description

start job handler process in a background. n to specifiy how many executor you want.

# Go To R Studio Terminal # Start deamon Rscript -e "qrmarkdown::q.dispatcher(n=8, wdir='your.queue.log.dir')" Rscript -e "qrmarkdown::q.monitor(wdir='your.queue.log.dir')"

Job dispatcher takes a task from job request queue. No limit of executor, as long as you have enough multiple cpus & RAM. This works very well on ASW, remote IPC cluster environment.

Usage

  q.setwd('testlog')

  ii = q.push(code='ls()')

  q.dispatcher(wait = FALSE)

  q.wait(ii)

  q.shutdown()

Arguments

n	speicify n number of executor(s). Default is all avialable cores
wdir	default is q.wd()
wait	FALSE: default. Run in background. TRUE
timer	-1: never expire. or seconds to shutdown dispatcher (i.e. 4 seconds)
qprint	print job queue list if TRUE.

Author(s)

cokugami

Examples

## Not run: 
# Example show you how to create 2 different job queue
# for 2 application.
# Demo: we use timer=4 so dispatcher will expire to end session.
# For production, use timer=-1, won't expire until q.shutdown called.


  q.setwd('testlog/app1')
  for(i in 1:2)
    ii = q.push(code='ls()')
  q.dispatcher(n=5, timer = 4) # dispatcher for app1

  q.setwd('testlog/app2')
  for(i in 1:3)
    ii = q.push(code='ls()')
  q.dispatcher(n=5, timer = 4) # dispatcher for app2

  Sys.sleep(10)

  q.wd('testlog/app1')
  res1 <- q.show()

  q.wd('testlog/app2')
  res2 <- q.show()

  Total <- res1$outbox + res2$outbox
  expect(Total == 5, 'invalid job completion')


## End(Not run)

okux/qrmarkdown documentation built on Dec. 22, 2021, 4:17 a.m.