restbatch-server: Server control functions

Description Usage Arguments Details Value Server Configuration See Also Examples

Description

Start, validate, stop a batch server

Usage

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
server_alive(
  port = default_port(),
  host = default_host(allow0 = FALSE),
  protocol = default_protocol(),
  path = "validate/ping",
  ...
)

kill_server(
  wait = TRUE,
  host = default_host(allow0 = FALSE),
  port = default_port(),
  protocol = default_protocol(),
  path = "validate/shutdown"
)

stop_server(
  wait = TRUE,
  host = default_host(allow0 = FALSE),
  port = default_port(),
  protocol = default_protocol(),
  path = "validate/shutdown"
)

start_server(
  host = default_host(),
  port = default_port(),
  settings = system.file("default_settings.yaml", package = "restbatch"),
  protocol = default_protocol(),
  path_validate = "validate/ping",
  make_default = TRUE,
  supervise = FALSE,
  ...
)

autoclose_server(
  host = default_host(),
  port = default_port(),
  auto_close = TRUE
)

ensure_server(
  host = default_host(),
  port = default_port(),
  protocol = default_protocol(),
  make_default = TRUE,
  supervise = FALSE,
  validate = TRUE,
  validate_sleep = 0.1,
  validate_maxwait = 30,
  timeout = 3,
  ...
)

Arguments

port

integer, which port the server runs at; default see default_port

host

an 'IPv4' address where to run the server on; default see default_host

protocol

'http' or 'https'; default see default_protocol. 'https' needs extra server settings

path, path_validate

internally used

...

pass to other methods

wait

wait until the server is shut down; default is true.

settings

path to a server configuration file; a sample can be obtained via conf_sample; see details

make_default

whether to make the server default? If so, all tasks will be sent to this server by default.

supervise

whether to shutdown the server when current R session expires; default is FALSE

auto_close

same as supervise

validate

whether to check server is alive once created, default is TRUE; see details

validate_sleep

if validation is on, intervals to check alive

validate_maxwait

maximum waiting time in seconds to validate

timeout

'http' request timeout, default is 3, see details.

Details

The function ensure_server is a combination of start_server and server_alive: it checks whether the target server is alive first before starting the server. If validate is true, then the it also wait until the server is completely up and ready to serve.

timeout is the seconds that a request should wait to check if the server is alive. On most UNIX systems, request fail immediately once the server is not available. However on Windows, requests might block the session if the server is unavailable. timeout is the upper time limit that would take to fail the test. If the server fails to respond before the limit, the test will also fail.

kill_server,stop_server stop the server as soon as possible. They send signals to the server to cancel the unfinished tasks and shutdown. autoclose_server stops the server when the current R session exits.

Value

start_server returns a list; server_alive returns whether the server can be connected; others return nothing.

Server Configuration

A server configuration is a file that contains key-value pairs. They are used to tune your running servers on performance, security, and even customize some actions.

A sample configuration file can be obtained via conf_sample

Some values are characters. These characters will be parsed through glue function that evaluates dynamically. The adds more flexibility to your settings. For example, the default startup_script evaluates system.file("scheduler/startup.R", package = "restbatch") dynamically, and pass the results as your actual startup script.

startup_script

path to the R script to run when starting up. For example, setting up pools, load extra settings etc.

batch_cluster

path to the R script defining cluster functions. See makeClusterFunctions on how to set up computational nodes.

task,validate

R plumber files to handle new job and validation requests

debug

yes or no to enable debug mode

require_auth

whether default authentication is on. The default authentication uses 'openssl' that sends tokens in the request headers.

anonymous_request

whether to allow default tokens; works for a quick set up scenarios like debugging, running on local machines, but will introduce security issues is deployed on public addresses.

modules_require_auth

which modules require default authentications; use comma to separate.

keep_alive

part of authentication system. When default authentications is on, each request header needs to include a timestamp and an encrypted token of that timestamp. The server will block the requests that are too old to avoid someone accidentally "steals" your previous tokens. The keep_alive defines the maximum time in seconds between the encrypted request time and the actual time when server handles requests

max_concurrent_tasks

maximum of running tasks allowed

max_concurrent_jobs

maximum of running jobs for each task

max_release_tasks_per_second

A loop will be created in the server to regularly check the task status (see task_queue_interval). The loop blocks the session. If there are multiple tasks finished, releasing them all might be time consuming, max_release_tasks_per_second controls the maximum number of tasks to be released each task_queue_interval seconds

task_queue_interval

intervals in seconds to check and update task status queued or running on the server. Recommended to be at least 0.5 to avoid database lock.

task_root

where to store the task files

max_nodetime

sometimes a task might run forever or get lost. It will still be in the running status. Set this number to indicate the max time in seconds a job should execute; default is 10 days.

func_newjob

function to handle new tasks

func_validate_server

function to validate the server

See Also

conf_sample, makeClusterFunctions, glue.

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
if(interactive()){

# -------------------- Start a server tasks -----------------------

# set default host, port to play with
default_host("127.0.0.1")
default_port(7033)

# start a server
start_server()

server_alive()

# alternatively, you can just call
ensure_server()

# -------------------- Run tasks -----------------------------------

task <- new_task2(function(x){
  if(x == 2){
    stop("I stop no because.")
  }
  Sys.sleep(5)
  Sys.getpid()
}, x = 1:3)

# Submit and run batch jobs
task$submit()

# Check
task$server_status()

# print task
print(task)

# obtain results
task$collect()

# get result details
attributes(task$collect())

# -------------------- An interactive shiny client -----------------

source(system.file('dashboards/client/app.R', package = 'restbatch'))

# -------------------- Stop the server ------------------------------
kill_server()

# clean up
task$remove()

}

dipterix/restbatch documentation built on Oct. 13, 2021, 8:58 a.m.