starburst_session: Create a Detached Starburst Session
In starburst: Seamless AWS Cloud Bursting for Parallel R Workloads

starburst_session

R Documentation

Create a Detached Starburst Session

Description

Creates a new detached session that can run computations independently of your R session. You can close R and reattach later to collect results.

Usage

starburst_session(
  workers = 10,
  cpu = 4,
  memory = "8GB",
  region = NULL,
  timeout = 3600,
  session_timeout = 3600,
  absolute_timeout = 86400,
  launch_type = "EC2",
  instance_type = "c7g.xlarge",
  use_spot = TRUE,
  warm_pool_timeout = 3600
)

Arguments

`workers`	Number of parallel workers (default: 10)
`cpu`	vCPUs per worker (default: 4)
`memory`	Memory per worker, e.g., "8GB" (default: "8GB")
`region`	AWS region (default: from config or "us-east-1")
`timeout`	Task timeout in seconds (default: 3600)
`session_timeout`	Active timeout in seconds (default: 3600)
`absolute_timeout`	Maximum session lifetime in seconds (default: 86400)
`launch_type`	"EC2" or "FARGATE" (default: "EC2")
`instance_type`	EC2 instance type for EC2 launch (default: "c7g.xlarge")
`use_spot`	Use spot instances for EC2 (default: TRUE)
`warm_pool_timeout`	EC2 warm pool timeout in seconds (default: 3600)

Value

A StarburstSession object (also carrying $session_id, the handle you pass to starburst_session_attach) with methods:

submit(expr, globals = NULL, packages = NULL): Submit one task (a quoted expression). Returns the task id. Call repeatedly to fan out work.
status(): Return a progress summary (counts of pending / running / completed / failed tasks). Safe to call from a fresh R session after reattaching.
collect(wait = FALSE): Retrieve results, keyed by task id, in submission order. With wait = FALSE returns whatever has finished so far; wait = TRUE blocks until every submitted task is terminal. Every terminal task appears: a success carries its return value; a failed task carries a structured failure list(error = TRUE, message = ..., value = NULL, task_id = ...) so failures are visible rather than silently dropped.
extend(seconds = 3600): Extend the active/absolute timeout of a still-running session.
cleanup(stop_workers = TRUE, force = FALSE): Stop the session's workers and mark it terminated. By default S3 objects are preserved (so you can still inspect/collect); pass force = TRUE to also delete the session's S3 task/result objects. Otherwise the session self-terminates at absolute_timeout.

Lifecycle

starburst_session() launches workers immediately and returns a handle. Submit tasks, then either poll status()/collect() in the same session, or record session$session_id, close R, and later starburst_session_attach(session_id) to reconnect and collect. A session ends when you call cleanup(), when session_timeout elapses with no activity, or at absolute_timeout — whichever comes first.

Failure behavior

A failed task is recorded (surfaced via status()) and does not abort the others; collect() returns it as a structured failure entry (error = TRUE with a message) alongside the successful results, rather than dropping it or raising. If the client dies, workers keep running against S3 until a timeout, which is what makes reattaching possible. cleanup() is the only thing that frees resources early — sessions do not auto-clean on garbage collection.

Examples


if (starburst_is_configured()) {
  # Create detached session
  session <- starburst_session(workers = 10)

  # Submit tasks
  task_ids <- lapply(1:100, function(i) {
    session$submit(quote(expensive_computation(i)))
  })

  # Close R and come back later...
  session_id <- session$session_id

  # Reattach
  session <- starburst_session_attach(session_id)

  # Collect results
  results <- session$collect(wait = TRUE)
}

starburst documentation built on July 23, 2026, 1:07 a.m.