shards: Shard Descriptor Creation

In shard: Deterministic, Zero-Copy Parallel Execution for R

Shard Descriptor Creation

Description

Create shard descriptors for parallel execution with autotuning.

Produces shard descriptors (index ranges) for use with shard_map(). Supports autotuning based on worker count and memory constraints.

Usage

shards(
  n,
  block_size = "auto",
  workers = NULL,
  strategy = c("contiguous", "strided"),
  min_shards_per_worker = 4L,
  max_shards_per_worker = 64L,
  scratch_bytes_per_item = 0,
  scratch_budget = 0
)

Arguments

n	Integer. Total number of items to shard.
block_size	Block size specification. Can be: "auto" (default): Autotune based on worker count Integer: Explicit number of items per shard Character: Human-readable like "1K", "10K"
workers	Integer. Number of workers for autotuning (default: pool size or detectCores - 1).
strategy	Sharding strategy: "contiguous" (default) or "strided".
min_shards_per_worker	Integer. Minimum shards per worker for load balancing (default 4).
max_shards_per_worker	Integer. Maximum shards per worker to limit overhead (default 64).
scratch_bytes_per_item	Numeric. Expected scratch memory per item for memory budgeting.
scratch_budget	Character or numeric. Total scratch memory budget (e.g., "1GB").

Value

A shard_descriptor object containing:

• n: Total items

• block_size: Computed block size

• strategy: Strategy used

• shards: List of shard descriptors with id, start, end, idx fields

Examples

blocks <- shards(1e6, workers = 8)
length(blocks$shards)

blocks <- shards(1000, block_size = 100)

blocks$shards[[1]]$idx

shard documentation built on April 3, 2026, 9:08 a.m.