copy2shm: Copy to and Allocate from POSIX Shared Memory

In gfkse/bettermc: Enhanced Fork-Based Parallelization

Copy to and Allocate from POSIX Shared Memory

Description

Copy the data of a vector to a POSIX shared memory object and allocate from such.

Usage

copy2shm(x, name, overwrite = FALSE, copy = TRUE)

allocate_from_shm(obj, copy = obj$copy)

Arguments

x	a logical, integer, double, complex or raw vector, an S3 object based hereon or a factor. Long vectors are supported.
name	the name of the shared memory object to create. A portable name starts with a "/", followed by one or more (up to 253) characters, none of which are slashes. Note: on macOS the total length of the name must not exceed 31 characters.
overwrite	should an already existing shared memory object with the given name be overwritten? If FALSE, the copy fails if such an object already exists. Note: Due to bugs in the macOS implementation of POSIX shared memory, (as of now) only FALSE is supported.
copy	should the vector placed in shared memory be used directly (FALSE) by allocate_from_shm or rather a copy of it (TRUE)? FALSE is apparently faster (initially), but might require more memory in the long run (up to double what is normally required by such a vector): if we modify elements of such a vector, new memory (pages) will be allocated to hold these changed values. The original memory (pages) of the shared memory object will only be freed when the vector is garbage collected. If we initially copy the whole vector from shared memory to "regular" one, the former can be freed directly and the latter can be modified in place. Note: The value passed to copy2shm has no direct effect. It only sets the default value for allocate_from_shm, which can safely be changed. Note 2: FALSE is silently ignored on macOS.
obj	an object as returned by copy2shm, which was typically called in another process.

Value

copy2shm returns an S3 object of class "shm_obj", which is a list with the following elements: (name) the name of the shared memory object as given, (type) an integer specifying the type of x, (length) the number of elements in x as a double, (size) the size of the shared memory object in bytes as a double, (attributes) the attributes of x as a shallow copy of the corresponding pairlist, (copy) the default value for the copy argument passed to allocate_from_shm. Note: this function will not produce an error if an operation related directly to the copy to shared memory fails. In this case a character vector of length 1 containing the error message will be returned.

allocate_from_shm returns a vector. Note: this function cannot be called more than once on any obj, since it unlinks the shared memory object immediately after trying to open it. If copy = TRUE, the vector will be allocated using a custom allocator, but this is not guaranteed. As of now, vectors with less than two elements are allocated using R's default allocator. This implementational detail must not be relied on. If copy = FALSE, the custom allocator privately maps the shared memory object into the address space of the current process. In particular this means that changes made to this memory region by subsequently forked child processes are private to them: neither the parent nor a sibling process will see these changes. This is most probably what we want and expect.

Windows Support

Not supported on Windows.

Lifecycle

Note

See also the general notes on POSIX shared memory under mclapply.

Examples

if (tolower(Sys.info()[["sysname"]]) != "windows") {
  x <- runif(100)
  obj <- copy2shm(x, "/random")
  if (inherits(obj, "shm_obj")) {
    # copy2shm succeeded
    y <- allocate_from_shm(obj)
    stopifnot(identical(x, y))
  } else {
    # copy2shm failed -> print the error message
    print(obj)
  }
}

gfkse/bettermc documentation built on April 23, 2023, 6:51 a.m.