04 Advanced `SpaDES` use

In SpaDES.core: Core Utilities for Developing and Running Spatially Explicit Discrete Event Models

Advanced SpaDES use

This vignette is still a work in progress.

Memory monitoring

While profvis::profvis is an essential tool for memory monitoring using deep R internals, it is often not sufficient for a discrete event situation. For example, it may be useful to know the peak memory use of an event, as this may be the limiting step for setting up many parallel instances. There is an experimental tool that gets triggered with options("spades.memoryUseInterval" = xxx) where xxx is a numeric in seconds, e.g., 0.2. If this is set, and future and future.callr are installed, then whenever a spades call is made, the memory use will be assessed at that regular interval. The procedure is: 1. spawn a future session (i.e., a parallel session) that runs system('ps') which lists all processes. It only keeps the process that represents the process ID of the main R session; 2. that ps call writes to a text file every getOption('spades.memoryUseInterval'); 3. if you ran this with a spades call, setting options("spades.memoryUseInterval" = 0.5) or some other interval (in seconds), it will read that text file into the simList at the end (on.exit) of the spades call (doing this triggers a file deletion of the text file); 4. the object is then in sim$.memoryUse$obj.

At that point, the function memoryUse can be called on the simList and it will do a join on the sim$.memoryUse$obj with the completed(sim) by time stamp, so each event shows its memory use.

if (requireNamespace("future", quietly = TRUE) &&
  requireNamespace("future.callr", quietly = TRUE)) {
  options("spades.memoryUseInterval" = 0.5)

  # run your simInit and spades calls here
  # sim <- simInit()
  # sim <- spades(sim)  

  memoryUse(sim, max = TRUE) # this should show peak memory use by eventType -- i.e., summarizes if multiple times
  memoryUse(sim, max = FALSE) # this should show peak memory use by event
}

References

SpaDES.core documentation built on Nov. 10, 2023, 5:08 p.m.