mem: Monitor Memory Use

In kuwisdelu/matter: Out-of-core statistical computing and signal processing

Monitor Memory Use

Description

These are utility functions for checking memory used by R and matter in the current session and/or during the timed execution of an expression.

Usage

mem(x, reset = FALSE)

memcl(cl = bpparam(), reset = FALSE)

memtime(expr, verbose = NA, BPPARAM = NULL)

Arguments

x	An object, to summarize how much memory it is using.
reset	Should the maximum memory values be reset?
expr	An expression to be evaluated.
verbose	Should timing messages be printed?
cl, BPPARAM	A cluster or a SnowParam object with a cluster backend. Used to collect memory usage from the cluster during timing.

Details

These functions summarize the memory used by both traditional R objects and out-of-memory matter objects. "Real" memory managed by R is summarized using gc. "Virtual" memory managed by matter includes shared memory allocated by matter and temporary files created by matter in getOption("matter.temp.dir").

For timing parallel code, it is useful to use memtime in combination with its BPPARAM argument to monitor the amount of memory used by the cluster.

Value

For mem called with an x argument, a named vector summarizing the memory and storage used by the object. The named elements include:

• "real": The amount of R memory used.

• "shared": The amount of shared memory used.

• "virtual": The amount of virtual memory used, including both file storage and shared memory.

For mem called without an x argument, a named vector summarizing the memory and storage used by the current R session. The named elements include:

• "real": The amount of R memory used.

• "shared": The amount of shared memory used.

• "max real": The maximum amount of R memory used since the last reset.

• "max shared": The maximum amount of shared memory used since the last reset.

• "temp": The total size of temporary files managed by matter in getOption("matter.temp.dir").

For memcl, a data frame with columns corresponding to the elements described above and rows corresponding to cluster nodes.

For memtime, a list including:

• "start": Memory use at the start of timing.

• "end": Memory use at the end of timing.

• "cluser": (Optional.) A summary of the memory used by the cluster if BPPARAM is specified.

• "overhead": The amount of "real" memory used during the execution of expr that is freed by the end of timing.

• "change": The difference in "real" memory used before and after timing.

• "total": For the current R session, the sum of "max real" and "max shared" memory used and total cluster memory used. For a cluster, the sum of "max real" memory used by all workers (not including shared memory or the managing R process).

• "time": The execution time.

Author(s)

Kylie A. Bemis

See Also

gc

Examples

x <- 1:100

mem(x)

memtime(mean(x + 1))

kuwisdelu/matter documentation built on Oct. 19, 2024, 10:31 a.m.