By default, simpleCache
creates caches in the R session you use to call it. If you need to make lots of caches, or very large caches, you may want instead to sub these as jobs to a cluster resource manager (like SLURM). simpleCache can do this using functionality from the batchtools
package.
This vignette is unevaluated because it relies on the batchtools
package and a cluster environment.
To do this, first, create a batchtools
registry. You can follow more detailed documentation in the batchtools
package, but here's some code to get you started:
library(simpleCache) setCacheDir(tempdir()) registry = batchtools::makeRegistry(NA) templateFile = system.file("templates/slurm-advanced.tmpl", package = "simpleCache") registry$cluster.functions = batchtools::makeClusterFunctionsSlurm( template = templateFile) registry
Notice that I'm using a custom slurm template here. With a registry in hand, we next need to define the resources this cache job will require:
resources = list(ncpus=1, memory=1000, walltime=60, partition="serial")
Then, we simply add these as arguments to simpleCache()
like so:
simpleCache("testBatch", { rnorm(1e7, 0, 1) }, batchRegistry=registry, batchResources=resources)
This will now create and submit a job script to the cluster. That job script will have R code to create your testBatch
cache by calling the code in your simpleCache
call, rnorm(1e7, 0, 1)
. Next time you run this function, it will just load the cache without recreating it, as you would expect simpleCache to do. Now there's a bunch of other stuff you can use batchtools
to do with these jobs:
batchtools::getJobTable(reg=registry) batchtools::getJobPars() batchtools::getStatus() batchtools::getJobTable(reg=registry) batchtools::getJobPars(1, reg=registry) batchtools::loadResult(1, reg=registry) # batchtools::testJob(1, reg=registry) # killJobs()
When you're done, you may want to remove your temporary registry:
batchtools::removeRegistry(reg=registry)
See batchtools
documentation for more details on using registries.
Any scripts or data that you put into this service are public.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.