update_closure: Create or update a function closure with new values

In clarkfitzg/sparklite: Lightweight frontend to run R code on an Apache Spark cluster

Description

Adds data to a function so that it can be serialized and run on a remote cluster, avoiding errors of type Object not found.

Usage

update_closure(fun, varlist, envir = .GlobalEnv)

Arguments

fun	function with old or no environment
varlist	character vector of names of objects to export.
envir	environment from which to export variables

Details

The updating behavior means that if fun already has an environment containing a = 10 then running a <- 20; updateClosure(fun, "a") will return a function where a = 20.

This function is modeled after clusterExport.

Value

function with updated environment

See Also

clusterExport

Examples

# To apply f you'll need to also get it's dependencies
f <- makeClosure(f, c("a", "helperfunc"))
clusterApply(sc, 1:10, f)

clarkfitzg/sparklite documentation built on May 13, 2019, 7:35 p.m.