registerDoSpark: Register a Parallel Backend

Description Usage Arguments Value Examples

View source: R/do_spark.R

Description

Registers a parallel backend using the foreach package.

Usage

1
registerDoSpark(spark_conn, parallelism = NULL, ...)

Arguments

spark_conn

Spark connection to use

parallelism

Level of parallelism to use for task execution (if unspecified, then it will take the value of 'SparkContext.defaultParallelism()' which by default is the number of cores available to the 'sparklyr' application)

...

additional options for sparklyr parallel backend (currently only the only valid option is nocompile = T, F)

Value

None

Examples

1
2
3
4
5
6
## Not run: 

sc <- spark_connect(master = "local")
registerDoSpark(sc, nocompile = FALSE)

## End(Not run)

sparklyr documentation built on Nov. 30, 2021, 5:06 p.m.