registerDoSpark: Register a Parallel Backend

View source: R/do_spark.R

registerDoSparkR Documentation

Register a Parallel Backend

Description

Registers a parallel backend using the foreach package.

Usage

registerDoSpark(spark_conn, parallelism = NULL, ...)

Arguments

spark_conn

Spark connection to use

parallelism

Level of parallelism to use for task execution (if unspecified, then it will take the value of 'SparkContext.defaultParallelism()' which by default is the number of cores available to the 'sparklyr' application)

...

additional options for sparklyr parallel backend (currently only the only valid option is 'nocompile')

Value

None

Examples

## Not run: 

sc <- spark_connect(master = "local")
registerDoSpark(sc, nocompile = FALSE)

## End(Not run)


sparklyr documentation built on May 29, 2024, 2:58 a.m.