spark_lapply: Apply a Function over a List or Vector, Distribute operations...

In danzafar/tidyspark: A Tidy Interface to Spark

Description

Run a function over a list of elements, distributing the computations with Spark. Applies a function in a manner that is similar to doParallel or lapply to elements of a list. The computations are distributed using Spark. It is conceptually the same as the following code: lapply(list, func)

Usage

spark_lapply(.l, .f)

Arguments

.l	a vector (atomic or list)
.f	he function to be applied to each element of .l

Value

an in-memory list object

Examples

## Not run: 
spark_session()
doubled <- spark_lapply(1:10, function(x) {2 * x})

# or using tidyverse style lamdas
doubled <- spark_lapply(1:10, ~ 2 * .)

## End(Not run)

danzafar/tidyspark documentation built on Sept. 30, 2020, 12:19 p.m.