spark_collect_data: Collect a Spark DataFrame into R

In nathaneastwood/sparklio: Utility Functions to Extend 'sparklyr'\'s IO

Description

Given the name of a Spark DataFrame or a reference to a Spark DataFrame, collect it into R from a spark_connection.

Usage

spark_collect_data(x, sc = NULL)

Arguments

x	One of: A tbl_spark object. A character(1) name of the table to retrieve passed with sc.
sc	A spark_connection, only required if x is passed as a character(1) (def: NULL).

Value

A tibble.

Examples

## Not run: 
sc <- sparklyr::spark_connect(master = "local")
mtcars_spark <- dplyr::copy_to(sc, mtcars)

# We can collect the data using the R objet
spark_collect_data(mtcars_spark)

# Or by referencing the name within the spark connection
spark_collect_data(x = "mtcars", sc = sc)

## End(Not run)

nathaneastwood/sparklio documentation built on March 16, 2021, 7:51 p.m.