persist: Storage Functions

Description Usage Arguments Details Value Examples

Description

Functions to manage the storage of spark_tbl objects. persist is capable of various storage modes (see details), and cache is shorthand for "MEMORY_AND_DISK" storage level.

Usage

1
2
3
4
5
6
7
cache(.data)

persist(.data, newLevel)

unpersist(.data, blocking = TRUE)

storage_level(.data)

Arguments

.data

a spark_tbl

newLevel

storage level chosen for the persistence. See available options in the details.

blocking

boolean, whether to block until all blocks are deleted.

Details

For details of the supported storage levels, refer to rdd-persistence.

Value

a spark_tbl

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
## Not run: 
spark_session()
iris_tbl <- spark_tbl(iris)

storage_level(iris_tbl)

persist(iris_tbl, "MEMORY_AND_DISK")
storage_level(iris_tbl)

unpersist(iris_tbl)
storage_level(iris_tbl)

## End(Not run)

danzafar/tidyspark documentation built on Sept. 30, 2020, 12:19 p.m.