cache_table: Cache And Uncache Tables

Description Usage Arguments Value See Also Examples

View source: R/tables.R

Description

Spark SQL can cache tables using an in-memory columnar format by calling cache_table(). Spark SQL will scan only required columns and will automatically tune compression to minimize memory usage and GC pressure. You can call uncache_table() to remove the table from memory. Similarly you can call clear_cache() to remove all cached tables from the in-memory cache. Finally, use is_cached() to test whether or not a table is cached.

Usage

1
2
3
4
5
6
7

Arguments

sc

A spark_connection.

table

character(1). The name of the table.

Value

See Also

create_table(), get_table(), list_tables(), refresh_table(), table_exists(), uncache_table()

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
## Not run: 
sc <- sparklyr::spark_connect(master = "local")
mtcars_spark <- sparklyr::copy_to(dest = sc, df = mtcars)

# By default the table is not cached
is_cached(sc = sc, table = "mtcars")

# We can manually cache the table
cache_table(sc = sc, table = "mtcars")
# And now the table is cached
is_cached(sc = sc, table = "mtcars")

# We can uncache the table
uncache_table(sc = sc, table = "mtcars")
is_cached(sc = sc, table = "mtcars")

## End(Not run)

catalog documentation built on March 20, 2021, 5:06 p.m.