write_cache: write_cache

In skysyzygy/tessilake: Data lake for Tessitura

write_cache

Description

Function to write cached arrow files. Writes to the first defined storage in config::get("tessilake"), and then syncs to the other locations by calling sync_cache.

Usage

write_cache(
  x,
  table_name,
  type,
  depth = deprecated(),
  incremental = FALSE,
  sync = TRUE,
  ...
)

cache_write(
  x,
  table_name,
  depth,
  type,
  primary_keys = cache_get_attributes(x)$primary_keys,
  partition = !is.null(primary_keys),
  overwrite = FALSE,
  num_tries = 60,
  ...
)

Arguments

x	data.frame to be written
table_name	string
type	string, e.g. "tessi" or "stream"
depth	string, e.g. "deep" or "shallow", deprecated in write_cache
incremental	boolean, whether to call cache_update or cache_write to update the cached dataset.
sync	boolean, whether or not to sync the written cache to other storages
...	extra arguments passed on to arrow::write_feather, arrow::write_parquet or arrow::write_dataset
primary_keys	character vector of columns to be used as primary keys
partition	boolean or character, if TRUE, partition is derived from primary_keys; if character, partition identifies the column to use for partitioning
overwrite	boolean, whether or not to overwrite an existing cache
num_tries	integer number of times to try reading before failing

Value

invisible

Functions

• cache_write(): Underlying cache writer that invokes arrow::write_feather, arrow::write_parquet or arrow::write_dataset and handles partitioning

Examples

## Not run: 
x <- data.table(a = c(1, 2, 3))
write_cache(x, "test", "stream", primary_keys = c("a"))

## End(Not run)

skysyzygy/tessilake documentation built on June 14, 2025, 6:50 p.m.