dlt_readwrite: Write and read SparkDataFrame in delta format

In zero323/dlt: SparkR Delta Lake Interface

Write and read SparkDataFrame in delta format

Description

Write and read SparkDataFrame in delta format

Usage

dlt_write(df, path, ...)

write.delta(df, path, ...)

dlt_read(path, ...)

read.delta(path, ...)

## S4 method for signature 'SparkDataFrame,character'
dlt_write(df, path, ...)

## S4 method for signature 'SparkDataFrame,character'
write.delta(df, path, ...)

Arguments

df	SparkDataFrame
path	character path to write the data.
...	additional arguments passed to writer

Functions

• dlt_write(): Generic

• write.delta(): Generic

• dlt_read(): Load data stored as delta into SparkDataFrame

• read.delta(): Load data stored as delta into SparkDataFrame (alias of dlt_read)

• dlt_write(df = SparkDataFrame, path = character): Write SparkDataFrame' in delta format

• write.delta(df = SparkDataFrame, path = character): Write SparkDataFrame in delta format (alias of dlt_write)

Note

dlt_read since 1.0.0

read.delta since 1.0.0

dlt_write since 1.0.0

write.delta since 1.0.0

Examples

## Not run: 
set.seed(323)
path <- tempfile()

df <- data.frame(
  id = 1:12,
  key = rep(c("a", "b", "c"), each = 4),
  value = rnorm(12)
) %>%
  createDataFrame()

dlt_write(df, path)

dlt_read(path) %>%
  schema()

write.delta(df, path, mode = "overwrite")

read.delta(path) %>%
  schema()

## End(Not run)

zero323/dlt documentation built on Dec. 1, 2022, 11:25 p.m.