write_file: Write a 'spark_tbl' to an arbitrary file format

In danzafar/tidyspark: A Tidy Interface to Spark

Description

functions used to write Spark tables to file. These use the backend spark_write_source to write the actual file. Note, spark_write_source is not meant to write files to the the hive metastore, see spark_write_table for functionality similar to Spark's saveAsTable and insertInto.

Usage

spark_write_source(
  .data,
  path,
  source = NULL,
  mode = "error",
  partition_by = NULL,
  ...
)

Arguments

.data	a spark_tbl
path	string, the path where the file is to be saved.
source	string, can be file types like parquet or csv.
mode	string, usually "error" (default), "overwrite", "append", or "ignore"
partition_by	string, column names to partition by on disk
...	any other option to be passed. Must be a named argument.

danzafar/tidyspark documentation built on Sept. 30, 2020, 12:19 p.m.