spark_write_json: Write a 'spark_tbl' to JSON format

In danzafar/tidyspark: A Tidy Interface to Spark

Description

Write a spark_tbl to JSON

Usage

spark_write_json(.data, path, mode = "error", partition_by = NULL, ...)

Arguments

.data	a spark_tbl
path	string, the path where the file is to be saved.
mode	string, usually "error" (default), "overwrite", "append", or "ignore"
partition_by	string, column names to partition by on disk
...	any other named options. See details below.

Details

For JSON, additional options can be specified using ...: #'

compression

(default null), compression codec to use when saving to file. This can be one of the known case-insensitive shorten names (none, bzip2, gzip, lz4, snappy and deflate).

dateFormat

(default yyyy-MM-dd), sets the string that indicates a date format. Custom date formats follow the formats at java.text.SimpleDateFormat. This applies to date type.

timestampFormat

(default yyyy-MM-dd'T'HH:mm:ss.SSSXXX), sets the string that indicates a timestamp format. Custom date formats follow the formats at java.text.SimpleDateFormat. This applies to timestamp type.

encoding

(by default it is not set), specifies encoding (charset) of saved json files. If it is not set, the UTF-8 charset will be used.

lineSep

(default \n), defines the line separator that should be used for writing.

More information can be found here: https://spark.apache.org/docs/latest/api/java/org/apache/spark/sql/DataFrameWriter.html#json-java.lang.String-

danzafar/tidyspark documentation built on Sept. 30, 2020, 12:19 p.m.