addData: Add Key-Value Pairs to a Data Connection

In datadr: Divide and Recombine for Large, Complex Data

Description

Add key-value pairs to a data connection

Usage

addData(conn, data, overwrite = FALSE)

Arguments

conn	a kvConnection object
data	a list of key-value pairs (list of lists where each sub-list has two elements, the key and the value)
overwrite	if data with the same key is already present in the data, should it be overwritten? (does not work for HDFS connections)

Note

This is generally not recommended for HDFS as it writes a new file each time it is called, and can result in more individual files than Hadoop likes to deal with.

Author(s)

Ryan Hafen

See Also

removeData, localDiskConn, hdfsConn

Examples

## Not run: 
  # connect to empty HDFS directory
  conn <- hdfsConn("/test/irisSplit")
  # add some data
  addData(conn, list(list("1", iris[1:10,])))
  addData(conn, list(list("2", iris[11:110,])))
  addData(conn, list(list("3", iris[111:150,])))
  # represent it as a distributed data frame
  hdd <- ddf(conn)

## End(Not run)

datadr documentation built on May 1, 2019, 8:06 p.m.