makeExtractable: Take a ddo/ddf HDFS data object and turn it into a mapfile

In datadr: Divide and Recombine for Large, Complex Data

Description

Take a ddo/ddf HDFS data object and turn it into a mapfile

Usage

makeExtractable(obj, control = NULL)

Arguments

obj	object of class 'ddo' or 'ddf' with an HDFS connection
control	parameters specifying how the backend should handle things (most-likely parameters to rhwatch in RHIPE) - see rhipeControl and localDiskControl

Examples

## Not run: 
  conn <- hdfsConn("/test/irisSplit")
  # add some data
  addData(conn, list(list("1", iris[1:10,])))
  addData(conn, list(list("2", iris[11:110,])))
  addData(conn, list(list("3", iris[111:150,])))
  # represent it as a distributed data frame
  hdd <- ddf(conn)
  # try to extract values by key (this will result in an error)
  # (HDFS can only lookup key-value pairs by key if data is in a mapfile)
  hdd[["3"]]
  # convert hdd into a mapfile
  hdd <- makeExtractable(hdd)
  # try again
  hdd[["3"]]

## End(Not run)

datadr documentation built on May 1, 2019, 8:06 p.m.