hdfsConn: Connect to Data Source on HDFS

Description Usage Arguments Details Value Author(s) See Also Examples

Description

Connect to a data source on HDFS

Usage

1
2
hdfsConn(loc, type = "sequence", autoYes = FALSE, reset = FALSE,
  verbose = TRUE)

Arguments

loc

location on HDFS for the data source

type

the type of data ("map", "sequence", "text")

autoYes

automatically answer "yes" to questions about creating a path on HDFS

reset

should existing metadata for this object be overwritten?

verbose

logical - print messages about what is being done

Details

This simply creates a "connection" to a directory on HDFS (which need not have data in it). To actually do things with this data, see ddo, etc.

Value

a "kvConnection" object of class "hdfsConn"

Author(s)

Ryan Hafen

See Also

addData, ddo, ddf, localDiskConn

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
## Not run: 
  # connect to empty HDFS directory
  conn <- hdfsConn("/test/irisSplit")
  # add some data
  addData(conn, list(list("1", iris[1:10,])))
  addData(conn, list(list("2", iris[11:110,])))
  addData(conn, list(list("3", iris[111:150,])))
  # represent it as a distributed data frame
  hdd <- ddf(conn)

## End(Not run)

datadr documentation built on May 1, 2019, 8:06 p.m.