HDFS: HDFS (Hadoop Distributed File System) connections

In s-u/hdfsc: HDFS-based connections for R and HDFS-related file operations (Hadoop Distributed File System)

Description

HDFS creates a new HDFS connection that can be used to read files from HDFS.

Usage

  HDFS(path, mode = "r", fs, buffer = 8388608L)

Arguments

path	path of the file in HDFS
mode	mode to use for auto-open
fs	optional, FileSystem object
buffer	buffer size to use for pre-loading the data

Details

HDFS creates a connection that can be used for reading files from HDFS (currently write is unimplemented). The connection can be opened either for text reading "r" or for binary reads "rb".

The buffer parameter specifies the size (in bytes) of the buffer used to read the data.

Value

HDFS connection

Author(s)

Simon Urbanek

Examples

## Not run: 
f <- HDFS("/projects/foo/bar", "r")
d <- read.table(f, stringsAsFactors=FALSE)
close(f)

## End(Not run)

s-u/hdfsc documentation built on May 28, 2019, 10:47 a.m.