hsKeyValReader: Reads key value pairs

In HadoopStreaming: Utilities for using R scripts in Hadoop streaming

Description

Uses scan to read in chunkSize lines at a time, where each line consists of a key string and a value string. The first skip lines of input are skipped. Each group of key/value pairs are passed to FUN as a character vector of keys and character vector of values.

Usage

hsKeyValReader(file = "", chunkSize = -1, skip = 0, sep = "\t",FUN = function(k, v) cat(paste(k, v, sep = ": "), sep = "\n"))

Arguments

file	A connection object or a character string, as in scan.
chunkSize	The (maximal) number of lines to read at a time. The default is -1, which specifies that the whole file should be read at once.
skip	Number of lines to ignore at the beginning of the file
FUN	A function that takes a character vector as input
sep	The character separating the key and the value strings.

Value

No return value.

Author(s)

David S. Rosenberg. <[e-mail]>

Examples

  printFn <- function(k,v) {
    cat('A chunk:\n')
    cat(paste(k,v,sep=': '),sep='\n')
  }
  str <- "key1\tval1\nkey2\tval2\nkey3\tval3\n"
  cat(str)
  con <- textConnection(str, open = "r")
  hsKeyValReader(con,chunkSize=2,FUN=printFn)
  close(con)
  con <- textConnection(str, open = "r")
  hsKeyValReader(con,FUN=printFn)
  close(con)

HadoopStreaming documentation built on May 1, 2019, 9:12 p.m.