Description Usage Arguments Examples
Experimental helper function for reading text data on HDFS into a HDFS connection
1 2 |
input |
a ddo / ddf connection to a text input directory on HDFS, created with |
output |
an output connection such as those created with |
overwrite |
logical; should existing output location be overwritten? (also can specify |
fn |
function to be applied to each chunk of lines (input to function is a vector of strings) |
keyFn |
optional function to determine the value of the key for each block |
linesPerBlock |
how many lines at a time to read |
control |
parameters specifying how the backend should handle things (most-likely parameters to |
update |
should a MapReduce job be run to obtain additional attributes for the result data prior to returning? |
1 2 3 4 5 6 7 8 9 10 | ## Not run:
res <- readHDFStextFile(
input = Rhipe::rhfmt("/path/to/input/text", type = "text"),
output = hdfsConn("/path/to/output"),
fn = function(x) {
read.csv(textConnection(paste(x, collapse = "\n")), header = FALSE)
}
)
## End(Not run)
|
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.