Stream-class: Class to represent a Producer and zero or more Consumers
In Streamer: Enabling stream processing of large files

Description Usage Arguments Constructors Methods Internal Class Fields and Methods Author(s) See Also Examples

An ordered collection of Consumer and Producer components combined into a single entity. Applying a method such as yield to Stream invokes yield on the terminal Consumer component of the stream, yielding one batch from the stream. The result of yield is defined by the Producer and Consumer components of the stream.

Stream(x, ..., verbose=FALSE)

## S4 method for signature 'Stream'
length(x)

## S4 method for signature 'Stream,numeric'
x[[i, j, ...]]

## S4 method for signature 'Stream'
lapply(X, FUN, ...)

## S4 method for signature 'Stream'
sapply(X, FUN, ..., simplify=TRUE, USE.NAMES=TRUE)

`x, X`	For `Stream`, `x` is a `Producer` instance. For other functions, an instance of class `Stream`.
`FUN`	A function to be applied to each successful `yield()` of `X`.
`i, j`	Numeric index of the ith stream element (`j` is ignored by this method).
`...`	For `Stream`, zero or more `Consumer` instances. For `lapply`, `sapply`, additional arguments to `FUN`.
`simplify`	See `?base::sapply`.
`USE.NAMES`	See `?base::sapply` but note that names do not usually make sense for instances of class `Producer`.
`verbose`	A `logical(1)` indicating whether status information should be reported.

Arguments to Stream must consist of a single Producer and zero or more Consumer components.

When invoked with the Producer as the first argument, Stream(P, C1, C2) produces a stream in which the data is read by P, then processed by C1, then processed by C2.

When invoked with the Consumer as the first argument, the ... must include a Producer as the last argument. Stream(C1, C2, P) produces a stream in which the data is read by P, then processed by C2, then processed by C1.

Methods defined on this class include:

length: The number of components in this stream.
[[: The ith component (including inputs) of this stream.
yield: Yield a single result (e.g., data.frame) from the stream.
reset: Reset, if possible, each component of the stream.
lapply, sapply: Apply FUN to each result applied to yield(), simplifying (using simplify2array) if possible for sapply. Partial results on error can be recovered using tryCatch, as illustrated on the help page Producer.

Internal fields of this class are are described with, e.g., getRefClass("FunctionProducer")$fields.

Internal methods of this class are described with getRefClass("FunctionProducer")$methods() and getRefClass("FunctionProducer")$help().

Martin Morgan mtmorgan@fhcrc.org

Streamer-package, Consumer-class, Producer-class.

fl <- system.file("extdata", "s_1_sequence.txt", package="Streamer")
b <- RawInput(fl, 100L, reader=rawReaderFactory(1e4))
s <- Stream(b, Rev(), RawToChar())
s
yield(s)
reset(s)
while (length(yield(s))) cat("tick\n")
close(b)

strm <- Stream(Seq(to=10), FunctionConsumer(function(y) 1/y))
sapply(strm, c)