ConnectionProducer-classes: Producer classes to read file connections

In Streamer: Enabling stream processing of large files

Description

ConnectionProducer classes include ScanProducer, ReadLinesProducer, and ReadTableProducer, providing Streamer interfaces to scan, readLines, and read.table.

Usage

ScanProducer(file, ..., fileArgs=list(), scanArgs=list(...))
ReadLinesProducer(con, ..., conArgs=list(), readLinesArgs=list(...))
ReadTableProducer(file, ..., fileArgs=list(), readTableArgs=list(...))

## S3 method for class 'ConnectionProducer'
close(con, ...)

Arguments

file, con	The file or connection to be used for input. See connections.
...	Additional arguments, e.g., nlines, to scan, readLines, etc.
fileArgs, conArgs	Arguments, e.g., mode, encoding, to be used when invoking reset().
scanArgs, readLinesArgs, readTableArgs	Arguments to scan, readLines, etc., when reading a file or connection; provide this argument when ... contains arguments (especially verbose=TRUE) to be used by the class.

Methods

See Producer Methods.

Internal Class Fields and Methods

Internal fields of this class are are described with, e.g., getRefClass("ReadLinesProducer")$fields.

Internal methods of this class are described with getRefClass("ReadLinesProducer")$methods() and getRefClass("ReadLinesProducer")$help().

Author(s)

Martin Morgan [e-mail]

See Also

Streamer-package, Producer-class, Streamer-class.

Examples

fl <- system.file(package="Rsamtools", "extdata", "ex1.sam")

p <- ReadLinesProducer(fl, n = 1000)  # read 1000 lines at a time
while (length(y <- yield(p)))
    print(length(y))
close(p)

p <- ReadTableProducer(fl, quote="", fill=TRUE, nrows=1000)
while (length(y <- yield(p)))
    print(dim(y))

reset(p)
dim(yield(p))

## connections opened 'under the hood' are closed, with warnings
rm(p); gc() 

## avoid warnings by managing connections
p <- ScanProducer(file(fl, "r"), verbose=TRUE,
                  scanArgs=list(what=character()))
length(yield(p))
close(p)
rm(p); gc()

Streamer documentation built on Nov. 8, 2020, 5:53 p.m.