Streamer: Enabling stream processing of large files

Large data files can be difficult to work with in R, where data generally resides in memory. This package encourages a style of programming where data is 'streamed' from disk into R via a `producer' and through a series of `consumers' that, typically reduce the original data to a manageable size. The package provides useful Producer and Consumer stream components for operations such as data input, sampling, indexing, and transformation; see package?Streamer for details.

AuthorMartin Morgan, Nishant Gopalakrishnan
Date of publicationNone
MaintainerMartin Morgan <martin.morgan@roswellpark.org>
LicenseArtistic-2.0
Version1.20.0

View on Bioconductor

Files in this package

Streamer/DESCRIPTION
Streamer/NAMESPACE
Streamer/R
Streamer/R/AllGenerics.R Streamer/R/ConnectionProducer-classes.R Streamer/R/Consumer-class.R Streamer/R/DAGParam-class.R Streamer/R/DAGParam-methods.R Streamer/R/DAGTeam-class.R Streamer/R/Downsample-class.R Streamer/R/FunctionProducerConsumer-classes.R Streamer/R/ParallelParam-classes.R Streamer/R/Producer-class.R Streamer/R/RawInput-class.R Streamer/R/Reducer-class.R Streamer/R/Seq-class.R Streamer/R/Stream-class.R Streamer/R/Streamer-class.R Streamer/R/Team-class.R Streamer/R/Team-methods.R Streamer/R/Utility-classes.R Streamer/R/lapply-methods.R Streamer/R/plot-methods.R Streamer/R/stream-methods.R Streamer/R/zzz.R
Streamer/build
Streamer/build/vignette.rds
Streamer/inst
Streamer/inst/README-ncdf4.txt
Streamer/inst/doc
Streamer/inst/doc/Streamer.R
Streamer/inst/doc/Streamer.Rnw
Streamer/inst/doc/Streamer.pdf
Streamer/inst/extdata
Streamer/inst/extdata/NetCDFData.nc
Streamer/inst/extdata/s_1_sequence.txt
Streamer/inst/scripts
Streamer/inst/scripts/NetCDFFile.R
Streamer/inst/scripts/NetCDFInput.R
Streamer/inst/scripts/testParallel.R
Streamer/inst/unitTests
Streamer/inst/unitTests/test_ConnectionProducer.R
Streamer/inst/unitTests/test_DAGTeam.R
Streamer/inst/unitTests/test_Producer.R
Streamer/inst/unitTests/test_RawInput.R
Streamer/inst/unitTests/test_Reducer.R
Streamer/inst/unitTests/test_RevStream.R
Streamer/inst/unitTests/test_Stream.R
Streamer/inst/unitTests/test_Streamer.R
Streamer/inst/unitTests/test_Team.R
Streamer/man
Streamer/man/ConnectionProducer-classes.Rd Streamer/man/Consumer-class.Rd Streamer/man/DAGTeam-class.Rd Streamer/man/Downsample-class.Rd Streamer/man/FunctionProducerConsumer-classes.Rd Streamer/man/ParallelParam-classes.Rd Streamer/man/Producer-class.Rd Streamer/man/RawInput-class.Rd Streamer/man/Reducer-class.Rd Streamer/man/Seq-class.Rd Streamer/man/Stream-class.Rd Streamer/man/Streamer-package.Rd Streamer/man/Team-class.Rd Streamer/man/Utility-classes.Rd Streamer/man/reset.Rd Streamer/man/status.Rd Streamer/man/yield.Rd
Streamer/src
Streamer/src/R_init_Streamer.c
Streamer/src/raw_input.c
Streamer/src/raw_input.h
Streamer/tests
Streamer/tests/unit_tests.R
Streamer/vignettes
Streamer/vignettes/Streamer.Rnw
Streamer/vignettes/StreamerClasses.xmi

Questions? Problems? Suggestions? or email at ian@mutexlabs.com.

All documentation is copyright its authors; we didn't write any of that.