selectInBatches: Select in Batches

Description Usage Arguments Value Author(s) Examples

Description

When doing a select were the condition is a large number of ids it is not always possible to include them in a single SQL statement. This function will break the list of ids into chunks and send the query for each batch. The resutls are appended and returned as one data frame.

Usage

1
selectInBatches(conn, allIndices, genQuery, batchSize = 1e+05)

Arguments

conn

Database connection object

allIndices

A vector of indices to pass to the genQuery function in batches.

genQuery

A function which takes a vector of indices and constructs an SQL SELECT statement returning records for the given indicies.

batchSize

How many indicies to put in each batch.

Value

A data frame with the results of the query as if all inidices had been included in a single SELEcT statement.

Author(s)

Kevin Horan

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
##---- Should be DIRECTLY executable !! ----
##-- ==>  Define data, use random,
##--	or do  help(data=index)  for the standard data sets.

## The function is currently defined as
function (conn, allIndices, genQuery, batchSize = 1e+05) 
{
    batchByIndex(allIndices, function(indexBatch) {
        df = dbGetQuery(conn, genQuery(indexBatch))
        result = rbind(result, df)
    }, batchSize)
    result
  }


Search within the ChemmineR package
Search all R packages, documentation and source code

Questions? Problems? Suggestions? or email at ian@mutexlabs.com.

Please suggest features or report bugs with the GitHub issue tracker.

All documentation is copyright its authors; we didn't write any of that.