selectInBatches: Select in Batches

In girke-lab/ChemmineR: Cheminformatics Toolkit for R

Select in Batches

Description

When doing a select were the condition is a large number of ids it is not always possible to include them in a single SQL statement. This function will break the list of ids into chunks and send the query for each batch. The resutls are appended and returned as one data frame.

Usage

selectInBatches(conn, allIndices, genQuery, batchSize = 1e+05)

Arguments

conn	Database connection object
allIndices	A vector of indices to pass to the genQuery function in batches.
genQuery	A function which takes a vector of indices and constructs an SQL SELECT statement returning records for the given indicies.
batchSize	How many indicies to put in each batch.

Value

A data frame with the results of the query as if all inidices had been included in a single SELEcT statement.

Author(s)

Kevin Horan

Examples

##---- Should be DIRECTLY executable !! ----
##-- ==>  Define data, use random,
##--	or do  help(data=index)  for the standard data sets.

## The function is currently defined as
function (conn, allIndices, genQuery, batchSize = 1e+05) 
{
    batchByIndex(allIndices, function(indexBatch) {
        df = dbGetQuery(conn, genQuery(indexBatch))
        result = rbind(result, df)
    }, batchSize)
    result
  }

girke-lab/ChemmineR documentation built on July 28, 2023, 10:36 a.m.