batchByIndex: Batch by Index

In girke-lab/ChemmineR: Cheminformatics Toolkit for R

Batch by Index

Description

When doing a select were the condition is a large number of ids it is not always possible to include them in a single SQL statement. This function will break the list of ids into chunks and allow the indexProcessor to deal with just a small number of ids.

Usage

batchByIndex(allIndices, indexProcessor, batchSize = 1e+05)

Arguments

allIndices	A vector of values that will be broken into batches and passed as an argument to the indexProcessor function.
indexProcessor	A function that takes one batch if indices. It is called once for each batch. The return value from this function is ignored. To accumulate results you can write to a global variable using the "<<-" operator.
batchSize	The size of each batch. The last batch may be smaller than this value.

Value

No value is returned.

Author(s)

Kevin Horan

See Also

parBatchByIndex

Examples

	
	## Not run: 
		result=NA
		indices = 1:10000
		
		#run a query on each batch of indexes, appending each result to
		# "result" as we go.
		batchByIndex(indices, function(indexBatch){
				df = dbGetQuery(dbConnection, generateQuery(indexBatch))
				result <<- if(is.na(result)) df else  rbind(result,df)
		},1000)
	
## End(Not run)

girke-lab/ChemmineR documentation built on July 28, 2023, 10:36 a.m.