batchByIndex: Batch by Index

Description Usage Arguments Value Author(s) See Also Examples

Description

When doing a select were the condition is a large number of ids it is not always possible to include them in a single SQL statement. This function will break the list of ids into chunks and allow the indexProcessor to deal with just a small number of ids.

Usage

1
batchByIndex(allIndices, indexProcessor, batchSize = 1e+05)

Arguments

allIndices

A vector of values that will be broken into batches and passed as an argument to the indexProcessor function.

indexProcessor

A function that takes one batch if indices. It is called once for each batch. The return value from this function is ignored. To accumulate results you can write to a global variable using the "<<-" operator.

batchSize

The size of each batch. The last batch may be smaller than this value.

Value

No value is returned.

Author(s)

Kevin Horan

See Also

parBatchByIndex

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
	
	## Not run: 
		result=NA
		indices = 1:10000
		
		#run a query on each batch of indexes, appending each result to
		# "result" as we go.
		batchByIndex(indices, function(indexBatch){
				df = dbGetQuery(dbConnection, generateQuery(indexBatch))
				result <<- if(is.na(result)) df else  rbind(result,df)
		},1000)
	
## End(Not run)

girke-lab/ChemmineR-release documentation built on May 17, 2019, 5:26 a.m.