process_blocks: Blockwise processing of file
In djvanderlaan/LaF: Fast Access to Large ASCII Files

Reads the specified file block by block and feeds each block to the specified function.

process_blocks(x, fun, ...)

## S4 method for signature 'laf'
process_blocks(
  x,
  fun,
  columns = 1:ncol(x),
  nrows = 5000,
  allow_interupt = FALSE,
  progress = FALSE,
  ...
)

`x`	an object the supports the `process_blocks` method, such as an `laf` object.
`fun`	a function to apply to each block (see details).
`...`	additional parameters are passed on to `fun`.
`columns`	an integer vector with the columns that should be read in.
`nrows`	the (maximum) number of rows to read in one block
`allow_interupt`	when TRUE the function `fun` is expected to return a list. The second element is the result of the function. The first element should be a logical value indication whether `process_blocks` should continue (FALSE) or stop (TRUE). When interrupted the function is not called a last time with an empty `data.frame` to finalize the result.
`progress`	show a progress bar. Note that this triggers a calculation of the number of lines in the file which for CSV files can take some time. When numeric `code` is used as the style of the progress bar (see `txtProgressBar`).

The function should accept as the first argument the next block of data. When the end of the file is reached this is an empty (zero row) data.frame. As the second argument the function should accept the output of the previous call to the function. The first time the function is called the second argument has the value NULL.

djvanderlaan/LaF documentation built on April 2, 2020, 7:10 a.m.