table_indices: Returns indices to process large tables

In macroecology-society/masDMT: Data Management Tools

Returns indices to process large tables

Description

Returns indices to process large tables

Usage

table_indices(path, size)

Arguments

path	Path to tabular data file.
size	Numeric element specifying the max. data volume (in Mb).

Details

This function helps process tables that are larger than memory. Given a file path, the function infers the storage size of the input file and determines an optimal number of runs. To do, the argument size is used, which determines the maximum amount of RAM that should be occupied in a single read. Then, for each of the n runs, the function determines the corresponding start and end rows, which are reported as a list. The output can be used in combination with, e.g., read_delim, which allows users to specify starting and ending rows when importing a file.

Value

list object with indices

macroecology-society/masDMT documentation built on July 2, 2024, 4:08 a.m.