LG_splitting: Split local Gaussian related computations into smaller pieces

In LAJordanger/localgaussSpec: Local Gaussian Spectral Analysis

Split local Gaussian related computations into smaller pieces

Description

To reduce the chance for out of memory problems (see help("Memory-limits") for more information), some computations will be partitioned into smaller chunks. This function checks if a splitting is required, and it gives a partitioning of the arguments. This should (hopefully) ensure that the available amount of memory is sufficient for the task at hand. Warning: This is a rather crude approach, that does not measure how much memory that is available, i.e. it might fail if the work-space is cluttered with large objects.

Usage

LG_splitting(books)

Arguments

books

The (internal) result we get from LG_bookkeeping, which contains all the information needed for this function. This includes paths to data-files, the size and dimension-names of the objects found in these – and the arguments that will be used in the desired computation based upon these data.

Details

The argument and value of this function reflects that it is an internal function of the LG_scribe-functions. There are two aspects this function takes care of. First of all it aims at ensuring that the final objects that are saved to file(s) stay below some threshold. Moreover, as some intermediate objects can grow very large (in particular when dealing with bootstrap-replicates), it will also check if a computation should be divided into smaller chunks in order to avoid memory problems. The strategy used in order to obtain this is to "split" the array by chopping up some of the dimensions.

Value

This function will return a "loop-list" to be used internally in the LG_scribe-function. This is a list with the following components:

data_files

This contains the paths to the files that the main-function must read in order to access the data.

arg_list

This contains the arguments needed for the computation – and this part will also decide if a computation must be split into smaller chunks due to large intermediate objects.

LAJordanger/localgaussSpec documentation built on May 6, 2023, 4:31 a.m.