bigMatrix: Load a 'bigMatrix' (deprecated)

In NetRep: Permutation Testing Network Module Preservation Across Datasets

Load a 'bigMatrix' (deprecated)

Description

The 'bigMatrix' class is no longer implemented in the NetRep package: the shared memory approach was incompatabile with high performance compute clusters, so the parallel permutation procedure has been translated into C++ code (which is also much faster). The disk.matrix class should now be used instead when analysing large datasets.

Usage

load.bigMatrix(backingfile)

Arguments

backingfile

path to the backingfile for the 'bigMatrix'. The file extension must be omitted.

Details

This function will convert 'bigMatrix' data saved by previous versions of NetRep to a serialized R matrix saved in the same location and return a disk.matrix object with the associated file path. If this conversion has taken place already the function will throw a warning.

This function will also convert the 'bigMatrix' descriptor file to a big.matrix descriptor file to preserve compatability with functions in the bigmemory package. If this functionality is not required, the files with the extensions ".bin" and ".desc" may be removed.

A note for users using multi-node high performance clusters: 'big.matrix' objects are not suitable for general usage. Access to file-backed shared memory segments on multi-node systems is very slow due to consistency checks performed by the operating system. This becomes exponentially worse the more R sessions there are simultaneously accessing the shared memory segment, e.g. through parallel foreach loops.

NetRep documentation built on Aug. 19, 2023, 5:07 p.m.