threads_fst: Get or set the number of threads used in parallel operations

In fst: Lightning Fast Serialization of Data Frames

Description

For parallel operations, the performance is determined to a great extend by the number of threads used. More threads will allow the CPU to perform more computational intensive tasks simultaneously, speeding up the operation. Using more threads also introduces some overhead that will scale with the number of threads used. Therefore, using the maximum number of available threads is not always the fastest solution. With threads_fst the number of threads can be adjusted to the users specific requirements. As a default, fst uses a number of threads equal to the number of logical cores in the system.

Usage

threads_fst(nr_of_threads = NULL, reset_after_fork = NULL)

Arguments

nr_of_threads

number of threads to use or NULL to get the current number of threads used in multithreaded operations.

reset_after_fork

when fst is running in a forked process, the usage of OpenMP can create problems. To prevent these, fst switches back to single core usage when it detects a fork. After the fork, the number of threads is reset to it's initial setting. However, on some compilers (e.g. Intel), switching back to multi-threaded mode can lead to issues. When reset_after_fork is set to FALSE, fst is left in single-threaded mode after the fork ends. After the fork, multithreading can be activated again manually by calling threads_fst with an appropriate value for nr_of_threads. The default (reset_after_fork = NULL) leaves the fork behavior unchanged.

Details

The number of threads can also be set with options(fst_threads = N). NOTE: This option is only read when the package's namespace is first loaded, with commands like library, require, or ::. If you have already used one of these, you must use threads_fst to set the number of threads.

Value

the number of threads (previously) used

fst documentation built on Feb. 8, 2022, 9:06 a.m.