txtq: A Small Message Queue for Parallel Processes

name: Bottleneck about: txtq is too slow or consumes too many resources. title: '' labels: 'topic: performance' assignees: wlandau

[ ] Read and abide by txtq's code of conduct.
[ ] Search for duplicates among the existing issues, both open and closed.
[ ] Advanced users: verify that the bottleneck still persists in the current development version (i.e. remotes::install_github("wlandau/txtq")) and mention the SHA-1 hash of the Git commit you install.

Describe the bottleneck clearly and concisely.

Provide a minimal example with code that reproduces the problem.

How poorly does txtq perform? Please share benchmarks: runtimes, memory consumption, flame graphs, etc. Tools to consider:

In development txtq, make(console_log_file = "log.txt") now prepends sub-second time stamps to each line of log.txt. It is a super convenient way to see how fast things are going.
Rprof(), jointprof, profile, and pprof. Example here.
profvis, though beware https://github.com/rstudio/profvis/issues/104.
microbenchmark and bench.