tests/benchmarks/benchmark_tokens/dfm.R

txt <- rep(paste0(letters, collapse = ' '), 10000)
toks <- tokens(txt, hash = FALSE)
toks_hash <- tokens(txt, hash = TRUE)

profvis(
  dfm(toks)
)
profvis(
  dfm(toks_hash)
)
quanteda/quanteda documentation built on Sept. 4, 2024, 7:56 p.m.