tests/benchmarks/benchmark_tokens/dfm.R

txt <- rep(paste0(letters, collapse = ' '), 10000)
toks <- tokens(txt, hash = FALSE)
toks_hash <- tokens(txt, hash = TRUE)

profvis(
  dfm(toks)
)
profvis(
  dfm(toks_hash)
)
koheiw/quanteda.core documentation built on Sept. 21, 2020, 3:44 p.m.