tokenize_folder: Tokenize_folder
In fastai: Interface to 'fastai'

Description Usage Arguments Value

View source: R/text_core.R

Tokenize text files in 'path' in parallel using 'n_workers'

tokenize_folder(
  path,
  extensions = NULL,
  folders = NULL,
  output_dir = NULL,
  skip_if_exists = TRUE,
  output_names = NULL,
  n_workers = 6,
  rules = NULL,
  tok = NULL,
  encoding = "utf8"
)

`path`	path
`extensions`	extensions
`folders`	folders
`output_dir`	output_dir
`skip_if_exists`	skip_if_exists
`output_names`	output_names
`n_workers`	number of workers
`rules`	rules
`tok`	tokenizer
`encoding`	encoding

None

fastai documentation built on July 28, 2021, 5:06 p.m.

fastai index

README.md Audio Classification Basic Image Classification Basic Tabular Bayesian Optimisation Callbacks Custom Image Classification Data augmentation GPT2 Head pose Low-level ops Medical image Migrating from Catalyst Migrating from Ignite Migrating from Lightning Migrating from Pytorch Multilabel classification Object detection Optimizer Question-Answering RoBERTa Speech Recognition Super-Resolution GAN Text-summarization Time-Series

rdrr.io home R language documentation Run R code online

CRAN packages Bioconductor packages R-Forge packages GitHub packages

Note that we can't provide technical support on individual packages. You should contact the package authors for that.

Tweet to @rdrrHQ

GitHub issue tracker

ian@mutexlabs.com