parallel_tokenize: Parallel_tokenize

Description Usage Arguments Value

View source: R/text_core.R

Description

Calls optional 'setup' on 'tok' before launching 'TokenizeWithRules' using 'parallel_gen

Usage

1
parallel_tokenize(items, tok = NULL, rules = NULL, n_workers = 6)

Arguments

items

items

tok

tokenizer

rules

rules

n_workers

n_workers

Value

None


fastai documentation built on Oct. 25, 2021, 5:08 p.m.