parallel_tokenize: Parallel_tokenize

View source: R/text_core.R

parallel_tokenizeR Documentation

Parallel_tokenize

Description

Calls optional 'setup' on 'tok' before launching 'TokenizeWithRules' using 'parallel_gen

Usage

parallel_tokenize(items, tok = NULL, rules = NULL, n_workers = 6)

Arguments

items

items

tok

tokenizer

rules

rules

n_workers

n_workers

Value

None


fastai documentation built on March 31, 2023, 11:41 p.m.