parallel_tokenize: Parallel_tokenize

View source: R/text_core.R

parallel_tokenizeR Documentation

Parallel_tokenize

Description

Calls optional 'setup' on 'tok' before launching 'TokenizeWithRules' using 'parallel_gen

Usage

parallel_tokenize(items, tok = NULL, rules = NULL, n_workers = 6)

Arguments

items

items

tok

tokenizer

rules

rules

n_workers

n_workers

Value

None


EagerAI/fastai documentation built on April 16, 2024, 12:01 p.m.