tokenize1: Tokenize1

View source: R/text_core.R

tokenize1R Documentation

Tokenize1

Description

Call 'TokenizeWithRules' with a single text

Usage

tokenize1(text, tok, rules = NULL, post_rules = NULL)

Arguments

text

text

tok

tok

rules

rules

post_rules

post_rules

Value

None


fastai documentation built on March 31, 2023, 11:41 p.m.