BaseTokenizer: BaseTokenizer

View source: R/text_core.R

BaseTokenizerR Documentation

BaseTokenizer

Description

Basic tokenizer that just splits on spaces

Usage

BaseTokenizer(split_char = " ")

Arguments

split_char

separator

Value

None


EagerAI/fastai documentation built on April 16, 2024, 12:01 p.m.