char_tokenizer: Returns vector of tokens, tokenized by character

Description Usage Arguments Value Examples

View source: R/tokenization.R

Description

Returns vector of tokens, tokenized by character

Usage

1

Arguments

input

String to tokenize

Value

vector of characters

Examples

1
2
char_tokenizer('hello')
# c("h", "e", "l", "l", "o")

perfail-stud/Rstudfile documentation built on Dec. 22, 2021, 7:43 a.m.