tokenlist | R Documentation |
A tokenlist object is a thin wrapper around a list of character vectors, with a few attributes.
tokenlist(tokens = list(), lemma = NULL, pos = NULL)
tokens |
List of character vectors |
lemma |
List of character vectors, must be same size and shape as |
pos |
List of character vectors, must be same size and shape as |
a tokenlist object.
abc <- list(letters, LETTERS)
tokenlist(abc)
unclass(tokenlist(abc))
tibble(text = tokenlist(abc))
library(tokenizers)
library(modeldata)
data(tate_text)
tokens <- tokenize_words(as.character(tate_text$medium))
tokenlist(tokens)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.