wordpiece_vocab: Load a wordpiece Vocabulary
In wordpiece.data: Data for Wordpiece-Style Tokenization
wordpiece_vocab R Documentation
Load a wordpiece Vocabulary
Description
A wordpiece vocabulary is a named integer vector with class
"wordpiece_vocabulary". The names of the vector are the tokens, and the
values are the integer identifiers of those tokens. The vocabulary is
0-indexed for compatibility with Python implementations.
Usage
wordpiece_vocab(cased = FALSE)
Arguments
cased
Logical; load the uncased vocabulary, or the cased vocabulary?
Value
A wordpiece_vocabulary.
Examples
head(wordpiece_vocab())
head(wordpiece_vocab(cased = TRUE))
wordpiece.data documentation built on March 18, 2022, 7:26 p.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| wordpiece_vocab | R Documentation |
Load a wordpiece Vocabulary
Description
A wordpiece vocabulary is a named integer vector with class "wordpiece_vocabulary". The names of the vector are the tokens, and the values are the integer identifiers of those tokens. The vocabulary is 0-indexed for compatibility with Python implementations.
Usage
wordpiece_vocab(cased = FALSE)
Arguments
cased |
Logical; load the uncased vocabulary, or the cased vocabulary? |
Value
A wordpiece_vocabulary.
Examples
head(wordpiece_vocab()) head(wordpiece_vocab(cased = TRUE))
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.