FullTokenizer: Construct objects of FullTokenizer class.
In jonathanbratt/RBERT: R Implementation of BERT
FullTokenizer R Documentation
Construct objects of FullTokenizer class.
Description
Construct objects of FullTokenizer class.
Usage
FullTokenizer(vocab_file, do_lower_case = TRUE)
Arguments
vocab_file
Path to text file containing list of vocabulary tokens.
do_lower_case
Logical: do we convert everything to lowercase?
Value
An object of class FullTokenizer.
Examples
## Not run:
f_tokenizer <- FullTokenizer("vocab.txt", TRUE)
## End(Not run)
jonathanbratt/RBERT documentation built on Jan. 26, 2023, 4:15 p.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| FullTokenizer | R Documentation |
Construct objects of FullTokenizer class.
Description
Construct objects of FullTokenizer class.
Usage
FullTokenizer(vocab_file, do_lower_case = TRUE)
Arguments
vocab_file |
Path to text file containing list of vocabulary tokens. |
do_lower_case |
Logical: do we convert everything to lowercase? |
Value
An object of class FullTokenizer.
Examples
## Not run:
f_tokenizer <- FullTokenizer("vocab.txt", TRUE)
## End(Not run)
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.