dot-get_tokenizer: Look Up Tokenizer Function
In macmillancontentscience/torchtransformers: Transformer Models in Torch
.get_tokenizer R Documentation
Look Up Tokenizer Function
Description
Given a string representing the name of a tokenization algorithm, return the
corresponding tokenization function.
Usage
.get_tokenizer(tokenizer_name)
Arguments
tokenizer_name
Character; the name of the tokenization algorithm.
Value
The function implementing the specified algorithm.
macmillancontentscience/torchtransformers documentation built on Aug. 6, 2023, 5:35 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| .get_tokenizer | R Documentation |
Look Up Tokenizer Function
Description
Given a string representing the name of a tokenization algorithm, return the corresponding tokenization function.
Usage
.get_tokenizer(tokenizer_name)
Arguments
tokenizer_name |
Character; the name of the tokenization algorithm. |
Value
The function implementing the specified algorithm.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.