byte_text_encoder: Byte Text Encoder
In rstudio/tfds: Interface to 'TensorFlow' Collection of Datasets
Description
Usage
Arguments
See Also
View source: R/features_text.R
Description
Byte encodes text
Usage
1
byte_text_encoder(additional_tokens = NULL)
Arguments
additional_tokens
list of additional tokens. These will be assigned
vocab ids [1, 1+len(additional_tokens)]. Useful for things like "end-of-string"
tokens (e.g. "").
See Also
save_byte_text_encoder(), load_byte_text_encoder(), encode()
and decode()
rstudio/tfds documentation built on Nov. 25, 2021, 6:20 p.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Description Usage Arguments See Also
View source: R/features_text.R
Description
Byte encodes text
Usage
1 | byte_text_encoder(additional_tokens = NULL)
|
Arguments
additional_tokens |
list of additional tokens. These will be assigned vocab ids [1, 1+len(additional_tokens)]. Useful for things like "end-of-string" tokens (e.g. ""). |
See Also
save_byte_text_encoder(), load_byte_text_encoder(), encode()
and decode()
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.