tok_normalizer: Generic class for normalizers
In tok: Fast Text Tokenization
tok_normalizer R Documentation
Generic class for normalizers
Description
Generic class for normalizers
Public fields
.normalizerInternal pointer to normalizer object
Methods
Public methods
tok_normalizer$new()
Initializes a tokenizer
Usage
tok_normalizer$new(normalizer)
Arguments
normalizera raw pointer to a tokenizer
tok_normalizer$clone()
The objects of this class are cloneable with this method.
Usage
tok_normalizer$clone(deep = FALSE)
Arguments
deepWhether to make a deep clone.
See Also
Other normalizers:
normalizer_nfc,
normalizer_nfkc
tok documentation built on June 21, 2026, 5:07 p.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| tok_normalizer | R Documentation |
Generic class for normalizers
Description
Generic class for normalizers
Public fields
.normalizerInternal pointer to normalizer object
Methods
Public methods
tok_normalizer$new()
Initializes a tokenizer
Usage
tok_normalizer$new(normalizer)
Arguments
normalizera raw pointer to a tokenizer
tok_normalizer$clone()
The objects of this class are cloneable with this method.
Usage
tok_normalizer$clone(deep = FALSE)
Arguments
deepWhether to make a deep clone.
See Also
Other normalizers:
normalizer_nfc,
normalizer_nfkc
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.