text_tokenize_words: generic to tokenize text into words
In stringb: Convenient Base R String Handling
View source: R/text_tokenize.R
Description
A wrapper to text_tokenize that tokenizes text into words.
Since using text_tokenize()'s option non_token might slow things
down considerably this one purpose wrapper is a little more clever
than the general implementation and hence much faster.
Usage
1
2
3
4
text_tokenize_words(string, non_token = FALSE)
## Default S3 method:
text_tokenize_words(string, non_token = FALSE)
Arguments
string
the text to be tokenized
non_token
whether or not token as well as non tokens shall be returned.
stringb documentation built on Jan. 26, 2021, 1:07 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
View source: R/text_tokenize.R
Description
A wrapper to text_tokenize that tokenizes text into words. Since using text_tokenize()'s option non_token might slow things down considerably this one purpose wrapper is a little more clever than the general implementation and hence much faster.
Usage
1 2 3 4 | text_tokenize_words(string, non_token = FALSE)
## Default S3 method:
text_tokenize_words(string, non_token = FALSE)
|
Arguments
string |
the text to be tokenized |
non_token |
whether or not token as well as non tokens shall be returned. |
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.