get_tokens: Word Tokenization
In syuzhet: Extracts Sentiment and Sentiment-Derived Plot Arcs from Text
get_tokens R Documentation
Word Tokenization
Description
Parses a string into a vector of word tokens.
Usage
get_tokens(text_of_file, pattern = "\\W", lowercase = TRUE)
Arguments
text_of_file
A Text String
pattern
A regular expression for token breaking
lowercase
should tokens be converted to lowercase. Default equals TRUE
Value
A Character Vector of Words
syuzhet documentation built on Aug. 12, 2023, 1:05 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| get_tokens | R Documentation |
Word Tokenization
Description
Parses a string into a vector of word tokens.
Usage
get_tokens(text_of_file, pattern = "\\W", lowercase = TRUE)
Arguments
text_of_file |
A Text String |
pattern |
A regular expression for token breaking |
lowercase |
should tokens be converted to lowercase. Default equals TRUE |
Value
A Character Vector of Words
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.