simple wrappers around
base regular expressions.
For much more faster and functional tokenizers see
str_split_* functions in
The reason for not including this packages to
text2vec dependencies is our
desare to keep number of dependencies as small as possible.
1 2 3 4 5 6 7
other parameters to strsplit function, which is used under the hood.
Each element of list containts vector of tokens.
1 2 3 4 5