text_tokenize.rtext: function tokenizing rtext objects

In rtext: R6 Objects for Text and Data

Description

function tokenizing rtext objects

Usage

## S3 method for class 'rtext'
text_tokenize(
  string,
  regex = NULL,
  ignore.case = FALSE,
  fixed = FALSE,
  perl = FALSE,
  useBytes = FALSE,
  non_token = FALSE
)

Arguments

string	text to be tokenized
regex	regex expressing where to cut see (see grep)
ignore.case	whether or not reges should be case sensitive (see grep)
fixed	whether or not regex should be interpreted as is or as regular expression (see grep)
perl	whether or not Perl compatible regex should be used (see grep)
useBytes	byte-by-byte matching of regex or character-by-character (see grep)
non_token	should information for non-token, i.e. those patterns by which the text was splitted, be returned as well

rtext documentation built on Jan. 28, 2021, 9:05 a.m.