text_tokenize.rtext: function tokenizing rtext objects

Description Usage Arguments

View source: R/text_tools.R

Description

function tokenizing rtext objects

Usage

1
2
3
## S3 method for class 'rtext'
text_tokenize(string, regex = NULL, ignore.case = FALSE,
  fixed = FALSE, perl = FALSE, useBytes = FALSE, non_token = FALSE)

Arguments

string

text to be tokenized

regex

regex expressing where to cut see (see gregexpr)

ignore.case

whether or not reges should be case sensitive (see gregexpr)

fixed

whether or not regex should be interpreted as is or as regular expression (see gregexpr)

perl

whether or not Perl compatible regex should be used (see gregexpr)

useBytes

byte-by-byte matching of regex or character-by-character (see gregexpr)

non_token

should information for non-token, i.e. those patterns by which the text was splitted, be returned as well


petermeissner/rtext documentation built on May 24, 2017, 4:32 a.m.