text_to_word_sequence: Convert text to a sequence of words (or tokens).

In keras: R Interface to 'Keras'

Convert text to a sequence of words (or tokens).

Description

Convert text to a sequence of words (or tokens).

Usage

text_to_word_sequence(
  text,
  filters = "!\"#$%&()*+,-./:;<=>?@[\\]^_`{|}~\t\n",
  lower = TRUE,
  split = " "
)

Arguments

text	Input text (string).
filters	Sequence of characters to filter out such as punctuation. Default includes basic punctuation, tabs, and newlines.
lower	Whether to convert the input to lowercase.
split	Sentence split marker (string).

Value

Words (or tokens)

See Also

Other text preprocessing: make_sampling_table(), pad_sequences(), skipgrams(), text_hashing_trick(), text_one_hot()

keras documentation built on May 29, 2024, 3:20 a.m.