text_hashing_trick: Converts a text to a sequence of indexes in a fixed-size...

Description Usage Arguments Details Value See Also

View source: R/preprocessing.R

Description

Converts a text to a sequence of indexes in a fixed-size hashing space.

Usage

1
2
3
4
5
6
7
8
text_hashing_trick(
  text,
  n,
  hash_function = NULL,
  filters = "!\"#$%&()*+,-./:;<=>?@[\\]^_`{|}~\t\n",
  lower = TRUE,
  split = " "
)

Arguments

text

Input text (string).

n

Dimension of the hashing space.

hash_function

if NULL uses python hash function, can be 'md5' or any function that takes in input a string and returns a int. Note that hash is not a stable hashing function, so it is not consistent across different runs, while 'md5' is a stable hashing function.

filters

Sequence of characters to filter out such as punctuation. Default includes basic punctuation, tabs, and newlines.

lower

Whether to convert the input to lowercase.

split

Sentence split marker (string).

Details

Two or more words may be assigned to the same index, due to possible collisions by the hashing function.

Value

A list of integer word indices (unicity non-guaranteed).

See Also

Other text preprocessing: make_sampling_table(), pad_sequences(), skipgrams(), text_one_hot(), text_to_word_sequence()


dfalbel/keras documentation built on Nov. 27, 2019, 8:16 p.m.