serialize_tokens: Function to serialize list-of-character tokens

Description Usage Arguments Value

View source: R/tokens.R

Description

Creates a serialized object of tokens, called by tokens().

Usage

1
serialize_tokens(x, types_reserved = NULL, ...)

Arguments

x

a list of character vectors

types_reserved

optional pre-existing types for mapping of tokens

...

additional arguments

Value

a list the serialized tokens found in each text


koheiw/quanteda.core documentation built on Sept. 21, 2020, 3:44 p.m.