serialize_tokens: Function to serialize list-of-character tokens

View source: R/tokens.R

serialize_tokensR Documentation

Function to serialize list-of-character tokens

Description

Creates a serialized object of tokens, called by tokens().

Usage

serialize_tokens(x, types_reserved = NULL, ...)

Arguments

x

a list of character vectors

types_reserved

optional pre-existing types for mapping of tokens

...

additional arguments

Value

a list the serialized tokens found in each text


quanteda documentation built on May 31, 2023, 8:28 p.m.