tokenize_dictionary: Separate dictionary values using tokenizer

View source: R/dictionaries.R

tokenize_dictionaryR Documentation

Separate dictionary values using tokenizer

Description

Apply tokens() to dictionary values to improve pattern matching.

Usage

tokenize_dictionary(dictionary, ...)

Arguments

dictionary

a dictionary object.

...

passed to tokens().


quanteda documentation built on Oct. 21, 2026, 5:07 p.m.