tokens_restore: Restore special tokens

View source: R/tokens_restore.R

tokens_restoreR Documentation

Restore special tokens

Description

Compounds a sequence of tokens marked by special markers. The beginning and the end of the sequence should be marked by U+E001 and U+E002 respectively.

Usage

tokens_restore(x)

Arguments

x

tokens object


quanteda/quanteda documentation built on April 30, 2024, 3:13 p.m.