as_tokens: Coerce object to class 'tokens'

In mclm: Mastering Corpus Linguistics Methods

Coerce object to class tokens

Description

This function coerces a character object or another object that can be coerced to a character into an object of class tokens.

Usage

as_tokens(x, ...)

Arguments

x	Object to coerce.
...	Additional arguments (not implemented).

Value

An object of class tokens.

Examples

toy_corpus <- "Once upon a time there was a tiny toy corpus.
It consisted of three sentences. And it lived happily ever after."

tks <- tokenize(toy_corpus)
print(tks, n = 1000)

tks[3:12]
print(as_tokens(tks[3:12]), n = 1000)
as_tokens(tail(tks))

mclm documentation built on Oct. 3, 2022, 9:07 a.m.