ngram_tokenizer: Ngrams tokenizer

View source: R/pack.R

ngram_tokenizerR Documentation

Ngrams tokenizer

Description

Makes an ngram tokenizer function.

Usage

ngram_tokenizer(n = 1L)

Arguments

n

Integer.

Value

ngram tokenizer function

Examples

bigram <- ngram_tokenizer(2)
bigram(letters, sep = "-")

gibasa documentation built on Sept. 11, 2024, 9:23 p.m.