ngram_tokenizer: Ngrams tokenizer

View source: R/pack.R

ngram_tokenizerR Documentation

Ngrams tokenizer

Description

Makes an ngram tokenizer function.

Usage

ngram_tokenizer(n = 1L)

Arguments

n

Integer.

Value

ngram tokenizer function

Examples

bigram <- ngram_tokenizer(2)
bigram(letters, sep = "-")

paithiov909/kagomer documentation built on June 12, 2025, 7:44 a.m.