dot-process_mp_vocab: Process a Morphemepiece Vocabulary for Tokenization

.process_mp_vocabR Documentation

Process a Morphemepiece Vocabulary for Tokenization

Description

Process a Morphemepiece Vocabulary for Tokenization

Usage

.process_mp_vocab(v)

## Default S3 method:
.process_mp_vocab(v)

## S3 method for class 'morphemepiece_vocabulary'
.process_mp_vocab(v)

## S3 method for class 'integer'
.process_mp_vocab(v)

## S3 method for class 'character'
.process_mp_vocab(v)

Arguments

v

An object of class morphemepiece_vocabulary.

Value

A character vector of tokens for tokenization.


morphemepiece documentation built on April 16, 2022, 5:05 p.m.