tokenize_to_df: Create tokenizing data.frame using Sudachi

In sudachir: R Interface to 'Sudachi'

Description

Create tokenizing data.frame using Sudachi

Usage

tokenize_to_df(x, mode, instance = NULL)

Arguments

x	Input text vectors
mode	Select split mode (A, B, C)
instance	This is optional if you already have an instance of <sudachipy.tokenizer.Tokenizer> Giving them a predefined instance will speed up their execution.

Examples

## Not run: 
tokenizer("Tokyo, Japan", mode = "A")

## End(Not run)

sudachir documentation built on Nov. 10, 2020, 3:48 p.m.