sudachir-defunct: 'Sudachi' tokenizer

sudachir-defunctR Documentation

'Sudachi' tokenizer

Description

The old tokenizer() function was removed.

Usage

tokenizer(...)

Arguments

...

Not used.

Details

In general, users should not directly touch the <sudachipy.tokenizer.Tokenizer> and its MorphemeList objects. If you must access those objects, use the return value of the rebuild_tokenizer() function.


uribo/sudachir documentation built on Feb. 7, 2023, 11:09 a.m.