Man pages for wordpiece
R Implementation of Wordpiece Tokenization
| dot-get_casedness | Determine Casedness of Vocabulary |
| dot-infer_case_from_vocab | Determine Vocabulary Casedness |
| dot-new_wordpiece_vocabulary | Constructor for Class wordpiece_vocabulary |
| dot-process_vocab | Process a Vocabulary for Tokenization |
| dot-process_wp_vocab | Process a Wordpiece Vocabulary for Tokenization |
| dot-validate_wordpiece_vocabulary | Validator for Objects of Class wordpiece_vocabulary |
| dot-wp_tokenize_single_string | Tokenize an Input Word-by-word |
| dot-wp_tokenize_word | Tokenize a Word |
| load_or_retrieve_vocab | Load a vocabulary file, or retrieve from cache |
| load_vocab | Load a vocabulary file |
| prepare_vocab | Format a Token List as a Vocabulary |
| reexports | Objects exported from other packages |
| set_wordpiece_cache_dir | Set a Cache Directory for wordpiece |
| wordpiece_cache_dir | Retrieve Directory for wordpiece Cache |
| wordpiece_tokenize | Tokenize Sequence with Word Pieces |
