API for sentencepiece
Text Tokenization using Byte Pair Encoding and Unigram Modelling
| Global functions | |
|---|---|
| BPEembed | Man page Source code |
| BPEembedder | Man page Source code |
| predict.BPEembed | Man page Source code |
| print.BPEembed | Source code |
| print.sentencepiece | Source code |
| read_word2vec | Man page Source code |
| sentencepiece | Man page Source code |
| sentencepiece_decode | Man page Source code |
| sentencepiece_download_model | Man page Source code |
| sentencepiece_encode | Man page Source code |
| sentencepiece_load_model | Man page Source code |
| spc_decode_ids | Source code |
| spc_decode_subwords | Source code |
| spc_encode_as_ids | Source code |
| spc_encode_as_ids_nbest | Source code |
| spc_encode_as_ids_sample | Source code |
| spc_encode_as_subwords | Source code |
| spc_encode_as_subwords_nbest | Source code |
| spc_encode_as_subwords_sample | Source code |
| spc_load_model | Source code |
| spc_train | Source code |
| txt_remove_ | Man page Source code |
| wordpiece_encode | Man page Source code |
| wordpiece_encode_as_subwords | Source code |
