jonathanbratt/wordpiece: R Implementation of Wordpiece Tokenization
Apply 'Wordpiece' (<arXiv:1609.08144>) tokenization to input text, given an appropriate vocabulary. The 'BERT' (<arXiv:1810.04805>) tokenization conventions are used by default.
Getting started
Browse package contents
Package details |
|
|---|---|
| Maintainer | |
| License | Apache License (>= 2) |
| Version | 2.1.3 |
| URL | https://github.com/macmillancontentscience/wordpiece |
| Package repository | View on GitHub |
| Installation |
Install the latest version of this package by entering the following in R:
|
