Files in tokenizers.bpe
Byte Pair Encoding Text Tokenization

MD5
LICENSE.note
DESCRIPTION
README.md
NAMESPACE
inst/extdata/youtokentome.bpe
R/data.R R/RcppExports.R R/youtokentome.R R/pkg.R
src/rcpp_youtokentome.cpp
src/RcppExports.cpp
src/Makevars
src/parallel_hashmap/phmap_fwd_decl.h
src/parallel_hashmap/phmap.h
src/parallel_hashmap/meminfo.h
src/parallel_hashmap/phmap_utils.h
src/parallel_hashmap/phmap_config.h
src/parallel_hashmap/phmap_bits.h
src/parallel_hashmap/phmap_base.h
src/youtokentome/cpp/bpe.h
src/youtokentome/cpp/utils.cpp
src/youtokentome/cpp/utils.h
src/youtokentome/cpp/bpe.cpp
src/youtokentome/cpp/utf8.cpp
src/youtokentome/cpp/utf8.h
man/bpe_encode.Rd man/bpe_decode.Rd man/bpe_load_model.Rd man/belgium_parliament.Rd man/bpe.Rd
data/belgium_parliament.RData
tokenizers.bpe documentation built on Aug. 2, 2019, 5:05 p.m.