sentencepiece
Unsupervised text tokenizer and detokenizer
Description
SentencePiece is an unsupervised text tokenizer/detokenizer mainly designed for Neural Network-based text generation systems where the vocabulary size is predetermined prior to the neural model training.Upload more screenshots
Please help extend the collection of screenshots. Just make a screenshot and upload it here. You don't need to register or anything.
Upload a screenshotHint: upload an image here from your clipboard with Ctrl-V
Homepage
https://github.com/google/sentencepiece
Install this software package
If the package is available for the distribution you are currently using on your computer then install the software by clicking on…
Install sentencepiece