Our great sponsors
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
Ut vero tibi verum dicam, supra rem patenter exponere volui nec mihi tum utibile duxi de numerificatione amplius disserere. Quam ob rem tu fortasse me reprehendes. Sed nolui nimis obscure loqui. Quae cum ita sint, ad ipsum generatorem docendum numeros non singulis verbis, sed singulis particulis do ita, ut ab eo, quod dixisti, caveam. Nam ad verba numerificanda modo Wordpiece, modo BPE, modo SentencePiece utor. Quibus modis in spe habeo me vinculum inter singula verba, ut puta inter "ludit" et "ludunt", in numeris defigere.
NOTE:
The number of mentions on this list indicates mentions on common posts plus user suggested alternatives.
Hence, a higher number means a more popular project.
Related posts
- sentencepiece
- [P] TokenMonster Ungreedy ~ 35% faster inference and 35% increased context-length for large language models (compared to tiktoken). Benchmarks included.
- LLaMA tokenizer: is a JavaScript implementation available anywhere?
- [P] New tokenization method improves LLM performance & context-length by 25%+
- Code runs without definition of function (automatically calls a different function instead)