pythainlp
nlpo3
pythainlp | nlpo3 | |
---|---|---|
2 | 1 | |
926 | 30 | |
0.1% | - | |
9.5 | 1.6 | |
4 days ago | 5 months ago | |
Python | Rust | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
pythainlp
-
PyThaiNLP 2.4.0-dev0
Read more about PyThaiNLP v2.4.0-dev0: https://github.com/PyThaiNLP/pythainlp/releases/tag/v2.4.0-dev0
-
Thai word tokenizers benchmark: nlpo3 vs newmm
Thanathip Suntorntip Gorlph ported Korakot Chaovavanich's Thai word tokenizer - Newmm, written in Python, to Rust called nlpo3. The nlpo3 website claimed that nlpo3 is 2X faster than Newmm. I felt that Nlpo3 must be faster than this claim because in contrast to Python's Regex engine, Rust's regex runs in the linear time since it was constrained not to support looking back/ahead. Moreover, 2X faster is ambiguous.
nlpo3
-
Thai word tokenizers benchmark: nlpo3 vs newmm
Thanathip Suntorntip Gorlph ported Korakot Chaovavanich's Thai word tokenizer - Newmm, written in Python, to Rust called nlpo3. The nlpo3 website claimed that nlpo3 is 2X faster than Newmm. I felt that Nlpo3 must be faster than this claim because in contrast to Python's Regex engine, Rust's regex runs in the linear time since it was constrained not to support looking back/ahead. Moreover, 2X faster is ambiguous.
What are some alternatives?
PyProjects - Beginner Friendly Python-Projects
hck - A sharp cut(1) clone.
toiro - A comparison tool of Japanese tokenizers
sd - Intuitive find & replace CLI (sed alternative)
google-play-scraper - Google play scraper for Python inspired by <facundoolano/google-play-scraper>
regex - An implementation of regular expressions for Rust. This implementation uses finite automata and guarantees linear time matching on all inputs.
abydos - Abydos NLP/IR library for Python
oso - Oso is a batteries-included framework for building authorization in your application.
uni2db - The Unified University Database (uni2db). Tools to get information about courses offered at various colleges/universities.
whatlang-rs - Natural language detection library for Rust. Try demo online: https://whatlang.org/
transformers - 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
tiktoken-rs - Ready-made tokenizer library for working with GPT and tiktoken