cld2
Compact Language Detector 2 (by CLD2Owners)
lingua-py
The most accurate natural language detection library for Python, suitable for short text and mixed-language text (by pemistahl)
| cld2 | lingua-py | |
|---|---|---|
| 3 | 8 | |
| 801 | 912 | |
| 0.0% | - | |
| 0.0 | 7.4 | |
| almost 3 years ago | about 1 month ago | |
| C++ | Python | |
| Apache License 2.0 | Apache License 2.0 |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
cld2
Posts with mentions or reviews of cld2.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2022-02-11.
-
Opensourcing Whichlang, a fast language detection library for Rust!
This looks cool, and it’s good to have a pure-Rust option available now, but I wonder, have the authors considered binding CLD2? (It looks like a crate exists for it already, but it is deprecated.) It was originally developed as the language detector for Chromium (to do things like selecting the right spell check language), optimized for speed and size. It may be a bit dated (Chromium no longer uses it), but it is small and easy to use. I’ve used it with great success in a past project.
- Lingua-Go, the most accurate language detection for Go
-
Announcing Lingua 1.0.0: The most accurate natural language detection library for Python, suitable for long and short text alike
Python is widely used in natural language processing, so there are a couple of comprehensive open source libraries for this task, such as Google's CLD 2 and CLD 3, langid and langdetect. Unfortunately, except for the last one they have two major drawbacks:
lingua-py
Posts with mentions or reviews of lingua-py.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2022-12-26.
-
Typos — automatic language recognition and error detection in Word and Excel documents
ᅠ✅ Recognition of 75 languages
-
The hand-picked selection of the best Python libraries and tools of 2022
Hi u/dekked_, perhaps you want to add my natural language detection library Lingua to the NLP section of the long tail. It is pretty unique among the natural language detection libraries for Python because it is able to detect multiple languages in mixed-language text. It is also one of the most accurate libraries when detecting the language of short text. I would very much appreciate if you added my library to your list.
- Lingua 1.2.0 - The most accurate natural language detection library for Python - now with support for detecting multiple languages in mixed-language text.
- Lingua 1.1.0 - The most accurate natural language detection library for Python
-
Lingua-Go, the most accurate language detection for Go
I've compared the Python implementation of Lingua with fasttext. Lingua performs clearly better. Look here: https://github.com/pemistahl/lingua-py#4-how-good-is-it
- Lingua 1.0.1: The most accurate natural language detection library for Python - previously for Python >= 3.9, now compatible with Python >= 3.7
- Announcing Lingua 1.0.0: The most accurate natural language detection library for Python, suitable for long and short text alike
What are some alternatives?
When comparing cld2 and lingua-py you can also consider the following projects:
cld3
lingua-rs - The most accurate natural language detection library for Rust, suitable for short text and mixed-language text
lingua-go - The most accurate natural language detection library for Go, suitable for short text and mixed-language text
langid.py - Stand-alone language identification system
langdetect - Port of Google's language-detection library to Python.
LSTM_langid - Source code for the Apple reproduction
mito - The mitosheet package, trymito.io, and other public Mito code.
whylogs - An open-source data logging library for machine learning models and data pipelines. 📚 Provides visibility into data quality & model performance over time. 🛡️ Supports privacy-preserving data collection, ensuring safety & robustness. 📈