cld2
langid.py
| cld2 | langid.py | |
|---|---|---|
| 3 | 2 | |
| 801 | 2,242 | |
| 0.0% | - | |
| 0.0 | 0.0 | |
| almost 3 years ago | over 4 years ago | |
| C++ | Python | |
| Apache License 2.0 | BSD 3-clause "New" or "Revised" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
cld2
-
Opensourcing Whichlang, a fast language detection library for Rust!
This looks cool, and it’s good to have a pure-Rust option available now, but I wonder, have the authors considered binding CLD2? (It looks like a crate exists for it already, but it is deprecated.) It was originally developed as the language detector for Chromium (to do things like selecting the right spell check language), optimized for speed and size. It may be a bit dated (Chromium no longer uses it), but it is small and easy to use. I’ve used it with great success in a past project.
- Lingua-Go, the most accurate language detection for Go
-
Announcing Lingua 1.0.0: The most accurate natural language detection library for Python, suitable for long and short text alike
Python is widely used in natural language processing, so there are a couple of comprehensive open source libraries for this task, such as Google's CLD 2 and CLD 3, langid and langdetect. Unfortunately, except for the last one they have two major drawbacks:
langid.py
-
Curator v0.1.0: Auto-organize large movie collections (AI language detection+sync)
Right now it's in early stages: It can detect languages from audio and subtitles (Whisper+LangID) with good results so far tried with 52 movies here (failed with just 1 which was silent). I'm currently working on synchronization: Hopefully subtitle timestamps and audio sound effects can suffice for cross-correlation. After that, I'll work on the TUI (maybe add a proper GUI too) to improve UX.
-
Announcing Lingua 1.0.0: The most accurate natural language detection library for Python, suitable for long and short text alike
Python is widely used in natural language processing, so there are a couple of comprehensive open source libraries for this task, such as Google's CLD 2 and CLD 3, langid and langdetect. Unfortunately, except for the last one they have two major drawbacks:
What are some alternatives?
cld3
polyglot - Multilingual text (NLP) processing toolkit
lingua-py - The most accurate natural language detection library for Python, suitable for short text and mixed-language text
TextBlob - Simple, Pythonic, text processing--Sentiment analysis, part-of-speech tagging, noun phrase extraction, translation, and more.
lingua-go - The most accurate natural language detection library for Go, suitable for short text and mixed-language text
py3langid - Faster, modernized fork of the language identification tool langid.py
langdetect - Port of Google's language-detection library to Python.
spaCy - 💫 Industrial-strength Natural Language Processing (NLP) in Python
LSTM_langid - Source code for the Apple reproduction
NLTK - NLTK Source
stanfordnlp - [Deprecated] This library has been renamed to "Stanza". Latest development at: https://github.com/stanfordnlp/stanza
Jieba - 结巴中文分词