lingua-py VS fugue

ᅠ✅ Recognition of 75 languages

Hi u/dekked_, perhaps you want to add my natural language detection library Lingua to the NLP section of the long tail. It is pretty unique among the natural language detection libraries for Python because it is able to detect multiple languages in mixed-language text. It is also one of the most accurate libraries when detecting the language of short text. I would very much appreciate if you added my library to your list.

I've compared the Python implementation of Lingua with fasttext. Lingua performs clearly better. Look here: https://github.com/pemistahl/lingua-py#4-how-good-is-it

Please integrate it with Fugue.

https://github.com/fugue-project/fugue

Fugue is an interesting library in this space , though I haven’t tried it

https://github.com/fugue-project/fugue

A unified interface for distributed computing. Fugue executes SQL, Python, and Pandas code on Spark, Dask and Ray without any rewrites.

fugue — distributed computing done easy

This looks similar to fugue, am I right? How do they compare?

I am looking forward to how Substrait could help removing this friction. It aims to provide a standardised intermediate query language (lower level than SQL) to connect frontend user interfaces like SQL or data frame libraries with backend analytical computing engines. It is linked to the Arrow ecosystem. Something like Ibis or Fugue could become the front and DuckDB the backend engine.

There's dask-sql, but I think it is being abandoned for fugue-project. I'm actually excited for this project as it is trying to provide a backend agnostic solution, which would seem like a difficult, lofty goal. I wish them luck.