Daft: A High-Performance Distributed Dataframe Library for Multimodal Data

1. quokka

   1 23 1,164 2.9 Python

   Making data lake work for time series (by marsupialtail)

   

   SQL support is very challenging.

   I work on Quokka (https://github.com/marsupialtail/quokka). I support Iceberg reads. Recently we are adding SQL support from just parsing the DuckDB logical plan, though that is very challenging as well.

   The Python world lacks a standard for a plug and play SQL query optimizer. Apache Calcite is good for the JVM world, but not great if you are trying to cut out the JVM.

2. Judoscale

   judoscale.com featured

   Save 47% on cloud hosting with autoscaling that just works. Judoscale integrates with Django, FastAPI, Celery, and RQ to make autoscaling easy and reliable. Save big, and say goodbye to request timeouts and backed-up task queues.

   

3. Daft

   2 10 2,741 9.9 Rust

   Distributed data engine for Python/SQL designed for the cloud, powered by Rust

   

   Hi (one of the maintainers here), that is a good suggestion! I wasn't aware of that project. I went ahead and made an issue to add `export DO_NOT_TRACK=1` as one of the variables we track! https://github.com/Eventual-Inc/Daft/issues/1015

4. fugue

   3 11 2,072 3.4 Python

   A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.

   

   Please integrate it with Fugue.

   https://github.com/fugue-project/fugue

Suggest a related project