Daft: A High-Performance Distributed Dataframe Library for Multimodal Data

• quokka

  23 1,081 8.3 Python

  Making data lake work for time series (by marsupialtail)

SQL support is very challenging.

I work on Quokka (https://github.com/marsupialtail/quokka). I support Iceberg reads. Recently we are adding SQL support from just parsing the DuckDB logical plan, though that is very challenging as well.

The Python world lacks a standard for a plug and play SQL query optimizer. Apache Calcite is good for the JVM world, but not great if you are trying to cut out the JVM.

• Daft

  7 1,684 9.8 Rust

  Distributed DataFrame for Python designed for the cloud, powered by Rust

  

Hi (one of the maintainers here), that is a good suggestion! I wasn't aware of that project. I went ahead and made an issue to add `export DO_NOT_TRACK=1` as one of the variables we track! https://github.com/Eventual-Inc/Daft/issues/1015

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• fugue

  11 1,876 6.7 Python

  A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.

  

Please integrate it with Fugue.

https://github.com/fugue-project/fugue

Suggest a related project