Rust is showing a lot of promise in the DataFrame / tabular data space

1. polars

   1 151 34,403 10.0 Rust

   Dataframes powered by a multithreaded, vectorized query engine, written in Rust

   

   [Polars](https://github.com/pola-rs/polars) is a blazing fast DataFrame library with a beautiful user interface and an awesome getting started guide. The impressive h2o benchmark results have gotten Polars a lot of users.

2. Stream

   getstream.io featured

   Stream - Scalable APIs for Chat, Feeds, Moderation, & Video. Stream helps developers build engaging apps that scale to millions with performant and flexible Chat, Feeds, Moderation, and Video APIs and SDKs powered by a global edge network and enterprise-grade infrastructure.

   

3. db-benchmark

   2 92 337 0.0 R

   reproducible benchmark of database-like ops

   

   [Polars](https://github.com/pola-rs/polars) is a blazing fast DataFrame library with a beautiful user interface and an awesome getting started guide. The impressive h2o benchmark results have gotten Polars a lot of users.

4. datafusion

   3 66 7,434 10.0 Rust

   Apache DataFusion SQL Query Engine

   

   [arrow-datafusion](https://github.com/apache/arrow-datafusion) is another great DataFrame library, especially if you like running SQL queries. It's so easy to query a Parquet / CSV dataset with SQL using DataFusion. I've run local benchmarks and it's super fast. The DataFusion docs are a bit lacking, which is a shame, for such a developed and amazing library. I hope to make these better and help spread the world about how truly amazing this lib is.

5. arrow2

   4 25 1,071 0.0 Rust

   Discontinued Transmute-free Rust library to work with the Arrow format

   [arrow2](https://github.com/jorgecarleitao/arrow2) and [parquet2](https://github.com/jorgecarleitao/parquet2) are great foundational libraries for and DataFrame libs in Rust.

6. parquet2

   5 6 362 0.9 Rust

   Fastest and safest Rust implementation of parquet. `unsafe` free. Integration-tested against pyarrow

   

   [arrow2](https://github.com/jorgecarleitao/arrow2) and [parquet2](https://github.com/jorgecarleitao/parquet2) are great foundational libraries for and DataFrame libs in Rust.

7. delta-rs

   6 31 2,862 9.8 Rust

   A native Rust library for Delta Lake, with bindings into Python

   

   I'm working on [delta-rs](https://github.com/delta-io/delta-rs) which brings the power of Delta Lake to the Rust community. CSV / Parquet lakes are limited and Delta Lakes offer a ton of advantages (versioned data, time travel, ACID transactions, schema enforcement, etc). We're working to bring full Polars and DataFusion support to delta-rs, see the roadmap.

8. influxdb_iox

   7 14 1,803 9.9 Rust

   Discontinued Pronounced (influxdb eye-ox), short for iron oxide. This is the new core of InfluxDB written in Rust on top of Apache Arrow.

   

   Already is: https://github.com/influxdata/influxdb_iox Just still a work in progress.

9. InfluxDB

   www.influxdata.com featured

   InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.

   

10. PyO3

    8 158 14,004 9.6 Rust

    Rust bindings for the Python interpreter

    

    If you’re interested in python bindings take a look at https://pyo3.rs/

11. kafka-delta-ingest

    9 6 409 7.0 Rust

    A highly efficient daemon for streaming data from Kafka into Delta Lake

    

    kafka-delta-ingest is a good project to get streaming data into a Delta Lake. Here's a great talk on the topic.

Suggest a related project