SaaSHub helps you find the best software and product alternatives Learn more →
Top 17 Rust Arrow Projects
-
If you want to get into Polars, the library is very well documented, and I’d recommend you check out their getting started tutorial, their API docs, and when you’re all set up, you can also check out their Cookbooks to learn about many of the standard operations within Polars.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
-
Project mention: Show HN: Turn CSS files into high performance APIs | news.ycombinator.com | 2025-01-11
-
But why. Just ditch Spark and use https://github.com/apache/datafusion-ballista directly.
-
Project mention: Show HN: Vortex – a high-performance columnar file format in Rust | news.ycombinator.com | 2024-10-14
We have a TableProvider for use with Datafusion, checkout this crate and its examples: https://github.com/spiraldb/vortex/tree/develop/vortex-dataf...
-
Project mention: Amazon's Exabyte-Scale Migration from Apache Spark to Ray on Amazon EC2 | news.ycombinator.com | 2024-07-29
I wonder if similar performance can be achieved with Spark accelerator like https://github.com/apache/datafusion-comet. Of course it didn’t exist before
-
Project mention: Show HN: TonboLite – Scale SQLite with S3, Minimize ETL | news.ycombinator.com | 2025-01-07
Hi! I am Tzu and the team from Tonbo here.
TonboLite: https://github.com/tonbo-io/tonbolite is a SQLite extension based on Tonbo: https://github.com/tonbo-io/tonbo. It enables SQLite to create tables suitable for analytical processing on target platforms like WebAssembly in browser and efficiently write data. The data in the tables is organized as tiered Apache Parquet format files, stored on demand either on local disks (using OPFS as native I/O) or object storage services (such as S3). You can use it by creating virtual tables in regular SQLite.
TonboLite started with the exploration of Tonbo application. The goal of Tonbo is to write data for analytical processing (like log processing, metrics monitoring, or text search) to unlimited remote storage in SQLite and PostgreSQL.
We tried SQLite as it is the most popular transactional database for the edge. One of the most requested improvements for SQLite is better support for append-only writes (e.g., logs, time-series data), which are common in analytical data. Append-only writes present two main challenges for SQLite:
-
sail
LakeSail's computation framework with a mission to unify stream processing, batch processing, and compute-intensive (AI) workloads. (by lakehq)
-
-
-
-
-
-
-
-
-
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Rust Arrow discussion
Rust Arrow related posts
-
Apache DataFusion
-
Show HN: Turn CSS files into high performance APIs
-
Show HN: TonboLite – Scale SQLite with S3, Minimize ETL
-
Using Polars in Rust for high-performance data analysis
-
Pg_analytics is now PostgreSQL Licensed
-
ParadeDB pg_analytics is now PostgreSQL License
-
How to Store Vibration Sensor Data | ReductStore vs InfluxDB
-
A note from our sponsor - SaaSHub
www.saashub.com | 15 Jan 2025
Index
What are some of the best open-source Arrow projects in Rust? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | polars | 31,328 |
| 2 | datafusion | 6,559 |
| 3 | roapi | 3,249 |
| 4 | datafusion-ballista | 1,608 |
| 5 | vortex | 1,065 |
| 6 | datafusion-comet | 865 |
| 7 | tonbo | 854 |
| 8 | sail | 598 |
| 9 | duckdb-rs | 539 |
| 10 | parquet-wasm | 533 |
| 11 | pqrs | 301 |
| 12 | biobear | 166 |
| 13 | iceberg-rust | 132 |
| 14 | fastexcel | 129 |
| 15 | datafusion-dft | 126 |
| 16 | s2protocol-rs | 103 |
| 17 | myval | 63 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com