Rust Arrow

Open-source Rust projects categorized as Arrow

Top 9 Rust Arrow Projects

  • polars

    Fast multi-threaded, hybrid-out-of-core query engine focussing on DataFrame front-ends

    Project mention: Polars | /r/programming | 2023-08-30
  • arrow-datafusion

    Apache Arrow DataFusion SQL Query Engine

    Project mention: GlareDB: An open source SQL database to query and analyze distributed data | /r/dataengineering | 2023-06-08

    Apache Arrow is a pretty common memory structure these days. Datafusion is an open query engine built in Rust started by Andy Grove.

  • SonarQube

    Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.

  • roapi

    Create full-fledged APIs for slowly moving datasets without writing a single line of code.

    Project mention: Tuql: Automatically create a GraphQL server from a SQLite database | news.ycombinator.com | 2023-04-25

    If your use case is read-only I suggest taking a look at roapi[1]. It supports multiple read frontends (GraphQL, SQL, REST) and many backends like SQLite, JSON, google sheets, MySQL, etc.

    [1] https://github.com/roapi/roapi

  • arrow-ballista

    Apache Arrow Ballista Distributed Query Engine

    Project mention: Rust vs. Go in 2023 | news.ycombinator.com | 2023-08-13

    > Is Rust's compile-time GC about something other than performance somehow?

    AFAIK, memory safety and language features as RAII is also available in C++, for instance. About the reasons for slow compilation, take a look at https://www.reddit.com/r/rust/comments/xna9mb/why_are_rust_p...

    Not having a GC is also about not having a runtime as you mention (e.g. nice for creating Python extensions and embedded systems programming) and also more runtime deterministic performance: on that, if I'm not mistaken that was the reason for Discourse switching to Rust and also, e.g.: "the choice of Rust as the main execution language avoids the overhead of GC pauses and results in deterministic processing times" https://github.com/apache/arrow-ballista/blob/main/README.md

  • duckdb-rs

    Ergonomic bindings to duckdb for Rust

  • pqrs

    Command line tool for inspecting Parquet files

    Project mention: Querying Postgres Tables Directly from DuckDB | news.ycombinator.com | 2022-09-30

    This won't let you ETL from PG to parquet, but I used this in anger the other day https://github.com/manojkarthick/pqrs Worked quite well for my purposes!

  • parquet-wasm

    Rust-based WebAssembly bindings to read and write Apache Parquet data

  • InfluxDB

    Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.

  • myval

    Lightweight Apache Arrow data frame for Rust

    Project mention: myval - lightweight Apache Arrow data frame for Rust | /r/rust | 2023-04-21
  • skar

    Ethereum (EVM) indexer using Apache Arrow/Parquet

    Project mention: Looking for co-founder for a blockchain data indexing project | news.ycombinator.com | 2023-09-05
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2023-09-05.

Rust Arrow related posts

Index

What are some of the best open-source Arrow projects in Rust? This list will help you:

Project Stars
1 polars 20,406
2 arrow-datafusion 4,016
3 roapi 2,907
4 arrow-ballista 923
5 duckdb-rs 261
6 pqrs 192
7 parquet-wasm 175
8 myval 57
9 skar 14
Updating dependencies is time-consuming.
Solutions like Dependabot or Renovate update but don't merge dependencies. You need to do it manually while it could be fully automated! Add a Merge Queue to your workflow and stop caring about PR management & merging. Try Mergify for free.
blog.mergify.com