Rust Spark

Open-source Rust projects categorized as Spark
Edit details
Rust datafusion SQL Big Data Arrow
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured

Top 6 Rust Spark Projects

Spark

  • risingwave

    Best-in-class stream processing, analytics, and management. Perform continuous analytics, or build event-driven applications, real-time ETL pipelines, and feature stores in minutes. Unified streaming and batch. PostgreSQL compatible.

    Project mention: RisingWave: Process, manage, and analyze event streams with Postgres-style SQL | news.ycombinator.com | 2024-07-18

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo

  • blaze

    Blazing-fast query execution engine speaks Apache Spark language and has Arrow-DataFusion at its core. (by kwai)

    Project mention: DataFusion Comet: Apache Spark Accelerator | news.ycombinator.com | 2024-05-31

    I've been keeping an eye on these kinds of Spark accelerator libraries for a while now.

    How does it compare to Blaze[1] and Gluten[2]?

    I'm interested in running some benchmarks soon against all three for my project to see how they all go.

    [1] https://github.com/kwai/blaze

  • datafusion-comet

    Apache DataFusion Comet Spark Accelerator

    Project mention: Amazon's Exabyte-Scale Migration from Apache Spark to Ray on Amazon EC2 | news.ycombinator.com | 2024-07-29

    I wonder if similar performance can be achieved with Spark accelerator like https://github.com/apache/datafusion-comet. Of course it didn’t exist before

  • sail

    LakeSail's computation framework with a mission to unify stream processing, batch processing, and compute-intensive (AI) workloads. (by lakehq)

    Project mention: AI and All Data Weekly - 02 December 2024 | dev.to | 2024-12-02

  • kamu-cli

    Next-generation decentralized data lakehouse and a multi-party stream processing network

  • databricks-kube-operator

    A Kubernetes operator to enable GitOps style deploys for Databricks resources

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Rust Spark discussion

Log in or Post with

Rust Spark related posts

  • Ask HN: Fast data structures for disjoint intervals?

    9 projects | news.ycombinator.com | 23 Jul 2024

  • DataFusion Comet: Apache Spark Accelerator

    4 projects | news.ycombinator.com | 31 May 2024

  • Apache Arrow DataFusion Comet Spark Accelerator

    1 project | news.ycombinator.com | 7 Mar 2024

  • Show HN: SQL Polyglot

    4 projects | news.ycombinator.com | 16 Dec 2023

  • Ideas/Suggestions around setting up a data pipeline from scratch

    3 projects | /r/dataengineering | 9 Jun 2023

  • biobear -- python package with minimal dependencies for bioinformatic file parsing and querying using rust and polars as the backend

    5 projects | /r/bioinformatics | 24 Apr 2023

  • Apache Iceberg-based opensource analytics stack demo

    2 projects | /r/bigdata | 6 Feb 2023
  • A note from our sponsor - SaaSHub
    www.saashub.com | 3 Dec 2024
    SaaSHub helps you find the best software and product alternatives Learn more →

Index

What are some of the best open-source Spark projects in Rust? This list will help you:

Project Stars
1 risingwave 7,086
2 blaze 1,303
3 datafusion-comet 828
4 sail 558
5 kamu-cli 305
6 databricks-kube-operator 16

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com

Did you konow that Rust is
the 5th most popular programming language
based on number of metions?