Datafusion-ballista Alternatives

datafusion-ballista reviews and mentions

• DataFusion Comet: Apache Spark Accelerator
  4 projects | news.ycombinator.com | 31 May 2024

  But why. Just ditch Spark and use https://github.com/apache/datafusion-ballista directly.

• Polars
  11 projects | news.ycombinator.com | 8 Jan 2024

  Not super on topic because this is all immature and not integrated with one another yet, but there is a scaled-out rust data-frames-on-arrow implementation called ballista that could maybe? form the backend of a polars scale out approach: https://github.com/apache/arrow-ballista

• Rust vs. Go in 2023
  9 projects | news.ycombinator.com | 13 Aug 2023

  > Is Rust's compile-time GC about something other than performance somehow?

  AFAIK, memory safety and language features as RAII is also available in C++, for instance. About the reasons for slow compilation, take a look at https://www.reddit.com/r/rust/comments/xna9mb/why_are_rust_p...

  Not having a GC is also about not having a runtime as you mention (e.g. nice for creating Python extensions and embedded systems programming) and also more runtime deterministic performance: on that, if I'm not mistaken that was the reason for Discourse switching to Rust and also, e.g.: "the choice of Rust as the main execution language avoids the overhead of GC pauses and results in deterministic processing times" https://github.com/apache/arrow-ballista/blob/main/README.md

• Ballista (Rust) vs Apache Spark. A Tale of Woe.
  1 project | /r/dataengineering | 7 Jul 2023
• Evolution and Trends of Data Engineering 2022/23
  1 project | /r/dataengineering | 19 May 2023

  Ballista (Arrow-Rust), which is largely inspired by Apache Spark, there are some interesting differences.

• Data Engineering with Rust
  5 projects | /r/rust | 9 May 2023

  https://github.com/jorgecarleitao/arrow2 https://github.com/apache/arrow-datafusion https://github.com/apache/arrow-ballista https://github.com/pola-rs/polars https://github.com/duckdb/duckdb

• Any job processing framework like Spark but in Rust?
  4 projects | /r/dataengineering | 23 Mar 2023
• Is Apache Arrow DataFusion and Ballista the future of big data engineering/science?
  1 project | /r/dataengineering | 11 Mar 2023

  Source: https://github.com/apache/arrow-ballista

• Pure Python Distributed SQL Engine
  9 projects | news.ycombinator.com | 30 Dec 2022

  Can you explain how this might differ from something like https://github.com/apache/arrow-ballista

  I've seen several variants of "next-gen" spark, but nowhere have I really seen the different tradeoffs/advantages/disadvantages between them.

• Scala or Rust? which one will rule in future?
  4 projects | /r/dataengineering | 23 Dec 2022
• A note from our sponsor - SaaSHub
  www.saashub.com | 9 Oct 2024

  SaaSHub helps you find the best software and product alternatives Learn more →