DataFusion Comet: Apache Spark Accelerator

1. datafusion-comet

   1 6 914 9.7 Rust

   Apache DataFusion Comet Spark Accelerator

   

2. CodeRabbit

   coderabbit.ai featured

   CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

   

3. datafusion-ballista

   2 13 1,684 9.0 Rust

   Apache DataFusion Ballista Distributed Query Engine

   

   But why. Just ditch Spark and use https://github.com/apache/datafusion-ballista directly.

4. blaze

   3 9 1,428 9.6 Rust

   Blazing-fast query execution engine speaks Apache Spark language and has Arrow-DataFusion at its core. (by kwai)

   

   I've been keeping an eye on these kinds of Spark accelerator libraries for a while now.

   How does it compare to Blaze[1] and Gluten[2]?

   I'm interested in running some benchmarks soon against all three for my project to see how they all go.

   [1] https://github.com/kwai/blaze

5. incubator-gluten

   4 4 1,290 9.9 Scala

   Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.

   

Suggest a related project