incubator-gluten
velox
| incubator-gluten | velox | |
|---|---|---|
| 3 | 6 | |
| 1,002 | 3,158 | |
| 3.0% | 2.0% | |
| 9.9 | 10.0 | |
| about 6 hours ago | about 16 hours ago | |
| Scala | C++ | |
| Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
incubator-gluten
-
A glimpse into the future of data processing infrastructure.
When I first learned about the Gluten project from Intel, I thought Databricks was going to be in trouble.
- FLaNK Stack for 04 December 2023
-
Blaze: Fast query execution engine for Apache Spark
Interesting, looks like it is just DataFusion engine for Spark. There is a similar project: https://github.com/oap-project/gluten - it brings ClickHouse as an engine to Spark.
velox
- Meta Velox: A C++ vectorized database acceleration library
- FLaNK Stack for 04 December 2023
- A C++ vectorized database acceleration library
- Velox: An open source unified execution engine
- A new C++ vectorized DB acceleration library, optimizing query engines
-
Substrait: Cross-Language Serialization for Relational Algebra
4. The workers (if distributed) or the engine (if single-box) executing the execution plan, by performing the actual computations on actual bytes.
(Note that this is a cartoon version and any given engine will have differences, eg Presto/Trino does not have a clear distinction between LP and Physical Plan.)
1) and 4) are broadly similar across engines, while 2) and 3) vary widely partially because of different requirements (reliability, latency, etc). Projects such as Apache Arrow and Velox (https://github.com/facebookincubator/velox) are making common tools for 4), and as mentioned ANSI SQL, ZetaSQL, Calcite, and Substrait are making common tools for 1).
What are some alternatives?
LearningSparkV2 - This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]
substrait - A cross platform way to express data transformation, relational algebra, standardized record expression and plans.
opaque-sql - An encrypted data analytics platform
dwl - dwm for Wayland - ARCHIVE: development has moved to Codeberg
blaze - Blazing-fast query execution engine speaks Apache Spark language and has Arrow-DataFusion at its core.
wayland - Core Wayland protocol and libraries (mirror)
blaze - NumPy and Pandas interface to Big Data
swc - a library for making a simple Wayland compositor
Jupyter Scala - A Scala kernel for Jupyter
dwm - LEV Linux's window manager (a fork of dwm)
kyuubi - Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.
wlr-randr - An xrandr clone for wlroots compositors