Quack-reduce Alternatives
Similar projects and alternatives to quack-reduce based on common topics and language
-
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
quack-reduce reviews and mentions
- quack-reduce: duckdb as a stateless query engine over a data lake
-
Summing columns in remote Parquet files using DuckDB
We can run a DuckDb instance (EC2/S3) closer to the data so that sorta helps too.
What I'm really excited about using DuckDB in a similar way to map-reduce. What if there was a way to take some SQL's logical plan and turn it into a physical plan that uses compute resources from a pool serverless DuckDB instances. Starting at the leafs of the graph (physical plan) pulling data from the source (parquet), and returning their completed work up the branches, until it is completed and ready to be used as the results.
I've seen a few examples of this already, but nothing that I would consider production ready. I have a hunch that someone is going to drop such a project on us shortly, and it's going to change a lot of things we have become use to in the data world.
https://github.com/BauplanLabs/quack-reduce
Stats
BauplanLabs/quack-reduce is an open source project licensed under MIT License which is an OSI approved license.
The primary programming language of quack-reduce is Python.
Popular Comparisons
Sponsored