flowbase
PurefunctionPipelineDataflow
Our great sponsors
flowbase | PurefunctionPipelineDataflow | |
---|---|---|
3 | 172 | |
161 | 438 | |
1.9% | - | |
0.0 | 7.4 | |
almost 2 years ago | 29 days ago | |
Go | ||
MIT License | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
flowbase
-
The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software (2005)
I see a lot of potential in pipeline concurrency, as seen in dataflow (DF) and flow-based programming (FBP). That is, modeling computation as pipelines where one component sends data to the next component via message passing. As long as there is enough data it will be possible for multiple components in the chain to work concurrently.
The benefits are that no other synchronization is needed than the data sent between processes, and race conditions are ruled out as long as only one process is allowed to process a data item at a time (this is the rule in FBP).
The main blockers I think is that it requires quite a rethink of the architecture of software. I see this rethink happening in larger, especially distributed systems, which are modeled a lot around these principles already, using systems such as Kafka and message queues to communicate, which more or less forces people to model computations around the data flow.
I think the same could happen inside monolithic applications too, with the right tooling. The concurrency primitives in Go are superbly suited to this in my experience, given that you work with the right paradigm, which I've been writing about before [1, 2], and started making a micro-unframework for [3] (though the latter one will be possible to make so much nicer after we get generics in Go).
But then, I also think there are some lessons to be learned about the right granularity for processes and data in the pipeline. Due to the overhead of message passing, it will not make sense performance-wise to use dataflow for the very finest-grain data.
Perhaps this in a sense parallels what we see with distributed computing, where there is a certain breaking point before which it isn't really worth it to go with distributed computing, because of all the overhead, both performance-wise and complexity-wise.
[1] https://blog.gopheracademy.com/composable-pipelines-pattern/
[2] https://blog.gopheracademy.com/advent-2015/composable-pipeli...
PurefunctionPipelineDataflow
-
How can I learn functional programming?
The Math-based Grand Unified Programming Theory: The Pure Function Pipeline Data Flow with Principle-based Warehouse/Workshop Model
- M1 Ultra's architecture is a mistake, and Why Apple is not the creator of the M1 architecture?
-
M1 Ultra is a mistake
ThrudTheBarbarian's commend full screenshot
-
Some thoughts about Clojure, its latent potential and adoption
The Math-based Grand Unified Programming Theory: The Pure Function Pipeline Data Flow with Principle-based Warehouse/Workshop Model
-
Microsoft's AI computing platform Singularity's architecture adopts my Warehouse/Workshop Model
2021-06-14 Prediction: Intel will use "RISC-V plus x86 compatibility layer" or "RISC-V plus x86 heterogeneous computing architecture" to develop a new generation of "warehouse/workshop model" CPU
-
New SOC chip architecture: Math&Principle-based Warehouse/Workshop Model
2021-06-14, Prediction: Intel will use "RISC-V plus x86 compatibility layer" or "RISC-V plus x86 heterogeneous computing architecture" to develop a new generation of "warehouse/workshop model" CPU
-
Self-taught programmer: What books, Youtube Videos, Documetaries, Free Courses and other free resources should someone use to get a good backbone, structure and knowledge in CS?
The Math-based Grand Unified Programming Theory: The Pure Function Pipeline Data Flow with Principle-based Warehouse/Workshop Model
What are some alternatives?
concurrencpp - Modern concurrency for C++. Tasks, executors, timers and C++20 coroutines to rule them all
gophernotes - The Go kernel for Jupyter notebooks and nteract.
verona - Research programming language for concurrent ownership
chromium - The official GitHub mirror of the Chromium source
refterm - Reference monospace terminal renderer
BetterDummy - Unlock your displays on your Mac! Smooth scaling, HiDPI unlock, XDR/HDR extra brightness upscale, DDC, brightness and dimming, dummy displays, PIP and lots more! [Moved to: https://github.com/waydabber/BetterDisplay]
clojurust - A proof of concept version of Clojure in Rust.
rss-proxy - RSS-proxy allows you to do create an RSS or ATOM feed of almost any website, just by analyzing just the static HTML structure.
fa - Lin Pengcheng Financial Analyser Homepage (林鹏程财务分析软件)
Lark - Lark is a parsing toolkit for Python, built with a focus on ergonomics, performance and modularity.
penpot - Penpot - The Open-Source design & prototyping platform