Scout APM is great for developers who want to find and fix performance issues in their applications. With Scout, we'll take care of the bugs so you can focus on building great things 🚀. Learn more →
Top 23 Dataflow Open-Source Projects
Kestra is an infinitely scalable orchestration and scheduling platform, creating, running, scheduling, and monitoring millions of complex pipelines.
Simple flow library 🖥️🖱️Project mention: Improving drag'n'drop | reddit.com/r/vuejs | 2022-08-03
I also found LeaderLine and a more complex one, the DrawFlow. The Drawflow one will hard, as the project is on prod and we didn't used canvas. We drag'n'drop on grid and flex lists.
Clean code begins in your IDE with SonarLint. Up your coding game and discover issues early. SonarLint is a free plugin that helps you find & fix bugs and security issues from the moment you start writing code. Install from your favorite IDE marketplace today.
⛱ Broadly scoped ecosystem & mono-repository of 168+ TypeScript projects for functional, data driven developmentProject mention: Image from my current generative art project "Harmonium" | reddit.com/r/generative | 2022-06-23
OP here. This project is implemented in Typescript and Svelte, with support from the thi.ng libraries.
A Scala API for Apache Beam and Google Cloud Dataflow.Project mention: For the DE's that choose Java over Python in new projects, why? | reddit.com/r/dataengineering | 2022-06-02
I doubt it is possible because I suspect that GIL would like a word. So I could spend nights trying to make it work in Python (and possibly, if not likely, fail). Or I could just use this ready made solution.
A Static Analysis Tool for Detecting Security Vulnerabilities in Python Web Applications
A DSL for data-driven computational pipelinesProject mention: Nextflow vs Snakemake | reddit.com/r/bioinformatics | 2022-07-29
We could spend the day pointing to things we wish were different, but that doesn't change the fact that Nextflow is the leader when it comes to workflow orchestration. And feel free to create a new issue in the GitHub repository if you wish to request a feature :)
Flowistry is an IDE plugin for Rust that helps you focus on relevant code.Project mention: flowistry plugin? | reddit.com/r/neovim | 2022-09-10
Download talent.io’s Tech Salary Report. Median salaries, most in-demand technologies, state of the remote work... all you need to know your worth on the market by tech recruitment platform talent.io
The RaftLib C++ library, streaming/dataflow concurrency via C++ iostream-like operators
Dataflow programming and declarative concurrencyProject mention: Haskell Libraries I Love | reddit.com/r/haskell | 2022-05-30
I want to like streamly, but the API is so huge, yet I feel like I'm doing things on a too low level of abstraction. (And as long as it needs a ghc plugin I doubt it'll become the de facto standard.) Though maybe I just haven't used it enough. It does have great docs at https://streamly.composewell.com/ and they seem to be taking both performance, dependency weight and API design quite seriously.
Workflows and interfaces for neuroimaging packages
A Pythonic framework for threat modelingProject mention: Pytm | reddit.com/r/devopspro | 2022-03-09
Graph / node editor in the browser using VueJS
CRUD apps as a functionProject mention: [Research] Only 7% of web developers would use no-code/low-code tools to start web applications in 2022 | reddit.com/r/programming | 2022-06-16
You might want to have a look at hyperfiddle
The Pothos data-flow framework
A Flow-based Programming inspired micro-framework / un-framework for Go (Golang)
Functional relational programming for Clojure(Script).Project mention: ANN: relic - functional relational database and military grade anti-tar library. | reddit.com/r/Clojure | 2022-01-17
I have recently cut the first alpha release of relic that I'm happy to share: https://github.com/wotbrew/relic.
A lightweight (serverless) native python parallel processing framework based on simple decorators and call graphs.
Cross-platform .NET library for computations whose arguments and results are objects that implement INotifyPropertyChanged and INotifyCollectionChanged (ObservableCollection) interfaces.Project mention: The only .NET open-source project at HighLoad++ conference | reddit.com/r/dotnet | 2022-02-21
I am taking part in the contest of open-source projects. The prize is the opportunity to become a speaker at HighLoad++ conference in March this year in Moscow. The winner can present his open-source project. My project (ObservableComputations) is the only .NET at that contest. If you consider my project is best, please, vote for it. Otherwise, vote for another open-source project. Following is the link to vote (Facebook authorization is available):
Flowgraph package for scalable asynchronous system development (by vectaport)
DFiant: A Dataflow Hardware Descripition LanguageProject mention: FCCM'22 Tutorial: Recent Developments in Hardware Description Languages | reddit.com/r/FPGA | 2022-04-05
Dataflow based workflow framework
A set of libraries for rapidly developing Pipeline driven micro/macroservices. (by houseofcat)Project mention: Best way to process large amount of Tasks? | reddit.com/r/csharp | 2022-08-19
If you need more advanced stuff check out my Dataflows. https://github.com/houseofcat/tesseract
Code examples showing flow deployment to various types of infrastructureProject mention: [D] Should I go with Prefect, Argo or Flyte for Model Training and ML workflow orchestration? | reddit.com/r/MachineLearning | 2022-09-26
Have you used infrastructure blocks in Prefect? You could easily build a block for Sagemaker deploying infrastructure for the flow running with GPUs, then run other flow in a local process, yet another one as Kubernetes job, Docker container, ECS task, AWS batch, etc. Super easy to set up, even from the UI or from CI/CD. There are a bunch of templates and examples here: https://github.com/anna-geller/prefect-deployment-patterns
Truly a developer’s best friend. Scout APM is great for developers who want to find and fix performance issues in their applications. With Scout, we'll take care of the bugs so you can focus on building great things 🚀.
Dataflow related posts
1 project | reddit.com/r/neovim | 10 Sep 2022
Ask HN: Open-source with Kafka as dependencies, is this a instant turn off?
1 project | news.ycombinator.com | 5 May 2022
Datalog for HTTP APIs
3 projects | reddit.com/r/Clojure | 27 Apr 2022
ELT vs ETL: Why not both?
2 projects | reddit.com/r/dataengineering | 27 Apr 2022
Open source data orchestration and scheduling platform
2 projects | news.ycombinator.com | 1 Apr 2022
Show HN: Kestra - Open-Source Airflow Alternative
1 project | reddit.com/r/patient_hackernews | 24 Mar 2022
Show HN: Kestra - Open-Source Airflow Alternative
1 project | reddit.com/r/hackernews | 24 Mar 2022
A note from our sponsor - Scout APM
scoutapm.com | 28 Sep 2022
What are some of the best open-source Dataflow projects? This list will help you:
Are you hiring? Post a new remote job listing for free.