awesome-workflow-engines
Prefect
| awesome-workflow-engines | Prefect | |
|---|---|---|
| 11 | 19 | |
| 5,588 | 14,724 | |
| - | 2.3% | |
| 5.8 | 10.0 | |
| 11 days ago | about 24 hours ago | |
| Java | Python | |
| Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
awesome-workflow-engines
-
Looking for an orchestration engine for HTTP requests similar to “Postman flows” I can selfhost.
I've come across this list, but I'm uncertain about which option would suit my requirements without going through a trial-and-error process. Do you have any recommendations for a tool similar to "Postman flows" that enables the creation of chained HTTP requests and allows for response analysis?
-
Building a distributed workflow engine from scratch
True, there are many options out there. And we looked at a good number before we made the decision to build one ourselves. But at least at the time (circa 2014), many of the existing options were either not designed for a distributed environment, were designed more particularly for data-processing use cases, were seemingly abandoned, or simply felt over-engineered to our taste.
- Building some “marketing automation” esque features into a CRM. Am I looking for a rules engine?
-
Dabbling with Dagster vs. Airflow
I'd say give Temporal (https://temporal.io) a look, but there are a lot of options (https://github.com/meirwah/awesome-workflow-engines).
- Are there any good resources for building data pipelines?
- Any bpmn engine out there?
-
CSV File automation
I believe the type of software you're looking for is a workflow engine. I've found this list of them, perhaps there's some in the list that could work for your needs: https://github.com/meirwah/awesome-workflow-engines
- Looking for genuine feedback on if my idea is good or not!
-
Looking for a tutorial to develop a workflow service
https://github.com/meirwah/awesome-workflow-engines links to a bunch of open source workflow engines. If you don’t find a tutorial, maybe try one of those and see if the code is small enough to read easily.
-
State machines are wonderful tools
[2] https://github.com/meirwah/awesome-workflow-engines
Prefect
- Prefect: A workflow orchestration tool for data pipelines
- self hosted Alternative to easycron.com?
-
Example typescript project repos?
If I was answering this question but for python, I'd recommend something like prefect, boto3, or tortoise-orm -- not extremely complex and with a pretty comprehensible featureset.
-
I have developed a simple Task Orchestrator
However, if you are looking for something like this, but much more mature and something of a bloat to be frank, there's Prefect. Honestly, woflo borrows a lot from Prefect conceptually.
-
Dabbling with Dagster vs. Airflow
Disclaimer: I work for Prefect.
It looks like we added cron and other schedule types to the deployment CLI just under a month ago[1].
Over the last couple of releases, we've also made it easier to pull deployments from GitHub or bake your flow code into Docker images instead of needing S3-like storage.
As with any product, there's always more to do, so I appreciate you sharing your thoughts. More than anywhere else I've worked, community feedback is a huge driver of product enhancements and feature development. Feel free to join our Slack community[2] if you'd like to share more feedback or ask questions.
[1] https://github.com/PrefectHQ/prefect/blob/main/RELEASE-NOTES...
- Prefect - The easiest way to automate your data
- Ask HN: Codebases with great, easy to read code?
-
Prefect CLI Action
GitHub Action for running Prefect commands using the Prefect CLI.
- Perfect – Data workflow automation with Python
What are some alternatives?
xstate - Actor-based state management & orchestration for complex app logic.
dagster - An orchestration platform for the development, production, and observation of data assets.
RE2 - RE2 is a fast, safe, thread-friendly alternative to backtracking regular expression engines like those used in PCRE, Perl, and Python. It is a C++ library.
APScheduler - Task scheduling library for Python
common-workflow-language - Repository for the CWL standards. Use https://cwl.discourse.group/ for support 😊
airbyte - The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
processus - A simple lightweight nodejs workflow engine designed to help orchestrate multiple tasks.
schedule - Python job scheduling for humans.
tork-web - Web UI for Tork Workflow Engine
doit - task management & automation tool
proposals - Temporal proposals
django-schedule - A calendaring app for Django. It is now stable, Please feel free to use it now. Active development has been taken over by bartekgorny.