toil
wakaq
toil | wakaq | |
---|---|---|
2 | 3 | |
870 | 565 | |
0.3% | 0.5% | |
9.2 | 8.7 | |
3 days ago | 2 months ago | |
Python | Python | |
Apache License 2.0 | BSD 3-clause "New" or "Revised" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
toil
-
Show HN: Hatchet – Open-source distributed task queue
a little late now, but I wonder if https://github.com/DataBiosphere/toil might meet your requirements
- Lessons Learned from Running Apache Airflow at Scale
wakaq
- Show HN: Hatchet – Open-source distributed task queue
- Ask HN: What apps have you created for your own use?
-
Building a distributed task queue in Python
https://github.com/wakatime/wakaq/blob/main/wakaq/__init__.p...
and
https://github.com/wakatime/wakaq/blob/main/wakaq/worker.py
is the meat of it. The blog post talks about the Redis data structures used, and there's not much to it beyond that.
What are some alternatives?
common-workflow-language - Repository for the CWL standards. Use https://cwl.discourse.group/ for support 😊
rqmonitor - Flask based more dynamic and actionable frontend dashboard for monitoring Redis Queue 👩🏿💻 http://python-rq.org
luigi - Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
wakaq-ts - Background task queue for TypeScript backed by Redis, a super minimal Celery
direktiv - Serverless Container Orchestration
SSDB - SSDB - A fast NoSQL database, an alternative to Redis
windmill - Open-source developer platform to turn scripts into workflows and UIs. Fastest workflow engine (5x vs Airflow). Open-source alternative to Airplane and Retool.
conductor - Conductor is an event driven orchestration platform
states-language-cadence - States Language on Cadence
arq - Fast job queuing and RPC in python with asyncio and redis.
cronitor-airflow - Cronitor integration for Airflow
Conveyor MQ - A fast, robust and extensible distributed task/job queue for Node.js, powered by Redis.