Top 15 task-scheduler Open-Source Projects
Distributed Task Queue (development branch)Project mention: [P] Opyrator - Turn python functions into microservices with auto-generated HTTP API, interactive UI, and more. | reddit.com/r/MachineLearning | 2021-04-22
That's a good point. What we have on the roadmap is that you can deploy it in a task queue mode (as an alternative option to the synchronous deployment). This still provides the same web API and UI, but the actual execution is happening within a background task, most likely using something like celery https://github.com/celery/celery
Apache DolphinScheduler is a distributed and extensible workflow scheduler platform with powerful DAG visual interfaces, dedicated to solving complex job dependencies in the data pipeline and providing various types of jobs available `out of the box`.Project mention: The Apache Software Foundation Announces Apache® DolphinScheduler™ as a Top-Level Project | reddit.com/r/u_mrlifeng | 2021-04-11
Apache DolphinScheduler software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project’s day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache DolphinScheduler, visit https://DolphinScheduler.apache.org/ , https://twitter.com/DolphinSchedule , and https://asf-dolphinscheduler.slack.com/ .
Scout APM - Leading-edge performance monitoring starting at $39/month. Scout APM uses tracing logic that ties bottlenecks to source code so you know the exact line of code causing performance issues and can get back to building a great product faster.
Machinery is an asynchronous task queue/job queue based on distributed message passing.Project mention: Exploring Redis High Availability | dev.to | 2021-04-22
Such a deployment would improve task and queue-based systems like Celery in python or Machinery in go. Payloads in these systems tend to be small, relatively short-lived, and/or backed up elsewhere. This means we rarely need to shard data across nodes.
A fast and reliable background task processing library for Python 3.Project mention: Why is do_maintenance not locked? | reddit.com/r/dramatiq | 2021-02-10
Please see my first attempt here https://github.com/Bogdanp/dramatiq/pull/383
A lightweight library that adds job scheduling capabilities to RQ (Redis Queue)Project mention: Need direction on how to add asynchronous / scheduled tasks on my flask app running on aws beanstalk | reddit.com/r/flask | 2021-03-14
Provides a .NET wrapper for the Windows Task Scheduler. It aggregates the multiple versions, provides an editor and allows for localization. (by dahall)Project mention: I need an event system | reddit.com/r/learnprogramming | 2021-01-25
If it is the latter, and C# is your jam, this might do the trick for you! Example here.
Modern concurrency for C++. Tasks, executors, timers and C++20 coroutines to rule them allProject mention: Concurrencpp v0.0.9 – Modern concurrency for C++ | news.ycombinator.com | 2021-01-01
Persistent cluster-friendly scheduler for Java
Java framework for Cadence Workflow ServiceProject mention: Cannot Resolve Symbol Issues | reddit.com/r/java | 2021-02-17
Read contribution.md. Looks like you need to build it with Gradle.
Rqueue aka Redis Queue [Task Queue, Message Broker] for Spring framework
You had one job, or more then one, which can be done in steps
Smart Processes ManagementProject mention: Runnerty y Cypress para comprobar que tu app está online | dev.to | 2021-04-21
Bash script to automatically update Plex Media Server on Synology NASProject mention: PMS version mismatch on Synology (DSM 6) | reddit.com/r/PleX | 2021-04-09
It is automated which is how I noticed, I get an email every morning telling me it failed (even though it didn't). It's not my script (I use this one) but I'll see if there's an easy fix for comparing versions.
A fast, robust and extensible distributed task/job queue for Node.js, powered by Redis.
NVIDIA GPU compute task scheduling utilityProject mention: [D] How to be more productive while doing Deep Learning experiments? | reddit.com/r/MachineLearning | 2021-02-25
Sure. No, a simple bash script is not enough. In my case, we have several machines shared in the department, some with GPUs, some without. What I have is a python script that gets a list of jobs and then it schedule them in the first available machine (according to memory/CPU/GPU availability). Unfortunately, what I have is really entangled with our computing platform (Docker-based with a shared filesystem) and not really easy to have it as standalone project (that's why I said "know you infrastructure"). The most similar thing that I could find online is this project. I believe there are then some HPC tools that could be useful (e.g. Slurm), but that's way too much for what we need.
What are some of the best open-source task-scheduler projects? This list will help you: