Open-source projects categorized as task-scheduler | Edit details

Top 15 task-scheduler Open-Source Projects

  • GitHub repo celery

    Distributed Task Queue (development branch)

    Project mention: [P] Opyrator - Turn python functions into microservices with auto-generated HTTP API, interactive UI, and more. | reddit.com/r/MachineLearning | 2021-04-22

    That's a good point. What we have on the roadmap is that you can deploy it in a task queue mode (as an alternative option to the synchronous deployment). This still provides the same web API and UI, but the actual execution is happening within a background task, most likely using something like celery https://github.com/celery/celery

  • GitHub repo incubator-dolphinscheduler

    Apache DolphinScheduler is a distributed and extensible workflow scheduler platform with powerful DAG visual interfaces, dedicated to solving complex job dependencies in the data pipeline and providing various types of jobs available `out of the box`.

    Project mention: The Apache Software Foundation Announces Apache® DolphinScheduler™ as a Top-Level Project | reddit.com/r/u_mrlifeng | 2021-04-11

    Apache DolphinScheduler software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project’s day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache DolphinScheduler, visit https://DolphinScheduler.apache.org/ , https://twitter.com/DolphinSchedule , and https://asf-dolphinscheduler.slack.com/ .

  • GitHub repo machinery

    Machinery is an asynchronous task queue/job queue based on distributed message passing.

    Project mention: Exploring Redis High Availability | dev.to | 2021-04-22

    Such a deployment would improve task and queue-based systems like Celery in python or Machinery in go. Payloads in these systems tend to be small, relatively short-lived, and/or backed up elsewhere. This means we rarely need to shard data across nodes.

  • GitHub repo dramatiq

    A fast and reliable background task processing library for Python 3.

    Project mention: Why is do_maintenance not locked? | reddit.com/r/dramatiq | 2021-02-10

    Please see my first attempt here https://github.com/Bogdanp/dramatiq/pull/383

  • GitHub repo rq-scheduler

    A lightweight library that adds job scheduling capabilities to RQ (Redis Queue)

    Project mention: Need direction on how to add asynchronous / scheduled tasks on my flask app running on aws beanstalk | reddit.com/r/flask | 2021-03-14
  • GitHub repo TaskScheduler

    Provides a .NET wrapper for the Windows Task Scheduler. It aggregates the multiple versions, provides an editor and allows for localization. (by dahall)

    Project mention: I need an event system | reddit.com/r/learnprogramming | 2021-01-25

    If it is the latter, and C# is your jam, this might do the trick for you! Example here.

  • GitHub repo concurrencpp

    Modern concurrency for C++. Tasks, executors, timers and C++20 coroutines to rule them all

    Project mention: Concurrencpp v0.0.9 – Modern concurrency for C++ | news.ycombinator.com | 2021-01-01
  • GitHub repo db-scheduler

    Persistent cluster-friendly scheduler for Java

  • GitHub repo cadence-java-client

    Java framework for Cadence Workflow Service

    Project mention: Cannot Resolve Symbol Issues | reddit.com/r/java | 2021-02-17

    Read contribution.md. Looks like you need to build it with Gradle.

  • GitHub repo Rqueue

    Rqueue aka Redis Queue [Task Queue, Message Broker] for Spring framework

  • GitHub repo leprechaun

    You had one job, or more then one, which can be done in steps

  • GitHub repo runnerty

    Smart Processes Management

    Project mention: Runnerty y Cypress para comprobar que tu app está online | dev.to | 2021-04-21
  • GitHub repo syno.plexupdate

    Bash script to automatically update Plex Media Server on Synology NAS

    Project mention: PMS version mismatch on Synology (DSM 6) | reddit.com/r/PleX | 2021-04-09

    It is automated which is how I noticed, I get an email every morning telling me it failed (even though it didn't). It's not my script (I use this one) but I'll see if there's an easy fix for comparing versions.

  • GitHub repo Conveyor MQ

    A fast, robust and extensible distributed task/job queue for Node.js, powered by Redis.

  • GitHub repo nvidia-gpu-scheduler

    NVIDIA GPU compute task scheduling utility

    Project mention: [D] How to be more productive while doing Deep Learning experiments? | reddit.com/r/MachineLearning | 2021-02-25

    Sure. No, a simple bash script is not enough. In my case, we have several machines shared in the department, some with GPUs, some without. What I have is a python script that gets a list of jobs and then it schedule them in the first available machine (according to memory/CPU/GPU availability). Unfortunately, what I have is really entangled with our computing platform (Docker-based with a shared filesystem) and not really easy to have it as standalone project (that's why I said "know you infrastructure"). The most similar thing that I could find online is this project. I believe there are then some HPC tools that could be useful (e.g. Slurm), but that's way too much for what we need.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2021-04-22.


What are some of the best open-source task-scheduler projects? This list will help you:

Project Stars
1 celery 17,212
2 incubator-dolphinscheduler 5,715
3 machinery 5,156
4 dramatiq 2,559
5 rq-scheduler 1,111
6 TaskScheduler 656
7 concurrencpp 411
8 db-scheduler 381
9 cadence-java-client 90
10 Rqueue 85
11 leprechaun 82
12 runnerty 75
13 syno.plexupdate 66
14 Conveyor MQ 31
15 nvidia-gpu-scheduler 3