Building a distributed scheduler

This page summarizes the projects mentioned and recommended in the original post on dev.to

Our great sponsors
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern identity platform for B2B SaaS
  • SaaSHub - Software Alternatives and Reviews
  • Fenzo

    Discontinued Extensible Scheduler for Mesos Frameworks

  • Netflix Fenzo checks a lot of requirements too. The main problem is that it relies on Apache Mesos and we use Kubernetes not Mesos as our Orchestrator. Changing our Orchestrator just for the scheduler system is a huge and risky change.

  • scheduler

    Discontinued A Scala library for scheduling arbitrary code to run at an arbitrary time. (by PagerDuty)

  • Pager Duty Scheduler checks a lot of requirements. The main problem is it uses Cassandra and Kafka; we don’t have any experience for both neither do we have other use cases than the scheduler which will need Cassandra or Kafka. I’m always reluctant to hosting new database systems, database systems are complex by nature and are not easy when it comes to scaling them. It’s a no go then.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • gocron

    Easy and fluent Go cron scheduling. This is a fork from https://github.com/jasonlvhit/gocron

  • This task will be scheduled using GoCron to run every minute. You can tweak this to run more often if you want, in our case we don’t need second precision so one minute is enough.

  • metronome

    Discontinued Metronome is a distributed and fault-tolerant event scheduler (by ovh)

  • OVH Metronome seems to check all the requirements, the point is it requires KAFKA and we already host similar services. But I can live with that, the main problem IMO is that it hasn’t been updated since November 2018 and it lacks a documentation on how to set this up, and how to schedule jobs. In the other hand I really like the idea of decoupling scheduling and execution, it’ll allow us to rely on a single scheduler no matter if we want to schedule jobs using Go, Scala or whatever language we’ll had in the future.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts