Top 10 Python Scheduler Projects
Apache Airflow - A platform to programmatically author, schedule, and monitor workflowsProject mention: How to Serve Massive Computations Using Python Web Apps. | dev.to | 2021-11-23
In this demo, we use the request itself as the trigger and begin computation immediately. But it may vary according to the nature of your application. Often, you might have to use a separate pipeline as well. In such scenarios, you may need technologies such as Apache Airflow or Prefect.
An orchestration platform for the development, production, and observation of data assets.Project mention: Airflow 2.0 vs Prefect | reddit.com/r/dataengineering | 2021-10-20
It has been such a pleasure to use dagster. The testability is nice. It was designed to be type aware, so you can leverage type checks and it is also designed to be data aware when it comes to passing data between tasks. One negative I dont like is its handling of instances where a task does not produce output, but need to still indicate dependency of another task, so you utilize its Nothing abstraction. The syntax for this situation is awkward IMO and they've recognized that. Its UI called dagit is hands down, the best as it provides rich information on each task in your DAG. The developer experience is definitely better with dagster compared to Airflow. I briefly looked at Airflow 2.0 examples, and I still think dagster's API is better ( with version 0.13.x ). However, on the managed environment side, there is no 3rd party managed dagster provider other than the creator of dagster called Elementl has their cloud offering which is currently in beta. So there is no mature managed services for dagster yet. Again, this is due to dagster being a relatively new library - less than 3 years old.
Run Linux Software Faster and Safer than Linux with Unikernels.
A lightweight library that adds job scheduling capabilities to RQ (Redis Queue)Project mention: Need direction on how to add asynchronous / scheduled tasks on my flask app running on aws beanstalk | reddit.com/r/flask | 2021-03-14
Adds APScheduler support to FlaskProject mention: Add Dynamically Job Scheduling | news.ycombinator.com | 2021-08-14
Any framework to add dynamically job scheduling like cron ? I find https://github.com/viniciuschiele/flask-apscheduler . Anything else like this?
A Flask extension for RQ.Project mention: Wondering if I should use Celery vs threads for what I want to do | reddit.com/r/flask | 2021-08-12
From experience i would not use threads for this or any background jobs. I would use Celery or Flask-RQ2 to be your workers, you will also probably end up using them to run other tasks as you encounter the need for other jobs. They both use Redis as a broker and job store and you can use Redis for other things like caching and so many other useful features. I kind of like RQ2 more then Celery because its a little simpler but Celery has a lot more to offer, more features. RQ2 has rq-dashboard for monitoring jobs and Celery has Flower.
Custom component for HA that enables the creation of scheduler entitiesProject mention: Testing a new looks for my dashboard. More info in the comments. | reddit.com/r/LovelaceHA | 2021-09-01
A job scheduler for Docker containers, configured via labels.Project mention: crontab container | reddit.com/r/docker | 2021-09-30
I would use deck-chores for this: https://github.com/funkyfuture/deck-chores
Scout APM: A developer's best friend. Try free for 14-days. Scout APM uses tracing logic that ties bottlenecks to source code so you know the exact line of code causing performance issues and can get back to building a great product faster.
Another Autoscaler is a Kubernetes controller that automatically starts, stops, or restarts pods from a deployment at a specified time using a cron expression.Project mention: Another Scheduler is a Kubernetes controller that automatically starts, stops, or restarts pods from a deployment at a specified time using a cron annotation. | reddit.com/r/kubernetes | 2021-07-05
Schedule Twitter updates with easy
Advanced scheduling library for Python.Project mention: Automated Photo taker on schedule | reddit.com/r/RASPBERRY_PI_PROJECTS | 2021-11-02
It can schedule complex logic pretty easily and has a bunch of features. If this is something that might be useful, here's the documentation: Red Engine, docs and the source code: Red Engine, Github
Python Scheduler related posts
Automated Photo taker on schedule
1 project | reddit.com/r/RASPBERRY_PI_PROJECTS | 2 Nov 2021
Red Engine: A scheduler for productivity.
1 project | reddit.com/r/Python | 3 Oct 2021
Cron UI for status overview (Dashboard/Control Panel)
4 projects | reddit.com/r/selfhosted | 3 Sep 2021
Wondering if I should use Celery vs threads for what I want to do
4 projects | reddit.com/r/flask | 12 Aug 2021
Easy Cron Alternative
2 projects | reddit.com/r/selfhosted | 17 Jul 2021
Call function every x minutes
1 project | reddit.com/r/flask | 17 Jun 2021
New to data orchestration? Start here.
2 projects | dev.to | 2 Jun 2021
What are some of the best open-source Scheduler projects in Python? This list will help you:
Are you hiring? Post a new remote job listing for free.