Top 10 Python Scheduler Projects
-
In this demo, we use the request itself as the trigger and begin computation immediately. But it may vary according to the nature of your application. Often, you might have to use a separate pipeline as well. In such scenarios, you may need technologies such as Apache Airflow or Prefect.
-
It has been such a pleasure to use dagster. The testability is nice. It was designed to be type aware, so you can leverage type checks and it is also designed to be data aware when it comes to passing data between tasks. One negative I dont like is its handling of instances where a task does not produce output, but need to still indicate dependency of another task, so you utilize its Nothing abstraction. The syntax for this situation is awkward IMO and they've recognized that. Its UI called dagit is hands down, the best as it provides rich information on each task in your DAG. The developer experience is definitely better with dagster compared to Airflow. I briefly looked at Airflow 2.0 examples, and I still think dagster's API is better ( with version 0.13.x ). However, on the managed environment side, there is no 3rd party managed dagster provider other than the creator of dagster called Elementl has their cloud offering which is currently in beta. So there is no mature managed services for dagster yet. Again, this is due to dagster being a relatively new library - less than 3 years old.
-
-
Project mention: Need direction on how to add asynchronous / scheduled tasks on my flask app running on aws beanstalk | reddit.com/r/flask | 2021-03-14
-
Any framework to add dynamically job scheduling like cron ? I find https://github.com/viniciuschiele/flask-apscheduler . Anything else like this?
-
Project mention: Wondering if I should use Celery vs threads for what I want to do | reddit.com/r/flask | 2021-08-12
From experience i would not use threads for this or any background jobs. I would use Celery or Flask-RQ2 to be your workers, you will also probably end up using them to run other tasks as you encounter the need for other jobs. They both use Redis as a broker and job store and you can use Redis for other things like caching and so many other useful features. I kind of like RQ2 more then Celery because its a little simpler but Celery has a lot more to offer, more features. RQ2 has rq-dashboard for monitoring jobs and Celery has Flower.
-
Project mention: Testing a new looks for my dashboard. More info in the comments. | reddit.com/r/LovelaceHA | 2021-09-01
scheduler-component
-
I would use deck-chores for this: https://github.com/funkyfuture/deck-chores
-
Scout APM
Scout APM: A developer's best friend. Try free for 14-days. Scout APM uses tracing logic that ties bottlenecks to source code so you know the exact line of code causing performance issues and can get back to building a great product faster.
-
another-autoscaler
Another Autoscaler is a Kubernetes controller that automatically starts, stops, or restarts pods from a deployment at a specified time using a cron expression.
Project mention: Another Scheduler is a Kubernetes controller that automatically starts, stops, or restarts pods from a deployment at a specified time using a cron annotation. | reddit.com/r/kubernetes | 2021-07-05 -
-
Project mention: Automated Photo taker on schedule | reddit.com/r/RASPBERRY_PI_PROJECTS | 2021-11-02
It can schedule complex logic pretty easily and has a bunch of features. If this is something that might be useful, here's the documentation: Red Engine, docs and the source code: Red Engine, Github
Python Scheduler related posts
Index
What are some of the best open-source Scheduler projects in Python? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | Airflow | 23,923 |
| 2 | dagster | 4,047 |
| 3 | rq-scheduler | 1,181 |
| 4 | flask-apscheduler | 846 |
| 5 | Flask-RQ2 | 189 |
| 6 | scheduler-component | 179 |
| 7 | deck-chores | 130 |
| 8 | another-autoscaler | 48 |
| 9 | coo | 43 |
| 10 | red-engine | 3 |
Are you hiring? Post a new remote job listing for free.
