procrastinate
rq
Our great sponsors
procrastinate | rq | |
---|---|---|
7 | 26 | |
714 | 9,448 | |
9.0% | 1.0% | |
9.6 | 8.3 | |
6 days ago | 2 days ago | |
Python | Python | |
MIT License | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
procrastinate
- Anything can be a message queue if you use it wrongly enough
-
The Many Problems with Celery
What about https://github.com/procrastinate-org/procrastinate (postgresql task queue with transactions & stuff)
-
Keep the Monolith, but Split the Workloads
If you're using PostgreSQL, then
django-postgres-queue: https://github.com/gavinwahl/django-postgres-queue
procrastinate: https://github.com/procrastinate-org/procrastinate/
-
Issues/Experience with Procrastinate library for distributed tasks
We chose the Procrastinate library to run periodic tasks.
- Alchemical Queues: (task) queues on pure SQLAlchemy
- Grafana releases OnCall open source project
rq
-
The Many Problems with Celery
https://github.com/rq/rq is to the rescue.
-
Keep the Monolith, but Split the Workloads
We use RQ[0], it has Redis as a dependency. Itโs pretty straightforward and weโre very happy with it. If you are using Django you may want to look at Django RQ[1] as well. RQ has built in scheduling capabilities these days, but historically it did not so we used (and still use) RQ Scheduler[2] which I think still has some advantages over the built in stuff.
-
SQL Maxis: Why We Ditched RabbitMQ and Replaced It with a Postgres Queue
Also had a similar experience using RabbitMQ with Django+Celery. Extremely complicated and workers/queues would just stop for no reason.
Moved to Python-RQ [1] + Redis and been rock solid for years now.
- What libraries do you use the most alongside django?
-
Recommendations other than celery to send an API processing in background, which would only take 5 mins to process and API usage would be once a month or so.
Yep, rq is simple and good: https://python-rq.org/ It also has a Django wrapper: https://github.com/rq/django-rq
- Dynamically update periodic tasks in Celery and Django
- Celery + RabbitMQ alternatives
-
Are there any self-hosted web front-ends or Discord bots I can run in front of the Python scripts?
It's just Flask for the API, rq for the queue and then a modified version of txt2img.py
-
What is your favourite task queuing framework?
RQ -> ...?
-
How to create multiple workers in Python-RQ?
We are using Django and Python's rq library. We are not using django-rq (Please let me know if there are advantages in using this)
What are some alternatives?
celery - Distributed Task Queue (development branch)
huey - a little task queue for python
RabbitMQ - Open source RabbitMQ: core server and tier 1 (built-in) plugins
mrq - Mr. Queue - A distributed worker task queue in Python using Redis & gevent
Apache Kafka - Mirror of Apache Kafka
Flask-RQ2 - A Flask extension for RQ.
KQ - Kafka-based Job Queue for Python
kombu - Messaging library for Python.
Streamz - Real-time stream processing for python
budibase - Budibase is an open-source low code platform that helps you build internal tools in minutes ๐
Celery-Kubernetes-Operator - An operator to manage celery clusters on Kubernetes (Work in Progress)
starlette - The little ASGI framework that shines. ๐