Our great sponsors
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
django-readers
A lightweight function-oriented toolkit for better organisation of business logic and efficient selection and projection of data in Django projects.
-
rapidpro
TextIt is a hosted service allowing organizations to visually build scalable interactive messaging applications. (by nyaruka)
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
* For read endpoints and associated business logic, I'd encourage https://www.django-readers.org/ (disclaimer: I'm the author).
Since everyone suggests alternatives to Celery, may I plug my own healthier celery?
https://github.com/NicolasLM/spinach
Maybe you need to turn this into an article because over the last decade of working with Django, we've learnt the same lessons, sometimes the hard way. Learning to never import real models into data migrations was a big one.
I recently wanted to move a model between apps and ended up going the route of create new table, copy all rows over, delete old table. It was annoying, but the only way to make it work with regular migrations.
We ended up writing our own script[1] to squash migrations, and I'd love to know if there's a better way. We needed something that works for clean installs or existing installs that already have the current migrations installed - so it generates empty migrations which get applied on existing installs, and then they get replaced with real initial migrations on clean installs starting from a new release.
1. https://github.com/nyaruka/rapidpro/blob/main/tools/squash_m...
Also, do not forget to integrate with Sentry [https://sentry.io/] or something that does the same thing.
It will also keep you sane.
I spent 3 years building a high scale crawler on top of Celery.
I can't recommend it. We found many bugs in the more advanced features of Celery (like Canvas) we also ran into some really weird issues like tasks getting duplicated for no reason [1].
The most concerning problem is that the project was abandoned. The original creator is not working on it anymore and all issues that we raised were ignored. We had to fork the project and apply our own fixes to it. This was 4 years ago so maybe things improved since them.
Celery is also extremely complex.
I would recommend https://dramatiq.io/ instead.
[1]: https://github.com/celery/celery/issues/4426
I spent 3 years building a high scale crawler on top of Celery.
I can't recommend it. We found many bugs in the more advanced features of Celery (like Canvas) we also ran into some really weird issues like tasks getting duplicated for no reason [1].
The most concerning problem is that the project was abandoned. The original creator is not working on it anymore and all issues that we raised were ignored. We had to fork the project and apply our own fixes to it. This was 4 years ago so maybe things improved since them.
Celery is also extremely complex.
I would recommend https://dramatiq.io/ instead.
[1]: https://github.com/celery/celery/issues/4426