celery-kubernetes-example
kombu
celery-kubernetes-example | kombu | |
---|---|---|
1 | 2 | |
113 | 2,753 | |
- | 0.8% | |
0.0 | 8.8 | |
about 2 years ago | 4 days ago | |
Python | Python | |
MIT License | BSD 3-clause "New" or "Revised" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
celery-kubernetes-example
-
how to have a continuously running script with flask?
I had a similar question a week ago and celery was the move but it took a while to figure it out. Some things that helped me was https://blog.miguelgrinberg.com/post/using-celery-with-flask and https://github.com/matiaslindgren/celery-kubernetes-example If you want to deploy it in kube I move out of google cloud run because of the redis and celery workers. (I added a pull request to add a gke branch to deploy it there.)
kombu
-
Variables pool system
It is feasible and there are a number of ways to do it. There are a few issues, though, mainly how to make sure that you are accessing the latest data and not getting part of the data mid write. A database handles these issues pretty well, since generally they are meant to be an intermediary between different processes. Another solution, that is often more flexible, is a message queue, like this one: https://github.com/celery/kombu
-
Messaging and Madness: Sending Messages with AMQP and Amazon MQ
Above I use the library Kombu to create some connections and send some stuff. I started by setting up our environment variables. Then created exchange and queue objects. Finally, I made our connection object and the producer object, and then we sent a simple “Hello” message.
What are some alternatives?
celery-exporter - A Prometheus exporter for Celery metrics
celery - Distributed Task Queue (development branch)
guane-intern-fastapi - FastAPI-PostgreSQL-Celery-RabbitMQ-Redis bakcend with Docker containerization
pika - Pure Python RabbitMQ/AMQP 0-9-1 client library
kubernetes-workshop - ⚙️ A Gentle introduction to Kubernetes with more than just the basics. 🌟 Give it a star if you like it.
rq - Simple job queues for Python
nagios-plugins - 450+ AWS, Hadoop, Cloud, Kafka, Docker, Elasticsearch, RabbitMQ, Redis, HBase, Solr, Cassandra, ZooKeeper, HDFS, Yarn, Hive, Presto, Drill, Impala, Consul, Spark, Jenkins, Travis CI, Git, MySQL, Linux, DNS, Whois, SSL Certs, Yum Security Updates, Kubernetes, Cloudera etc...
huey - a little task queue for python
flower - Real-time monitor and web admin for Celery distributed task queue
gevent - Coroutine-based concurrency library for Python
rele - Easy to use Google Pub/Sub
KQ - Kafka-based Job Queue for Python