Top 23 Python Redi Projects
-
Would recommend checking out celery: https://github.com/celery/celery
-
Project mention: Preventing multiple request for a long running process | reddit.com/r/flask | 2021-10-28
For Queues, look into RQ Worker.
-
OPS
OPS - Build and Run Open Source Unikernels. Quickly and easily build and deploy open source unikernels in tens of seconds. Deploy in any language to any cloud.
-
dev-setup
macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.
donnemartin - dev setup
-
Project mention: Wondering if I should use Celery vs threads for what I want to do | reddit.com/r/flask | 2021-08-12
From experience i would not use threads for this or any background jobs. I would use Celery or Flask-RQ2 to be your workers, you will also probably end up using them to run other tasks as you encounter the need for other jobs. They both use Redis as a broker and job store and you can use Redis for other things like caching and so many other useful features. I kind of like RQ2 more then Celery because its a little simpler but Celery has a lot more to offer, more features. RQ2 has rq-dashboard for monitoring jobs and Celery has Flower.
-
Stream-Framework
Stream Framework is a Python library, which allows you to build news feed, activity streams and notification systems using Cassandra and/or Redis. The authors of Stream-Framework also provide a cloud service for feed technology:
I also came around Getstream.io it looks awesome.
-
Project mention: What is the best option for a (Python 3) task queue on Windows now that Celery 4 has dropped Windows support? | reddit.com/r/codehunter | 2021-12-11
huey
-
AFAIK the only one is Dramatiq [1] with dramatiq-pg [2]: a 3rd party message broker using Postgres LISTEN/NOTIFY.
[1]: https://dramatiq.io/
-
SonarQube
Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.
-
Project mention: Any good example projects that I can use to compare my own flask site? | reddit.com/r/flask | 2021-02-22
I came across (flask-base)[https://github.com/hack4impact/flask-base], which is a nice and robust boilerplate for large flask apps, you could use this as a reference point.
-
I'm hoping to leverage Redis with my project, and I was curious as to if anyone had any general pointers as to how best to manage the DB connection. I'm using aioredis such that I can leverage the async functionality, but I haven't been very happy with the library's documentation surrounding how best to leverage the connection pool. Most of the examples create and tear down the pool immediately without showing how best to manage its lifespan.
-
Project mention: Is the Web Application Hacker's Handbook still relevant? | reddit.com/r/Pentesting | 2021-05-06
-
-
Since you're already using Redis, if you use Django-RQ instead of Celery to queue background tasks you may be able to use your existing worker dyno to process background tasks.
-
Project mention: Need direction on how to add asynchronous / scheduled tasks on my flask app running on aws beanstalk | reddit.com/r/flask | 2021-03-14
-
Project mention: I made a simple async queueing framework called SAQ! It includes a built in web UI to manage jobs. | reddit.com/r/Python | 2022-01-06
I need to process a lot of long running IO heavy jobs with background workers. I've been using ARQ for a while but decided to take a crack at writing my own distributed queue.
-
nagios-plugins
450+ AWS, Hadoop, Cloud, Kafka, Docker, Elasticsearch, RabbitMQ, Redis, HBase, Solr, Cassandra, ZooKeeper, HDFS, Yarn, Hive, Presto, Drill, Impala, Consul, Spark, Jenkins, Travis CI, Git, MySQL, Linux, DNS, Whois, SSL Certs, Yum Security Updates, Kubernetes, Cloudera etc...
Did you check this issue for possible fixes of your first problem? https://github.com/HariSekhon/Nagios-Plugins/issues/328
-
Project mention: FaPro: A mass network protocol server simulator tool | reddit.com/r/netsec | 2021-09-30
-
Project mention: Kvrocks: NoSQL database based on RocksDB and compatible with Redis protocol | reddit.com/r/redis | 2021-11-12
I remember testing a bunch of redis clones and found the following test suite quite good: https://github.com/coleifer/walrus/tree/master/walrus/tests
-
Project mention: If I keep making URL requests in a forloop, is that harmful? | reddit.com/r/learnpython | 2021-09-04
-
redbeat
RedBeat is a Celery Beat Scheduler that stores the scheduled tasks and runtime metadata in Redis.
Project mention: How I made Python/Django Apps deploy themselves | news.ycombinator.com | 2021-09-09of course it does. Appliku out of the box provides a way to provision redis and rabbmitmq.
I have yet to write a tutorial about celery.
To specify processes to run you use Procfile. Specify your celery commands and enable them in Process tab in Appliku interface.
In order to have celery beat working you should use redbeat: https://github.com/sibson/redbeat
Other than that – no specific requirements. Drop me a line in contacts on our site if you need any help. Or in our Discord Community: https://appliku.com/discord
-
Project mention: Worth wrapping pottery functions for compliance with async? | reddit.com/r/Python | 2021-08-01
I have a question about https://github.com/brainix/pottery. It provides a nice Pythonic API by wrapping Redis constructs with Python Redis-backed data structures (Dict, Deque, etc.). I am using it in a Fastapi microservice project, which is obviously async.
-
-
Project mention: GitHub - SlavikMIPT/tgcloud: Opensource Telegram based cloud storage | reddit.com/r/DonOperInfo | 2021-05-31
-
Python Redis related posts
- I made a simple async queueing framework called SAQ! It includes a built in web UI to manage jobs.
- What is the best option for a (Python 3) task queue on Windows now that Celery 4 has dropped Windows support?
- Django 4.0 released
- Project with Faust and Django
- Ping Pong 🏓 Authx 0.1.3 is here 🥳
- Kvrocks: NoSQL database based on RocksDB and compatible with Redis protocol
- Pydbantic - A single model ( DB & Pydantic) with automatic migrations
Index
What are some of the best open-source Redi projects in Python? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | celery | 18,495 |
| 2 | rq | 8,092 |
| 3 | dev-setup | 5,590 |
| 4 | flower | 5,064 |
| 5 | Stream-Framework | 4,585 |
| 6 | huey | 3,788 |
| 7 | dramatiq | 2,862 |
| 8 | flask-base | 2,693 |
| 9 | aioredis | 2,031 |
| 10 | Gopherus | 1,733 |
| 11 | iredis | 1,693 |
| 12 | django-rq | 1,474 |
| 13 | rq-scheduler | 1,193 |
| 14 | arq | 1,063 |
| 15 | nagios-plugins | 1,034 |
| 16 | fapro | 986 |
| 17 | walrus | 942 |
| 18 | requests-cache | 878 |
| 19 | redbeat | 701 |
| 20 | pottery | 608 |
| 21 | FeedHQ | 530 |
| 22 | tgcloud | 427 |
| 23 | Flask-Redis | 395 |
Are you hiring? Post a new remote job listing for free.
