Top 23 Python Redi Projects

• celery

  12 22,615 9.3 Python

  Distributed Task Queue (development branch)

  

  Project mention: Streaming responses to websockets with multiple LLMs, am I going about this wrong? | /r/LangChain | 2023-06-20

  So this might be my understanding, but stuff like celery is more like an orchestrator that chunks up workloads (think Hadoop with multiple nodes).

• rq

  7 9,232 7.0 Python

  Simple job queues for Python

  

  Project mention: The Many Problems with Celery | /r/Python | 2023-05-22

  https://github.com/rq/rq is to the rescue.

• InfluxDB

  www.influxdata.com

  sponsored

  Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.

  

• dev-setup

  0 5,999 0.0 Python

  macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.

• flower

  2 5,959 8.2 Python

  Real-time monitor and web admin for Celery distributed task queue

  Project mention: Scaling Celery to handle workflows and multiple queues | dev.to | 2023-02-04

  Use flower to monitor workers and tasks.

• scrapy-redis

  2 5,380 0.0 Python

  Redis-based components for Scrapy.

  Project mention: How to make scrapy run multiple times on the same URLs? | /r/scrapy | 2023-06-26

• Stream-Framework

  14 4,705 0.0 Python

  Stream Framework is a Python library, which allows you to build news feed, activity streams and notification systems using Cassandra and/or Redis. The authors of Stream-Framework also provide a cloud service for feed technology:

  Project mention: Recommendations for an external messenger integration/API? | /r/rails | 2023-10-30

  I have looked into a getstream.io integration, however it seems that the Ruby SDK is really treated as a second class citizen. There's bugs with the documented API (I'm having issues even creating users and querying users), the usage of the gem is low and there is an open issue since May that no one has even looked at, which doesn't give me hope for long term support.

• huey

  3 4,700 6.4 Python

  a little task queue for python

  Project mention: Nextflow: Data-Driven Computational Pipelines | news.ycombinator.com | 2023-08-10

  I've considered using Nextflow for bioinformatics pipelines but have yet to take the plunge. At work, I develop a proteomics pipeline that is composed of huey¹ tasks (Python library; simple alternative to Celery) which either use subprocess to call out to some external tool, or are just pure python. It runs in a worker container which is created by docker swarm, and all containers pull jobs from redis. For our scale, it works great. However, I don't have control over the resource utilization of individual steps, and in the past I've had issues with the pipeline blocking as a result of how I was chaining tasks together. I think something like Nextflow would remove these limitations, but one thing I think I would miss is the ability to debug individual pipeline steps locally with an interactive debugger. As far as I can tell, Nextflow has logging/tracing facilities but nothing quite like an interactive debugger. I'd be happy to be told I'm wrong, or even that I'm doing it wrong.

  ____

  ¹ https://github.com/coleifer/huey/

• Onboard AI

  getonboard.dev

  sponsored

  Learn any GitHub repo in 59 seconds. Onboard AI learns any GitHub repo in minutes and lets you chat with it to locate functionality, understand different parts, and generate new code. Use it for free at www.getonboard.dev.

  

• dramatiq

  4 3,843 8.2 Python

  A fast and reliable background task processing library for Python 3.

  Project mention: Inngest raises $3M seed to build the reliable workflow platform for every dev | news.ycombinator.com | 2023-07-12

  Using something like Dramatiq [1] with Redis, writing a background job takes minutes, and can be deployed alongside an existing Python web app. There are probably JS equivalents.

  I think Inngest could be a useful service, but the comparison felt off for me - it made me feel like this wasn't solving a real problem.

  [1] https://dramatiq.io/

• flask-base

  0 2,934 0.0 Python

  A simple Flask boilerplate app with SQLAlchemy, Redis, User Authentication, and more.

  

• kombu

  0 2,649 8.7 Python

  Messaging library for Python.

  

• Gopherus

  0 2,508 0.0 Python

  This tool generates gopher link for exploiting SSRF and gaining RCE in various servers

• iredis

  0 2,395 8.3 Python

  Interactive Redis: A Terminal Client for Redis with AutoCompletion and Syntax Highlighting.

• arq

  1 1,729 0.0 Python

  Fast job queuing and RPC in python with asyncio and redis.

  Project mention: The Many Problems with Celery | /r/Python | 2023-05-22

• django-rq

  5 1,707 7.4 Python

  A simple app that provides django integration for RQ (Redis Queue)

  

  Project mention: Background jobs with Django | /r/django | 2023-05-30

  For simple stuff, I tend to use https://github.com/rq/django-rq. Although scheduling tasks there does not work well in my experience. If I need something to run by schedule, than better just plain cron.

• rq-scheduler

  2 1,358 5.9 Python

  A lightweight library that adds job scheduling capabilities to RQ (Redis Queue)

  

  Project mention: Keep the Monolith, but Split the Workloads | news.ycombinator.com | 2023-04-24

• faust

  5 1,289 8.2 Python

  Python Stream Processing. A Faust fork (by faust-streaming)

  

  Project mention: New release of FastKafka supporting Redpanda | /r/Python | 2023-03-24

  We have many similar ideas and paradigms, as one would expect from modern frameworks tackling the same problem. However, Faust has been deprecated and no longer managed or supported (there is a fork being maintained by the community at https://github.com/faust-streaming/faust). I guess Robinhood at some point decided not to spend any more resources on it, an often destiny of such side projects by large corporations. On the other hand, we hope to stay around for a long time and build to a framework and a community that loves it. This initial version was built using many libraries in order to have a working system and to gather feedback from the large community. We plan to replace eventually all, or at least most of dependancies, and replace then with Rust lib with Python bindings.

• requests-cache

  0 1,186 8.1 Python

  Transparent persistent cache for python requests

  

• WordOps

  4 1,167 0.0 Python

  Install and manage a high performance WordPress stack with a few keystrokes

  

  Project mention: [Guide] Do a full setup of WordPress using Nginx, Redis and WP CLI | /r/selfhosted | 2023-03-24

  https://wordops.net is also nice to automate parts, if you don't mind that it's all running under the same user.

• walrus

  0 1,114 0.0 Python

  Lightweight Python utilities for working with Redis

• nagios-plugins

  0 1,110 9.1 Python

  450+ AWS, Hadoop, Cloud, Kafka, Docker, Elasticsearch, RabbitMQ, Redis, HBase, Solr, Cassandra, ZooKeeper, HDFS, Yarn, Hive, Presto, Drill, Impala, Consul, Spark, Jenkins, Travis CI, Git, MySQL, Linux, DNS, Whois, SSL Certs, Yum Security Updates, Kubernetes, Cloudera etc...

• redis-py-cluster

  0 1,093 0.0 Python

  Python cluster client for the official redis cluster. Redis 3.0+.

• docker-django-example

  11 1,009 0.0 Python

  A production ready example Django app that's using Docker and Docker Compose.

  Project mention: Django 5.0 Is Released | news.ycombinator.com | 2023-12-04

  Congrats on the release to the Django community!

  If anyone is curious, I updated my Django / Docker starter kit app to use Django 5.0 at: https://github.com/nickjj/docker-django-example

  It pulls together gunicorn, Celery, Redis, Postgres, esbuild and Tailwind with Docker Compose. It's set up to run in both development and production.

• aiocache

  0 953 0.0 Python

  Asyncio cache manager for redis, memcached and memory

  

• SaaSHub

  www.saashub.com

  sponsored

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Python Redis related posts

• Monitoring your logs is mostly a tarpit
  1 project | news.ycombinator.com | 7 Aug 2023
• Working with Docker Containers Made Easy with the Dexec Bash Script
  5 projects | news.ycombinator.com | 26 Jun 2023
• How to make scrapy run multiple times on the same URLs?
  2 projects | /r/scrapy | 26 Jun 2023
• Ask HN: Who has a smaller Redis DB with lots of reads and writes?
  1 project | news.ycombinator.com | 9 Jun 2023
• What's the correct way to install or version up packages when using Docker and Poetry?
  1 project | /r/django | 9 Jun 2023
• Background jobs with Django
  4 projects | /r/django | 30 May 2023
• The Many Problems with Celery
  5 projects | /r/Python | 22 May 2023
• A note from our sponsor - #<SponsorshipServiceOld:0x00007f0fa1b1bc48>
  www.saashub.com | 4 Dec 2023

  SaaSHub helps you find the best software and product alternatives Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com

Do not miss the trending Python projects with our weekly report!