Python Redis

Open-source Python projects categorized as Redis

Top 23 Python Redi Projects

  • celery

    Distributed Task Queue (development branch)

    Project mention: Django Styleguide | news.ycombinator.com | 2023-01-11

    I spent 3 years building a high scale crawler on top of Celery.

    I can't recommend it. We found many bugs in the more advanced features of Celery (like Canvas) we also ran into some really weird issues like tasks getting duplicated for no reason [1].

    The most concerning problem is that the project was abandoned. The original creator is not working on it anymore and all issues that we raised were ignored. We had to fork the project and apply our own fixes to it. This was 4 years ago so maybe things improved since them.

    Celery is also extremely complex.

    I would recommend https://dramatiq.io/ instead.

    [1]: https://github.com/celery/celery/issues/4426

  • rq

    Simple job queues for Python

    Project mention: What libraries do you use the most alongside django? | reddit.com/r/django | 2023-01-02
  • Sonar

    Write Clean Python Code. Always.. Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.

  • openreplay

    :tv: OpenReplay is developer-friendly, open-source session replay.

    Project mention: Awesome list of VCs investing in commercial open-source startups | reddit.com/r/opensource | 2022-09-14

    OpenReplay - Elastic license

  • dev-setup

    macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.

    Project mention: Automate Mac setup? | reddit.com/r/macsysadmin | 2022-04-10

    Something like this at least is the most direct answer to your question, as opposed to "you're doing it wrong" which unfortunately seems to be more upvoted. An example of something like this might be https://github.com/donnemartin/dev-setup

  • flower

    Real-time monitor and web admin for Celery distributed task queue

    Project mention: Python Celery - high level overview(animated video) | reddit.com/r/Python | 2023-01-29

    Not sure what your criteria for good observability ist, but flower served me well enough for this in the last couple of years.

  • scrapy-redis

    Redis-based components for Scrapy.

    Project mention: Ask HN: What are the best tools for web scraping in 2022? | news.ycombinator.com | 2022-08-10

    11. With some work, you can use Scrapy for distributed projects that are scraping thousands (millions) of domains. We are using https://github.com/rmax/scrapy-redis.

  • Stream-Framework

    Stream Framework is a Python library, which allows you to build news feed, activity streams and notification systems using Cassandra and/or Redis. The authors of Stream-Framework also provide a cloud service for feed technology:

    Project mention: Create a customer-admin chat portal for online ordering website | reddit.com/r/react | 2023-01-30

    Have you worked with Stream before? I think it's what you're looking for.

  • InfluxDB

    Build time-series-based applications quickly and at scale.. InfluxDB is the Time Series Platform where developers build real-time applications for analytics, IoT and cloud-native services. Easy to start, it is available in the cloud or on-premises.

  • huey

    a little task queue for python

    Project mention: What's the best thing you've learned about Django this year? | reddit.com/r/django | 2022-12-14

    Funny, just this moment i finally switched from Celery to huey. And so far I don't regret. huey looks very promising, has good documentation and is well integrated into DJango. You should give it a try: https://github.com/coleifer/huey

  • dramatiq

    A fast and reliable background task processing library for Python 3.

    Project mention: Django Styleguide | news.ycombinator.com | 2023-01-11

    I spent 3 years building a high scale crawler on top of Celery.

    I can't recommend it. We found many bugs in the more advanced features of Celery (like Canvas) we also ran into some really weird issues like tasks getting duplicated for no reason [1].

    The most concerning problem is that the project was abandoned. The original creator is not working on it anymore and all issues that we raised were ignored. We had to fork the project and apply our own fixes to it. This was 4 years ago so maybe things improved since them.

    Celery is also extremely complex.

    I would recommend https://dramatiq.io/ instead.

    [1]: https://github.com/celery/celery/issues/4426

  • flask-base

    A simple Flask boilerplate app with SQLAlchemy, Redis, User Authentication, and more.

    Project mention: Wrong dashboard while adding flask-admin to project | reddit.com/r/codehunter | 2022-03-14

    I'm trying to extend the flask-base project https://github.com/hack4impact/flask-base/tree/master/app. This uses the the application factory pattern in app/init.py and blueprints.

  • kombu

    Messaging library for Python.

    Project mention: Variables pool system | reddit.com/r/learnpython | 2022-05-04

    It is feasible and there are a number of ways to do it. There are a few issues, though, mainly how to make sure that you are accessing the latest data and not getting part of the data mid write. A database handles these issues pretty well, since generally they are meant to be an intermediary between different processes. Another solution, that is often more flexible, is a message queue, like this one: https://github.com/celery/kombu

  • aioredis

    asyncio (PEP 3156) Redis support

    Project mention: Pooling in aioredis may be dangerous | dev.to | 2022-06-05

    First, it was aioredis library. We are using sentinel based client because with this we can achieve failover easily. Aioredis spawn pool of connections, that transparently reconnects (and here third thing — FOREVER, hello DDOS) to our sentinel nodes, and then to master node. It supposed to do so. Also, we found that if you are not limiting maximum connections count, library will do it for you and set it as 2 ** 31 (here you can see it) — this is fourth thing. Furthermore, pool in our version (2.0.1) not closing automatically, and it makes the problem worse.

  • iredis

    Interactive Redis: A Terminal Client for Redis with AutoCompletion and Syntax Highlighting.

    Project mention: 5 Useful Database Command Line Tools | dev.to | 2022-04-25

    wget https://github.com/laixintao/iredis/releases/latest/download/iredis.tar.gz \ && tar -xzf iredis.tar.gz \ && ./iredis

  • Gopherus

    This tool generates gopher link for exploiting SSRF and gaining RCE in various servers

  • django-rq

    A simple app that provides django integration for RQ (Redis Queue)

    Project mention: Recommendations other than celery to send an API processing in background, which would only take 5 mins to process and API usage would be once a month or so. | reddit.com/r/django | 2022-12-12

    Yep, rq is simple and good: https://python-rq.org/ It also has a Django wrapper: https://github.com/rq/django-rq

  • arq

    Fast job queuing and RPC in python with asyncio and redis.

  • rq-scheduler

    A lightweight library that adds job scheduling capabilities to RQ (Redis Queue)

    Project mention: RQ-Scheduler for tasks in far future? | reddit.com/r/Python | 2022-12-28

    RQ-Scheduler is another simpler alternative (rq/rq-scheduler: A lightweight library that adds job scheduling capabilities to RQ (Redis Queue) (github.com)) that appears to be good for such purposes. It's not immediately clear if it would suffer from the same issues, but it seems not (Redis manages issues with data loss well, a separate queue is used for the scheduled tasks, etc.). Is anyone aware of any drawbacks to using RQ-Scheduler for something like this?

  • nagios-plugins

    450+ AWS, Hadoop, Cloud, Kafka, Docker, Elasticsearch, RabbitMQ, Redis, HBase, Solr, Cassandra, ZooKeeper, HDFS, Yarn, Hive, Presto, Drill, Impala, Consul, Spark, Jenkins, Travis CI, Git, MySQL, Linux, DNS, Whois, SSL Certs, Yum Security Updates, Kubernetes, Cloudera etc...

  • redis-py-cluster

    Python cluster client for the official redis cluster. Redis 3.0+.

    Project mention: MOVED error with Redis cluster | dev.to | 2022-02-05

    redis-py-cluster A port of redis-rb-cluster to Python. Supports majority of redis-py functionality. Is in active development.

  • walrus

    Lightweight Python utilities for working with Redis

  • requests-cache

    Transparent persistent cache for python requests

    Project mention: Web Scraping with Python: from Fundamentals to Practice | reddit.com/r/Python | 2022-06-23

    For anyone who goes with requests as your HTTP client, I would highly recommend adding requests-cache for a nice performance boost.

  • WordOps

    Install and manage a high performance WordPress stack with a few keystrokes

    Project mention: Hosting recommendation for a new e-commerce website. | reddit.com/r/webhosting | 2023-01-23

    Also, WordOps

  • faust

    Python Stream Processing. A Faust fork (by faust-streaming)

    Project mention: How to join using Faust Streaming (Python implementation of Kafka Streams API)? | reddit.com/r/apachekafka | 2023-01-31

    The forked one, https://github.com/faust-streaming/faust, has been updated but still doesn’t seem to support joins.

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2023-01-31.

Python Redis related posts

Index

What are some of the best open-source Redi projects in Python? This list will help you:

Project Stars
1 celery 20,878
2 rq 8,737
3 openreplay 6,817
4 dev-setup 5,874
5 flower 5,545
6 scrapy-redis 5,230
7 Stream-Framework 4,678
8 huey 4,340
9 dramatiq 3,430
10 flask-base 2,841
11 kombu 2,493
12 aioredis 2,263
13 iredis 2,232
14 Gopherus 2,174
15 django-rq 1,610
16 arq 1,458
17 rq-scheduler 1,280
18 nagios-plugins 1,083
19 redis-py-cluster 1,077
20 walrus 1,062
21 requests-cache 1,060
22 WordOps 995
23 faust 845
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com