The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning. Learn more →
Top 23 Python Redi Projects
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
-
dev-setup
macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
Stream-Framework
Stream Framework is a Python library, which allows you to build news feed, activity streams and notification systems using Cassandra and/or Redis. The authors of Stream-Framework also provide a cloud service for feed technology:
-
-
-
-
-
-
-
-
nagios-plugins
450+ AWS, Hadoop, Cloud, Kafka, Docker, Elasticsearch, RabbitMQ, Redis, HBase, Solr, Cassandra, ZooKeeper, HDFS, Yarn, Hive, Presto, Drill, Impala, Consul, Spark, Jenkins, Travis CI, Git, MySQL, Linux, DNS, Whois, SSL Certs, Yum Security Updates, Kubernetes, Cloudera etc...
-
fastapi-cache
fastapi-cache is a tool to cache fastapi response and function result, with backends support redis and memcached.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Project mention: Streaming responses to websockets with multiple LLMs, am I going about this wrong? | /r/LangChain | 2023-06-20So this might be my understanding, but stuff like celery is more like an orchestrator that chunks up workloads (think Hadoop with multiple nodes).
My jaw dropped when I saw the postmortem — it was exactly the same bug concept, just in a different library! A reminder that hard things often transcend particular languages and libraries. And boy, is this a hard bug. It sits at the intersection of caching, shared resource management, and state corruption — infamously tricky problem spaces.
That's pretty cool. Reckon it would work with existing code that calls Redis over the wire for RQ?
https://python-rq.org
Project mention: Ask HN: What are the drawbacks of caching LLM responses? | news.ycombinator.com | 2024-03-15Just found this: https://github.com/zilliztech/GPTCache which seems to address this idea/issue.
I've considered using Nextflow for bioinformatics pipelines but have yet to take the plunge. At work, I develop a proteomics pipeline that is composed of huey¹ tasks (Python library; simple alternative to Celery) which either use subprocess to call out to some external tool, or are just pure python. It runs in a worker container which is created by docker swarm, and all containers pull jobs from redis. For our scale, it works great. However, I don't have control over the resource utilization of individual steps, and in the past I've had issues with the pipeline blocking as a result of how I was chaining tasks together. I think something like Nextflow would remove these limitations, but one thing I think I would miss is the ability to debug individual pipeline steps locally with an interactive debugger. As far as I can tell, Nextflow has logging/tracing facilities but nothing quite like an interactive debugger. I'd be happy to be told I'm wrong, or even that I'm doing it wrong.
____
¹ https://github.com/coleifer/huey/
I have looked into a getstream.io integration, however it seems that the Ruby SDK is really treated as a second class citizen. There's bugs with the documented API (I'm having issues even creating users and querying users), the usage of the gem is low and there is an open issue since May that no one has even looked at, which doesn't give me hope for long term support.
Project mention: Dramatiq: A fast and reliable distributed task processing library for Python | news.ycombinator.com | 2024-02-08
For simple stuff, I tend to use https://github.com/rq/django-rq. Although scheduling tasks there does not work well in my experience. If I need something to run by schedule, than better just plain cron.
Project mention: faust VS quix-streams - a user suggested alternative | libhunt.com/r/faust-streaming/faust | 2023-12-07
Project mention: Caching Api route response in redis/in-memory/memcache other methods to reduce db/external api calls? | /r/sveltejs | 2023-05-04Recently I had been working with a stock/derivatve market indicator platform for a client. It requires us to fetch data from timescaledb and then perform computation on it and display results. Now db call is expensive since ton of data and computation is also expensive. So waht we did was used Sveltekit woth sveltequery configured with auto refetch in frontend and used fastapi for creating apis which return the data pre computed. We used a https://github.com/long2ice/fastapi-cache library with it to make sure that computation happens only once and all the clients get cached response. I wonder if there is any library like this one which is easy to plug directly in sveltekit API route. So that we can move completely to sveltekit rather than maintaining seperate servers. (We would be using Danfo.js as replacement to pandas for computations)
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Python Redis related posts
- Do you *really* need to store all that telemetry?
- Future Plan for Arq
- Monitoring your logs is mostly a tarpit
- Working with Docker Containers Made Easy with the Dexec Bash Script
- How to make scrapy run multiple times on the same URLs?
- Ask HN: Who has a smaller Redis DB with lots of reads and writes?
- What's the correct way to install or version up packages when using Docker and Poetry?
-
A note from our sponsor - WorkOS
workos.com | 23 Apr 2024
Index
What are some of the best open-source Redi projects in Python? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | celery | 23,439 |
| 2 | redis-py | 12,227 |
| 3 | rq | 9,503 |
| 4 | GPTCache | 6,387 |
| 5 | flower | 6,158 |
| 6 | dev-setup | 6,032 |
| 7 | scrapy-redis | 5,451 |
| 8 | huey | 4,877 |
| 9 | Stream-Framework | 4,718 |
| 10 | dramatiq | 4,051 |
| 11 | flask-base | 2,978 |
| 12 | kombu | 2,748 |
| 13 | Gopherus | 2,644 |
| 14 | iredis | 2,486 |
| 15 | arq | 1,904 |
| 16 | django-rq | 1,764 |
| 17 | faust | 1,442 |
| 18 | rq-scheduler | 1,384 |
| 19 | requests-cache | 1,254 |
| 20 | WordOps | 1,226 |
| 21 | walrus | 1,136 |
| 22 | nagios-plugins | 1,119 |
| 23 | fastapi-cache | 1,120 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com