baseplate.py
reddit's python service framework (by reddit)
redsync
Distributed mutual exclusion lock using Redis for Go (by go-redsync)
baseplate.py | redsync | |
---|---|---|
11 | 12 | |
529 | 3,103 | |
0.0% | 2.0% | |
8.5 | 7.9 | |
5 days ago | 20 days ago | |
Python | Go | |
BSD 3-clause "New" or "Revised" License | BSD 3-clause "New" or "Revised" License |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
baseplate.py
Posts with mentions or reviews of baseplate.py.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-05-12.
-
The .zip TLD sucks and it needs to be immediately revoked.
Almost any download link on the internet (for example an attachment to any Wordpress blog post) could serve as an example. Since we are on a programming subreddit let's use Github as an example. When you open a repository and click the "Download ZIP" button it's a link straight to an URL like this one: https://github.com/reddit/baseplate.py/archive/refs/heads/develop.zip (this particular one is for Reddit's Python library).
-
Python use by SWEs
Even Reddit has python backends https://github.com/reddit/baseplate.py based on Pyramid. They also have a go one. https://github.com/reddit/baseplate.go
-
Reddit System Design/Architecture
there's a multitude of services in reddit's architecture. as far as i can tell, they mostly using reddit's baseplate framework (which has implementations in both python and go).
-
Reddit Recap Series: Backend Performance Tuning
Finally, the problem that we didn’t experience directly, but it was mentioned during consultations with another team that had experience with pgBouncer: the Baseplate.py framework that both of us are using sometimes leaked the connections, leaving them open after the request, but not returning them back into the pool.
- What is an example of a fully finished python software product on github?
-
Is the Pyramid framework dead?
Also reddit team using pyramid for services https://github.com/reddit/baseplate.py
-
is flask only for "smaller" projects or it can also be used for large scalable projects ?
Reddit is built on neither Flask nor Django. The old monolith predates Flask and Django and is built on its own framework. Our new microservices are built on our Baseplate.py framework.
-
Evolving Reddit’s ML Model Deployment and Serving Architecture
Minsky is an internal baseplate.py (Reddit’s python web services framework) thrift service owned by Reddit’s Machine Learning team that serves data or derivations of data related to content relevance heuristics — such as similarity between subreddits, a subreddits topic or a users propensity for a given subreddit — from various data stores such as Cassandra or in process caches. Clients of Minsky use this data to improve Redditor’s experiences with the most relevant content. Over the last few years a set of new ML capabilities, referred to as Gazette, were built into Minsky. Gazette is responsible for serving ML model inferences for personalization tasks along with configuration based schema resolution and feature fetching / transformation.
-
Deadline Budget Propagation for Baseplate.py
Baseplate is implemented in Python and Go, and although they share the same main functionality, smaller features differ between the two. One such feature that was previously on the Go implementation but not Python was deadline budget propagation, which passes on the remaining timeout available from the initial client request all the way through the server and any other requests that may follow. The lack of this feature in Baseplate.py meant that many resources were being wasted by servers doing unnecessary work, despite clients no longer awaiting their response due to timeout.
-
Solving The Three Stooges Problem
In order to make this solution work, you’ll need a web stack that can handle many concurrent requests. Reddit’s stack for most microservices is Python 3, Baseplate, and gevent. Django/Flask also work well when run with gevent. gevent is a Python library that transparently enables your microservice to handle high concurrency and I/O without requiring changes to your code. It is the secret sauce that allows you to run tens of thousands of pseudo-threads called greenlets (one per concurrent request) on a small number of instances. It allows for threads handling concurrent duplicate requests to be enqueued while waiting to acquire the lock, and then for those queues to be drained as threads acquire the lock and execute serially, all without exhausting the thread pool.
redsync
Posts with mentions or reviews of redsync.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-03-30.
-
How can I lock distributed resources in multi-step HTTP requests?
I value and rely on the guarantees of the Redlock algorithm for all my distributed locking.
-
Looking for an elegant solution to a synchronization problem in a Golang backend
I thought about using a distributed lock (https://github.com/go-redsync/redsync). Before querying the DB, I would first try to acquire the lock for the resource with id `user_id/resource_id`. Now, the requests for different combinations of `user_id/resource_id` can be done in parallel, but the requests for the same combination of `user_id/resource_id` will be done sequentially.
-
Redis distributed locks In MYSQL
A client implementation is found here: https://github.com/go-redsync/redsync
- How to handle simultaneous array modification fron database?
-
Go cron with distributed lock. Run your cron job once at a time in multiple instances of your service
Thanks for highlighting this. Yes, I know about this discussion. And I more agree with Salvatore points. There is also Disclaimer in redsync github page I have linked in my readme file. You definitely should understand how it works before using it.
-
How to do distributed locking(mysql vs redis)?
Redis : https://redis.io/topics/distlock
-
Serialize Access to a shared resource in distributed systems with DLM (Distributed Lock Manager)
2- We have several implementations of the RedLock algorithm in different languages Redis Official website. and you do not need to implement it again.
-
Need Help. How to design proper processing pipeline (with rabbitmq)?
https://redis.io/topics/distlock I haven't used Redis specifically for locking, but it appears to be a popular use case. Or you could probably use the locking mechanism of your database to keep track of a shared store of user IDs.
- What is the best way to implement time-based locks on resource editing?
-
Solving The Three Stooges Problem
This decorator works by forcing the flow of control through a distributed lock on the request hash, ensuring that no two duplicate requests can proceed concurrently. In this example, we’ve used Pottery’s implementation of Redlock (backed by shared Redis instances) which implements Python’s excellent threading.Lock API as closely as is feasible.
What are some alternatives?
When comparing baseplate.py and redsync you can also consider the following projects:
xhtml2pdf - A library for converting HTML into PDFs using ReportLab
pottery - Redis for humans. 🌎🌍🌏
Apache Thrift - Apache Thrift
Pyramid - Pyramid - A Python web framework
rtv - Browse Reddit from your terminal
cornice - Build Web Services with Pyramid.
SQLAlchemy - The Database Toolkit for Python
Sentry - Developer-first error tracking and performance monitoring