Solving The Three Stooges Problem

This page summarizes the projects mentioned and recommended in the original post on /r/RedditEng

Our great sponsors
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern identity platform for B2B SaaS
  • SaaSHub - Software Alternatives and Reviews
  • baseplate.py

    reddit's python service framework

  • In order to make this solution work, you’ll need a web stack that can handle many concurrent requests. Reddit’s stack for most microservices is Python 3, Baseplate, and gevent. Django/Flask also work well when run with gevent. gevent is a Python library that transparently enables your microservice to handle high concurrency and I/O without requiring changes to your code. It is the secret sauce that allows you to run tens of thousands of pseudo-threads called greenlets (one per concurrent request) on a small number of instances. It allows for threads handling concurrent duplicate requests to be enqueued while waiting to acquire the lock, and then for those queues to be drained as threads acquire the lock and execute serially, all without exhausting the thread pool.

  • pottery

    Redis for humans. 🌎🌍🌏

  • Pottery — Pythonic Redis utilities, including a distributed lock

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • redsync

    Distributed mutual exclusion lock using Redis for Go

  • This decorator works by forcing the flow of control through a distributed lock on the request hash, ensuring that no two duplicate requests can proceed concurrently. In this example, we’ve used Pottery’s implementation of Redlock (backed by shared Redis instances) which implements Python’s excellent threading.Lock API as closely as is feasible.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts