Keep the Monolith, but Split the Workloads

• packwerk

  16 1,481 7.6 Ruby

  Good things come in small packages.

  

  Yep, that article is about very similar concepts but grounded in Spring as the framework.

  I like what they do around package imports and it looks a lot like what we do at incident.io, with some rules about which packages can import what.

  For people in the Ruby world who want a similar solution, Shopify provide an open-source framework called packwerk that is designed just for this:

  https://github.com/Shopify/packwerk

• oban

  27 3,017 9.3 Elixir

  💎 Robust job processing in Elixir, backed by modern PostgreSQL and SQLite3

  > Bad code in a specific part of the codebase bringing down the whole app, as in our November incident.

  This is a non-issue if you're using a Elixir/Erlang monolith given its fault tolerant nature.

  The noisy neighbour issue (resource hogging) is still something you need to manage though. If you use something like Oban[1] (for background job queues and cron jobs), you can set both local and global limits. Local being the current node, and global the cluster.

  Operating in a shared cluster (vs split workload deployments) give you the benefit of being much more efficient with your hardware. I've heard many stories of massive infra savings due to moving to an Elixir/Erlang system.

  1. https://github.com/sorentwo/oban

• WorkOS

  workos.com

  sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• Gin

  152 75,354 8.5 Go

  Gin is a HTTP web framework written in Go (Golang). It features a Martini-like API with much better performance -- up to 40 times faster. If you need smashing performance, get yourself some Gin.

  

  CustomRecoveryWithWriter returns a middleware for a given writer that recovers from any panics and calls the provided handle func to handle it.

  (https://github.com/gin-gonic/gin/blob/master/recovery.go)

• django-rq

  15 1,760 7.0 Python

  A simple app that provides django integration for RQ (Redis Queue)

  

• rq

  27 9,503 8.3 Python

  Simple job queues for Python

  

  We use RQ[0], it has Redis as a dependency. It’s pretty straightforward and we’re very happy with it. If you are using Django you may want to look at Django RQ[1] as well. RQ has built in scheduling capabilities these days, but historically it did not so we used (and still use) RQ Scheduler[2] which I think still has some advantages over the built in stuff.

  [0] https://python-rq.org/

• rq-scheduler

  4 1,384 2.2 Python

  A lightweight library that adds job scheduling capabilities to RQ (Redis Queue)

  

• django-postgres-queue

  4 115 0.0 Python

  A task queue for django

  If you're using PostgreSQL, then

  django-postgres-queue: https://github.com/gavinwahl/django-postgres-queue

  procrastinate: https://github.com/procrastinate-org/procrastinate/

• InfluxDB

  www.influxdata.com

  sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• procrastinate

  7 725 9.6 Python

  PostgreSQL-based Task Queue for Python

  

  If you're using PostgreSQL, then

  django-postgres-queue: https://github.com/gavinwahl/django-postgres-queue

  procrastinate: https://github.com/procrastinate-org/procrastinate/

Suggest a related project