Keep the Monolith, but Split the Workloads

• packwerk

  16 1,528 7.3 Ruby

  Good things come in small packages.

  

Yep, that article is about very similar concepts but grounded in Spring as the framework.

I like what they do around package imports and it looks a lot like what we do at incident.io, with some rules about which packages can import what.

For people in the Ruby world who want a similar solution, Shopify provide an open-source framework called packwerk that is designed just for this:

https://github.com/Shopify/packwerk

• Scout Monitoring

  www.scoutapm.com featured

  Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

  

• oban

  27 3,126 9.3 Elixir

  💎 Robust job processing in Elixir, backed by modern PostgreSQL and SQLite3

> Bad code in a specific part of the codebase bringing down the whole app, as in our November incident.

This is a non-issue if you're using a Elixir/Erlang monolith given its fault tolerant nature.

The noisy neighbour issue (resource hogging) is still something you need to manage though. If you use something like Oban[1] (for background job queues and cron jobs), you can set both local and global limits. Local being the current node, and global the cluster.

Operating in a shared cluster (vs split workload deployments) give you the benefit of being much more efficient with your hardware. I've heard many stories of massive infra savings due to moving to an Elixir/Erlang system.

1. https://github.com/sorentwo/oban

• Gin

  154 76,246 8.8 Go

  Gin is a HTTP web framework written in Go (Golang). It features a Martini-like API with much better performance -- up to 40 times faster. If you need smashing performance, get yourself some Gin.

  

CustomRecoveryWithWriter returns a middleware for a given writer that recovers from any panics and calls the provided handle func to handle it.

(https://github.com/gin-gonic/gin/blob/master/recovery.go)

• django-rq

  15 1,785 7.2 Python

  A simple app that provides django integration for RQ (Redis Queue)

  

• rq

  27 9,597 8.5 Python

  Simple job queues for Python

  

We use RQ[0], it has Redis as a dependency. It’s pretty straightforward and we’re very happy with it. If you are using Django you may want to look at Django RQ[1] as well. RQ has built in scheduling capabilities these days, but historically it did not so we used (and still use) RQ Scheduler[2] which I think still has some advantages over the built in stuff.

[0] https://python-rq.org/

• rq-scheduler

  4 1,399 0.0 Python

  A lightweight library that adds job scheduling capabilities to RQ (Redis Queue)

  

• django-postgres-queue

  4 116 0.0 Python

  A task queue for django

If you're using PostgreSQL, then

django-postgres-queue: https://github.com/gavinwahl/django-postgres-queue

procrastinate: https://github.com/procrastinate-org/procrastinate/

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• procrastinate

  7 763 9.6 Python

  PostgreSQL-based Task Queue for Python

  

If you're using PostgreSQL, then

django-postgres-queue: https://github.com/gavinwahl/django-postgres-queue

procrastinate: https://github.com/procrastinate-org/procrastinate/

Suggest a related project