Devious SQL: Message Queuing Using Native PostgreSQL

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • Scout APM - Less time debugging, more time building
  • SonarQube - Static code analysis for 29 languages.
  • SaaSHub - Software Alternatives and Reviews
  • django-postgres-queue

    A task queue for django

  • pg-boss

    Queueing jobs in Node.js using PostgreSQL like a boss

  • Scout APM

    Less time debugging, more time building. Scout APM allows you to find and fix performance issues with no hassle. Now with error monitoring and external services monitoring, Scout is a developer's best friend when it comes to application development.

  • Que

    A Ruby job queue that uses PostgreSQL's advisory locks for speed and reliability.

    Implementations that use advisory locks like https://github.com/que-rb/que are much more efficient (atleast when I last tested) and will easily reach 10k job/s on even very modest hardware.

    There is a Go port of Que but you can also easily port it to any language you like. I have a currently non-OSS implementation in Rust that I might OSS someday when I have time to clean it up.

  • worker

    High performance Node.js/PostgreSQL job queue (also suitable for getting jobs generated by PostgreSQL triggers/functions out into a different work queue)

    We're currently keeping it as simple as possible. We just use a table and this query that runs every second: `DELETE FROM job_queue RETURNING type, info;`. Works great and reliably so far. I think our current volume is on the order of 100-1000 req/sec.

    We'll likely switch to graphile-worker when we need more performance, but we're all about avoiding premature optimization. That library has been benchmarked to handle 10k req/sec: https://github.com/graphile/worker#performance

  • redislite

    Redis in a python module.

    If you are developing on Linux, rq + redis is probably the simplest message queueing system around. Redis can even be installed from pip,

    https://pypi.org/project/redis/

    or run within python,

    https://github.com/yahoo/redislite

    it's pretty much the easiest thing to deploy.

  • OddJob

    OddJob - A Simple Job Scheduler for .NET

    Probably not the one you're thinking of, but I wrote something like this for C# once upon a time called OddJob [0]. If nothing else it shows decent patterns for a job queue that is friendly towards multiple producers/consumers.

    [0] - https://github.com/to11mtm/oddjob/tree/cleaning-aug-19

  • SonarQube

    Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts