ratus VS BeanstalkD

The Go ecosystem has many brilliant distributed task queue libraries, but sometimes we still have to settle for some bloated cross-platform solution in order to work with components written in other languages. To solve this problem, we created Ratus, which is a service, not a library. It translated concepts of distributed task queues into a set of resources that conform to REST principles and provides an easy-to-use HTTP API. Ratus is implemented in Go, and comes with a Go client library. Asynchronous task queues are typically used for long-running background tasks, so the overhead of the HTTP API is not significant compared to the time spent by the tasks themselves. On the other hand, the HTTP-based RESTful API can be easily accessed by all languages without using dedicated client libraries.

To solve this problem, we created Ratus, which is a service, not a library. It translated concepts of distributed task queues into a set of resources that conform to REST principles and provides an easy-to-use HTTP API. Ratus is implemented in Go, and comes with a Go client library. The current default storage engine is MongoDB (in-memory store coming soon), we will also try to integrate existing Go task queue libraries.

I've been working on https://github.com/hyperonym/ratus which is a RESTful asynchronous task queue server, it also comes with a Go client library.

There's beanstalkd, it has a few Python libraries and it works out of the box with ActiveJob via Backburner.

https://beanstalkd.github.io/

Messaging/Queueing Systems (Amazon SQS, RabbitMQ, Beanstalkd)

Not when a queue is involved. IME trying to replicate something like beanstalkd (https://beanstalkd.github.io/) in postgres is asking for trouble for anything but trivial workloads.

If you're measuring throughput in jobs/s, use a real work queue.

Before Kafka was a standard, I created a go library for beanstalkd that act like an RPC.

Then there are queue libraries like beanstalkd, RabbitMQ or built-in features like queues from Laravel. These will probably get you quicker to your goal then trying the process managing route.

The only task queue I loved was beanstalkd -- it's beautifully written and highly performant. Starting it takes seconds and it's been running for a decade:

https://beanstalkd.github.io/

It's not rust (it's C), but beanstalkd is a pretty incredible work queue that processes millions of jobs a day (10K+/s at peak) for my company. I know there are a few rust drivers available.