Faktory VS BeanstalkD

Mike Perham (the sidekiq maintainer) also maintains the less well known faktory[0] which is language agnostic and has runners for both Ruby and Python

[0] https://github.com/contribsys/faktory

Zen and the Art of Motorcycle Maintenance's entire thesis is "What is Quality?" How do you define it? How does it come about?

You can still get software quality but you have to be willing to devote time and effort to it. The binary for my modern, commercial background job engine written in Go, Faktory, is 5MB in size.

https://github.com/contribsys/faktory/releases/tag/v1.8.0

I know when I see an iOS app that is 5-10MB in size, I know it was crafted by someone who cares.

My recent queue foray put me on the scent of Faktory, a language-agnostic queue server made by Sidekiq's author. I noticed there wasn't a good PHP client (the one linked in the docs is pretty old), so I decided to build one.

At work we use https://github.com/contribsys/faktory

Have you heard of faktory before? Made by the sidekiq guy and allows you to d a similar thing.

I’ve started using Faktory with the Faktory Worker Python it also supports workers in any language.

Try https://github.com/contribsys/faktory which is written in go but you interact with it as a service.

There is faktory, from the author of Sidekiq, which is language-agnostic (the server is written in Go).

There's beanstalkd, it has a few Python libraries and it works out of the box with ActiveJob via Backburner.

https://beanstalkd.github.io/

Messaging/Queueing Systems (Amazon SQS, RabbitMQ, Beanstalkd)

Not when a queue is involved. IME trying to replicate something like beanstalkd (https://beanstalkd.github.io/) in postgres is asking for trouble for anything but trivial workloads.

If you're measuring throughput in jobs/s, use a real work queue.

Before Kafka was a standard, I created a go library for beanstalkd that act like an RPC.

Then there are queue libraries like beanstalkd, RabbitMQ or built-in features like queues from Laravel. These will probably get you quicker to your goal then trying the process managing route.

The only task queue I loved was beanstalkd -- it's beautifully written and highly performant. Starting it takes seconds and it's been running for a decade:

https://beanstalkd.github.io/

It's not rust (it's C), but beanstalkd is a pretty incredible work queue that processes millions of jobs a day (10K+/s at peak) for my company. I know there are a few rust drivers available.