good_job VS Delayed::Job

This is the most direct competitor of good_job in my opinion.

Once we are done with default gems, should we look into something we usually use? That's jwt because we need session tokens for our API. Next comes our one and only sidekiq. For a long period of time it was the best in town solution for background jobs. Now we could also consider solid_queue or good_job. In development and testing groups we need rspec-rails, factory_bot_rails and ffaker. Dealing with money? Start doing it properly from the beginning! Do not forget to install money-rails. Once everything is added to the Gemfile do not forget to trigger bundle install.

In the world of Ruby, GoodJob [0] has been doing a _good job_ so far.

[0] - https://github.com/bensheldon/good_job

For Rails apps, you can do this using the ActiveJob interface via

https://github.com/bensheldon/good_job

Had it in production for about a quarter and it’s worked well.

Idk about pgagent but any table is a resilient queue with the multiple locks available in pg along with some SELECT pg_advisory_lock or SELECT FOR UPDATE queries, and/or LISTEN/NOTIFY.

Several bg job libs are built around native locking functionality

> Relies upon Postgres integrity, session-level Advisory Locks to provide run-once safety and stay within the limits of schema.rb, and LISTEN/NOTIFY to reduce queuing latency.

https://github.com/bensheldon/good_job

> |> lock("FOR UPDATE SKIP LOCKED")

https://github.com/sorentwo/oban/blob/8acfe4dcfb3e55bbf233aa...

The suggestion from /u/tofus is a good one. If you are already using redis as your ActionCable adapter I would use sidekiq. If not and you're using postgres I would consider https://github.com/bensheldon/good_job

Probably worth mentioning that aside from delayed_job there are at least two more modern alternatives backed by the DB: Que and good_job.

Good article. Sidekiq is a good, well respected too. However if you are starting out I would recommend not using it, and instead choosing a DB based queue system. We have great success with que, but there are others like good_job.

Sidekiq Pro is great, we're paying for it! 10k a year I think.

But for people who are interested in alternatives, I'd also suggest Good Job (runs on Postgresql).

https://github.com/bensheldon/good_job

I'm the GoodJob author. Here's the class that is responsible for implementing Postgres's LISTEN/NOTIFY functionality in GoodJob:

https://github.com/bensheldon/good_job/blob/10e9d9b714a668dc...

That's heavily inspired by Rail's Action Cable (websockets) Adapter for Postgres, which is a bit simpler and easier to understand:

https://github.com/rails/rails/blob/be287ac0d5000e667510faba...

Briefly, it spins up a background thread with a dedicated database connection and doings a blocking Postgres LISTEN query returns results, and then it forwards the result to other subscribing objects.

I included this in the github issue but we had coverband induced log spam, it got fixed and suddenly we hit the bug. I did find this from someone with a similar issue but there was no conclusion.

So how do we trigger such a long-running process from a Rails request? The first option that comes to mind is a background job run by some of the queuing back-ends such as Sidekiq, Resque or DelayedJob, possibly governed by ActiveJob. While this would surely work, the problem with all these solutions is that they usually have a limited number of workers available on the server and we didn’t want to potentially block other important background tasks for so long.

Several gems support job queues and background processing in the Rails world — Delayed Job and Sidekiq being the two most popular ones.

Rails 7 support only just got added https://github.com/collectiveidea/delayed_job/pull/1161

Back in the day, before Sidekiq and such, we used Delayed Job https://github.com/collectiveidea/delayed_job

There are a few of popular systems. A few need a database, such as Delayed::Job, while others prefer Redis, such as Resque and Sidekiq.

I always wad a fan of Sidekiq, but recently I wanted to remove redis as a dependency. I ended up going back to classic delayed_job. My reason for that was I also didn't want dependency with Postgres, as I like to use sqlite on development and Postgres on production. The best solution I've seen is Que, which works with Postgres. It is complete, has all the features you need.

Ruby gets hammered a lot for its green threads and no real concurrency yet Ruby libraries such as Resque, Delayed Job and Sidekiq are some of the most popular choices to run background jobs in the industry. When you have a huge project and millions of requests and hundreds of thousands of operations to perform, reach out to one of these solutions which are feature complete and you don’t have to reinvent the wheel as its complex piece of software to implement and requires thousands of human hours. While they help you scale sometimes they are not what you are looking for or need as they are meant for a scale of 10000s of jobs per second and what you need is a background processing for few jobs that can be handled in memory without a dependency of a queue such as Redis. If you look at the benchmark numbers from Sidekiq they are for 100K jobs because that is the scale it is meant to be though one can use it for running a few hundred complex jobs as well where you require queue management and supervision.

Before we dive in, I wanted to go over my background real quick for context. I'm mostly a Rails developer professionally, and I am on the 'Infrastructure' team at work, so I integrate closely with Heroku and am pretty familiar with their product and offerings. Given my Rails background, this article will focus on deploying a web app with at least one background job process. For my hackathon app, I had a web process running Puma and a worker process running DelayedJob.