convoy VS pg-boss

Yup. Even DigitalOcean still doesn't have webhooks despite several user requests see here [0]

My business is building a Webhooks Gateway for delivering webhooks at scale - Convoy [1]. We use Convoy Cloud to power Nohooks. The goal is to show what's possible with Convoy so more providers can easily provide webhooks.

[0] https://github.com/digitalocean/api-v2/issues/14

[1] https://github.com/frain-dev/convoy

Speaking with platform engineers at events, in various online communities and even within the Convoy community validated the notion that managing webhooks in production is not always easy and can require a lot of effort. Platform engineers believe strongly in frictionless self-service developer experience, and solving for efficiency to quickly provide business value. Let’s take a look at why some haven't adopted a webhooks gateway such as Convoy to manage webhook events and integrations yet and what we are doing to change that.

If you feel like this article helped you understand Webhooks better! I would be super happy if you could give us a star! And let me also know in the comments ❤️ https://github.com/frain-dev/convoy

This is precisely what Convoy does. See here: https://github.com/frain-dev/convoy. If you have any questions, the team is reachable on Slack here.

My initial reaction would be to use a self-hosted webhook relay (e.g. Convoy) that will queue / retry until my internet is back online. However, I'm a bit hesitant to throw a large dependency in the mix in the name of reliability. My other thought I had was setting up some form of load balancer, that will initially attempt to be a passthrough straight to Discord, and only if that fails, then get a queue / retry software involved?

You should check out getconvoy.io, if you're interested in our cloud platform please reach to us at [email protected]

For running queues on Postgres with Node.js backend(s), I highly recommend https://github.com/timgit/pg-boss. I'm sure it has it scale limits. But if you're one of the 90% of the apps that never needs any kind of scale that a modern server can't easily handle then it's fantastic. You get transactional queueing of jobs, and it automatically handles syncing across multiple job processing servers using Postgres locks.

Jobs use pg-boss, a postgres extension, to queue and run tasks under the hood.

If you don't want to roll your own, look into https://github.com/timgit/pg-boss

I saw this https://github.com/timgit/pg-boss but it's more for jobs than for message with multiple consumers (having their own progress offset).

The best I've used till now. Has all kind of features and really great when you have a postgres dB in your stack. https://github.com/timgit/pg-boss

Check out these; - https://github.com/mitranim/posterus - https://github.com/timgit/pg-boss - https://github.com/FirebaseExtended/firebase-queue - https://www.npmjs.com/package/rabbit-queue

You can use the simplest option than implement a new service. Keep in mind that every running system can be a job scheduler, you can just use nodejs worker threads, Redis, or even your DB as a job scheduler, check PGBoss for example.

Both! For context, we're currently using https://github.com/timgit/pg-boss as a task queue on top of postgres and it works great. No need to complicate things with Redis. I believe it's quite straightforward to implement a task queue on top of postgres using the SKIP LOCKED functionality.