The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning. Learn more →
Top 13 Job Queue Open-Source Projects
-
-
-
SurveyJS
Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.
-
-
-
-
croner
Trigger functions or evaluate cron expressions in JavaScript or TypeScript. No dependencies. Most features. Node. Deno. Bun. Browser.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
-
-
Project mention: Prioritizing Concurrent Requests: Queuing system to handle distributed processes and messages with NodeJS and Bull | dev.to | 2024-03-15To implement the solution with a queue, I used a package called 'Bull' (https://github.com/OptimalBits/bull). It's a library that helps with distributed job control, providing some very useful solutions for this type of work, such as background job processing, queues with priorities (FIFO, LIFO, and others), among other features. 'Bull' uses Redis for queue storage, so if your application crashes for any reason, once it's back online, it will continue executing the processes that are in the queue. In our case, we'll use the FIFO (First in, first out) queue solution, meaning priority based on arrival order.
Graphile worker has been great for me because it's a library that works with Postgres that allows me to queue jobs and execute them on the server without adding too many additional layers of complexity for being able to accomplish async tasks. (I'm aware of how popular bull is, but I don't want to add another data-store only for async tasks)
> Can you define "low throughput"?
<1000 messages per minute
Not saying SKIP LOCKED can't work with that many. But you'll probably want to do something better.
FWIW, Que uses advisory locks [1]
[1] https://github.com/que-rb/que
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Job Queues related posts
- Prioritizing Concurrent Requests: Queuing system to handle distributed processes and messages with NodeJS and Bull
- Object Narrowing in Typescript with Graphile Worker
- Job Schedulers for Node: Bull or Agenda?
- Croner – Cron for JavaScript and TypeScript
- Instrumentation for Event Driven
- Redis master/slave setup on Kubernetes throwing error: BRPOPLPUSH { ReplyError: MOVED 2651
- Are there any generally accepted standards for inter-microservice communication? Or does everyone just go it their own?
-
A note from our sponsor - WorkOS
workos.com | 23 Apr 2024
Index
What are some of the best open-source Job Queue projects? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | bull | 15,018 |
| 2 | kue | 9,435 |
| 3 | The gist | 5,182 |
| 4 | Bee-Queue | 3,610 |
| 5 | Que | 2,284 |
| 6 | croner | 1,862 |
| 7 | rsmq | 1,703 |
| 8 | node-resque | 1,331 |
| 9 | RedisSMQ | 569 |
| 10 | better-queue | 482 |
| 11 | idoit | 72 |
| 12 | Conveyor MQ | 50 |
| 13 | Qedis | 0 |
Sponsored
The modern identity platform for B2B SaaS
The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
workos.com