Open-source projects categorized as Queue | Edit details

Top 23 Queue Open-Source Projects

  • GitHub repo nsq

    A realtime distributed messaging platform

    Project mention: For those running Go in production at scale, what do you use for distributed task queues? | reddit.com/r/golang | 2021-06-04

    Here is link number 1 - Previous text "NSQ"

  • GitHub repo Redisson

    Redisson - Redis Java client with features of In-Memory Data Grid. Over 50 Redis based Java objects and services: Set, Multimap, SortedSet, Map, List, Queue, Deque, Semaphore, Lock, AtomicLong, Map Reduce, Publish / Subscribe, Bloom filter, Spring Cache, Tomcat, Scheduler, JCache API, Hibernate, MyBatis, RPC, local cache ...

  • GitHub repo Sidekiq

    Simple, efficient background processing for Ruby

    Project mention: Corre tus tareas recurrentes con Sidekiq | dev.to | 2021-06-07

    2021-06-07T15:54:38.710Z 62465 TID-ownzi9r98 INFO: Running in ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-darwin20] 2021-06-07T15:54:38.710Z 62465 TID-ownzi9r98 INFO: See LICENSE and the LGPL-3.0 for licensing details. 2021-06-07T15:54:38.710Z 62465 TID-ownzi9r98 INFO: Upgrade to Sidekiq Pro for more features and support: http://sidekiq.org 2021-06-07T15:54:38.723Z 62465 TID-ownzi9r98 INFO: Loading Schedule 2021-06-07T15:54:38.723Z 62465 TID-ownzi9r98 INFO: Schedule empty! Set Sidekiq.schedule 2021-06-07T15:54:38.724Z 62465 TID-ownzi9r98 INFO: Schedules Loaded 2021-06-07T15:54:38.734Z 62465 TID-ownzi9r98 INFO: Reloading Schedule 2021-06-07T15:54:38.734Z 62465 TID-ownzi9r98 INFO: Loading Schedule 2021-06-07T15:54:38.790Z 62465 TID-ownzi9r98 INFO: Scheduling hello_world {"cron"=>"0 * * * * *", "class"=>"HelloWorldWorker", "queue"=>"default"} 2021-06-07T15:54:38.791Z 62465 TID-ownzi9r98 INFO: Schedules Loaded 2021-06-07T15:54:38.791Z 62465 TID-ownzi9r98 INFO: Starting processing, hit Ctrl-C to stop 2021-06-07T15:59:37.940Z 62990 TID-ov0v1hmpo HelloWorldWorker JID-4a465f9e9646157552de7f5c INFO: start Hello world 2021-06-07T15:59:37.940Z 62990 TID-ov0v1hmpo HelloWorldWorker JID-4a465f9e9646157552de7f5c INFO: done: 0.0 sec 2021-06-07T15:59:37.940Z 62990 TID-ov0v1hmpo HelloWorldWorker JID-64dc6b6cb06ebf580c3faae6 INFO: start Hello world 2021-06-07T15:59:37.940Z 62990 TID-ov0v1hmpo HelloWorldWorker JID-64dc6b6cb06ebf580c3faae6 INFO: done: 0.0 sec

  • GitHub repo bull

    Premium Queue package for handling distributed jobs and messages in NodeJS.

    Project mention: Run another function alongside express | reddit.com/r/node | 2021-05-23
  • GitHub repo Resque

    Resque is a Redis-backed Ruby library for creating background jobs, placing them on multiple queues, and processing them later.

    Project mention: Autoscaling Redis applications on Kubernetes 🚀🚀 | dev.to | 2021-03-02

    Redis Lists are quite versatile and used as the backbone for implementing scalable architectural patterns such as consumer-producer (based on queues), where producer applications push items into a List, and consumers (also called workers) process those items. Popular projects such as resque, sidekiq, celery etc. use Redis behind the scenes to implement background jobs.

  • GitHub repo rq

    Simple job queues for Python

    Project mention: Quick and dirty mock service with Starlette | dev.to | 2021-06-07

    An alternative to Celery is RQ, but it has similar problems and requires running Redis along with a separate worker process.

  • GitHub repo agenda

    Lightweight job scheduling for Node.js

    Project mention: 3 Task Scheduling Packages for Node.js | dev.to | 2021-05-28


  • GitHub repo Hangfire

    An easy way to perform background job processing in your .NET and .NET Core applications. No Windows Service or separate process required

    Project mention: ASP.NET Core API Checklist | dev.to | 2021-06-13


  • GitHub repo boltons

    🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.

  • GitHub repo machinery

    Machinery is an asynchronous task queue/job queue based on distributed message passing.

    Project mention: For those running Go in production at scale, what do you use for distributed task queues? | reddit.com/r/golang | 2021-06-04

    Machinery (https://github.com/RichardKnop/machinery) seems to be the most popular out of the box solution. I've played with this in the past and it's worked pretty well but I'm not sure how it handles under serious volume.

  • GitHub repo CAP

    Distributed transaction solution in micro-service base on eventually consistency, also an eventbus with Outbox pattern

    Project mention: 14 .NET packages I always recommend | reddit.com/r/dotnet | 2021-03-22

    Correct but I would actually consider CAP instead. The primary reason is that CAP implements the outbox pattern out-of-box where you would need to implement this yourself with Hangfire.

  • GitHub repo Delayed::Job

    Database based asynchronous priority queue system -- Extracted from Shopify

    Project mention: A quick look at background jobs in Ruby | dev.to | 2021-03-11

    There are a few of popular systems. A few need a database, such as Delayed::Job, while others prefer Redis, such as Resque and Sidekiq.

  • GitHub repo Jocko

    Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)

    Project mention: ZooKeeper-free Kafka is out. First Demo | news.ycombinator.com | 2021-03-30

    Reminded me of this project: https://github.com/travisjeffery/jocko

    Kafka implemented in Go without needing Zookeeper.

  • GitHub repo PHP AMQP

    The most widely used PHP client for RabbitMQ

    Project mention: Introducing my first open source project: Roger, Rabbit: A RabbitMQ client and drop-in replacement for streadway/amqp with automatic redials, method middleware, and more! | reddit.com/r/golang | 2021-04-21

    If you are not familiar with RabbitMQ or the streadway/amqp driver, you can try it out using the official tutorials. Just replace any "github.com/streadway/amqp" import statements with "github.com/peake100/rogerRabbit-go/pkg/amqp". If you find a tutorial this does not work for, please open a PR!

  • GitHub repo MassTransit

    Distributed Application Framework for .NET

  • GitHub repo huey

    a little task queue for python

  • GitHub repo Sucker Punch

    Sucker Punch is a Ruby asynchronous processing library using concurrent-ruby, heavily influenced by Sidekiq and girl_friday.

  • GitHub repo data-structures

    A collection of powerful data structures (by williamfiset)

    Project mention: Simple Kruskal implementation for java? | reddit.com/r/algorithms | 2021-05-15

    The Union-Find (DSU) algorithm is actually very simple, here is an example implementation in around 100 lines including comments, and you don't need all of it.

  • GitHub repo Chronicle Queue

    Micro second messaging that stores everything to disk

  • GitHub repo NetMQ

    A 100% native C# implementation of ZeroMQ for .NET

    Project mention: Question: Best way to do interprocess communication? | reddit.com/r/csharp | 2021-04-24

    Lots of answers here about named pipes but there are less verbose and time consuming ways to. Having just done this, for a very similar reason to yours, I highly suggest NetMQ, https://github.com/zeromq/netmq

  • GitHub repo kombu

    Messaging library for Python.

  • GitHub repo EasyNetQ

    An easy to use .NET API for RabbitMQ

  • GitHub repo Sneakers

    A fast background processing framework for Ruby and RabbitMQ

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2021-06-13.


What are some of the best open-source Queue projects? This list will help you:

Project Stars
1 nsq 19,753
2 Redisson 16,678
3 Sidekiq 11,013
4 bull 10,745
5 Resque 8,911
6 rq 7,762
7 agenda 7,538
8 Hangfire 6,765
9 boltons 5,480
10 machinery 5,323
11 CAP 4,864
12 Delayed::Job 4,627
13 Jocko 4,287
14 PHP AMQP 3,824
15 MassTransit 3,626
16 huey 3,539
17 Sucker Punch 2,538
18 data-structures 2,419
19 Chronicle Queue 2,357
20 NetMQ 2,243
21 kombu 2,160
22 EasyNetQ 2,140
23 Sneakers 2,116