Queue

Open-source projects categorized as Queue
Edit details
Language: + Go + C# + Ruby + Python + TypeScript + PHP + Java + JavaScript + Swift
Topics: Redis Rabbitmq Data structures Stack Python
The modern identity platform for B2B SaaS
The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
workos.com
Sponsored

Top 23 Queue Open-Source Projects

  • nsq

    A realtime distributed messaging platform

    • Project mention: NSQ: Open-source realtime distributed messaging, billions of messages / day | news.ycombinator.com | 2024-01-09

  • Redisson

    Redisson - Easy Redis Java client and Real-Time Data Platform. Sync/Async/RxJava/Reactive API. Over 50 Redis based Java objects and services: Set, Multimap, SortedSet, Map, List, Queue, Deque, Semaphore, Lock, AtomicLong, Map Reduce, Bloom filter, Spring Cache, Tomcat, Scheduler, JCache API, Hibernate, RPC, local cache ...

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • gods

    GoDS (Go Data Structures) - Sets, Lists, Stacks, Maps, Trees, Queues, and much more

    • Project mention: How do you go about the lack of built in data structure like stack, queue for LeetCode | /r/golang | 2023-05-24

    for len(stack) > 0 { n := len(stack) - 1 // Top element fmt.Print(stack[n]) stack = stack[:n] // Pop } ``` Another solution would be to import a package like https://github.com/emirpasic/gods

  • bull

    Premium Queue package for handling distributed jobs and messages in NodeJS.

    • Project mention: Prioritizing Concurrent Requests: Queuing system to handle distributed processes and messages with NodeJS and Bull | dev.to | 2024-03-15

    To implement the solution with a queue, I used a package called 'Bull' (https://github.com/OptimalBits/bull). It's a library that helps with distributed job control, providing some very useful solutions for this type of work, such as background job processing, queues with priorities (FIFO, LIFO, and others), among other features. 'Bull' uses Redis for queue storage, so if your application crashes for any reason, once it's back online, it will continue executing the processes that are in the queue. In our case, we'll use the FIFO (First in, first out) queue solution, meaning priority based on arrival order.

  • Sidekiq

    Simple, efficient background processing for Ruby

    • Project mention: solid_queue alternatives - Sidekiq and good_job | libhunt.com/r/solid_queue | 2024-04-21

    I'd say Sidekiq is the top competitor here.

  • rq

    Simple job queues for Python

    • Project mention: Redis Re-Implemented with SQLite | news.ycombinator.com | 2024-04-14

    That's pretty cool. Reckon it would work with existing code that calls Redis over the wire for RQ?

      https://python-rq.org

  • Resque

    Resque is a Redis-backed Ruby library for creating background jobs, placing them on multiple queues, and processing them later.

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo

  • agenda

    Lightweight job scheduling for Node.js

    • Project mention: Next.js vs Node.js: A Modern Contrast | dev.to | 2024-04-12

    While many ways to write scalable Node.js applications exist, all narrow down to the underlying architecture. However, common hurdles in Node.js include serving static files, task scheduling, and utilizing available resources. To combat these, you can use a tactical approach. Because serving static assets with Node.js takes CPU time, a proxy content delivery network like CloudFront should help fix it. Recurring tasks need to fire events, which isn’t reliable with setTimeout or setInterval. As a solution, you can use a task scheduler framework like agenda.js. To handle resource utilization, you can use the cluster feature in Node.js, which listens to ports, accepts new connections, and distributes them across workers with built-in smarts to avoid overloading the process.

  • Hangfire

    An easy way to perform background job processing in .NET and .NET Core applications. No Windows Service or separate process required

    • Project mention: Deno Cron | news.ycombinator.com | 2023-11-29

    Unpopular opinion incoming... What I see is yet another way that the backend JS world is finally achieving something .NET had years ago[0].

    Node/Deno/Bun/etc. + npm sounds super straightforward at first glance (and it is at first). But I've thought for years that it's far easier to be productive as an organization on .NET in Visual Studio, since it's simpler to design, deliver, and maintain infrastructure.

    [0] https://www.hangfire.io/

  • machinery

    Machinery is an asynchronous task queue/job queue based on distributed message passing.

  • MassTransit

    Distributed Application Framework for .NET

    • Project mention: Easy to use library for RabbitMQ in dotnet core. | /r/csharp | 2023-05-21

    What would be the benefits of using your library instead of something like MassTransit?

  • CAP

    Distributed transaction solution in micro-service base on eventually consistency, also an eventbus with Outbox pattern

    • Project mention: Mechanism for managing faulty consumer in asynchronous event broadcast in microservices / modular monolith | /r/dotnet | 2023-05-22

    You might want to implement the outbox pattern. There’s a decent library that’ll help and is really easy to integrate. In particular, I’d look at using the transaction functionality

  • boltons

    🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.

    • Project mention: Boltons is a set of over 250 BSD-licensed, pure-Python utilities | news.ycombinator.com | 2023-12-11

  • algodeck

    An Open-Source Collection of 200+ Flash Cards to Help You Preparing Your Algorithms & Data Structures Interview 💯

  • The gist

    BullMQ - Message Queue and Batch processing for NodeJS and Python based on Redis

    • Project mention: Object Narrowing in Typescript with Graphile Worker | dev.to | 2024-01-31

    Graphile worker has been great for me because it's a library that works with Postgres that allows me to queue jobs and execute them on the server without adding too many additional layers of complexity for being able to accomplish async tasks. (I'm aware of how popular bull is, but I don't want to add another data-store only for async tasks)

  • Jocko

    Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)

  • huey

    a little task queue for python

    • Project mention: Nextflow: Data-Driven Computational Pipelines | news.ycombinator.com | 2023-08-10

    I've considered using Nextflow for bioinformatics pipelines but have yet to take the plunge. At work, I develop a proteomics pipeline that is composed of huey¹ tasks (Python library; simple alternative to Celery) which either use subprocess to call out to some external tool, or are just pure python. It runs in a worker container which is created by docker swarm, and all containers pull jobs from redis. For our scale, it works great. However, I don't have control over the resource utilization of individual steps, and in the past I've had issues with the pipeline blocking as a result of how I was chaining tasks together. I think something like Nextflow would remove these limitations, but one thing I think I would miss is the ability to debug individual pipeline steps locally with an interactive debugger. As far as I can tell, Nextflow has logging/tracing facilities but nothing quite like an interactive debugger. I'd be happy to be told I'm wrong, or even that I'm doing it wrong.

    ____

    ¹ https://github.com/coleifer/huey/

  • Delayed::Job

    Database based asynchronous priority queue system -- Extracted from Shopify

  • PHP AMQP

    The most widely used PHP client for RabbitMQ

  • horizon

    Dashboard and code-driven configuration for Laravel queues.

  • Coravel

    Near-zero config .NET library that makes advanced application features like Task Scheduling, Caching, Queuing, Event Broadcasting, and more a breeze!

  • swift-collections

    Commonly used data structures for Swift

  • p-queue

    Promise queue with concurrency control

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Queue related posts

Index

What are some of the best open-source Queue projects? This list will help you:

Project Stars
1 nsq 24,530
2 Redisson 22,706
3 gods 15,385
4 bull 15,018
5 Sidekiq 12,931
6 rq 9,503
7 Resque 9,385
8 agenda 9,232
9 Hangfire 8,994
10 machinery 7,281
11 MassTransit 6,506
12 CAP 6,435
13 boltons 6,415
14 algodeck 5,385
15 The gist 5,182
16 Jocko 4,882
17 huey 4,877
18 Delayed::Job 4,801
19 PHP AMQP 4,400
20 horizon 3,783
21 Coravel 3,555
22 swift-collections 3,509
23 p-queue 3,227

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com