The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning. Learn more →
Top 11 Queuing Open-Source Projects
-
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
-
Project mention: Implementing OTel Trace Context Propagation Through Message Brokers with Go | dev.to | 2024-04-04Several message brokers, such as NATS and database queues, are not supported by OpenTelemetry (OTel) SDKs. This article will guide you on how to use context propagation explicitly with these message queues.
Project mention: Choosing Between a Streaming Database and a Stream Processing Framework in Python | dev.to | 2024-02-10Stream-processing platforms such as Apache Kafka, Apache Pulsar, or Redpanda are specifically engineered to foster event-driven communication in a distributed system and they can be a great choice for developing loosely coupled applications. Stream processing platforms analyze data in motion, offering near-zero latency advantages. For example, consider an alert system for monitoring factory equipment. If a machine's temperature exceeds a certain threshold, a streaming platform can instantly trigger an alert and engineers do timely maintenance.
Although they did not make it into production, I experimented with the RabbitMQ message broker, Python (Django, Flask), Kubernetes + minikube, JWT, and NGINX. This was a hobby project, but I intended to learn about microservices along the way.
There's beanstalkd, it has a few Python libraries and it works out of the box with ActiveJob via Backburner.
https://beanstalkd.github.io/
For running queues on Postgres with Node.js backend(s), I highly recommend https://github.com/timgit/pg-boss. I'm sure it has it scale limits. But if you're one of the 90% of the apps that never needs any kind of scale that a modern server can't easily handle then it's fantastic. You get transactional queueing of jobs, and it automatically handles syncing across multiple job processing servers using Postgres locks.
Project mention: [D] What are the compute options you've considered for your projects? | /r/MachineLearning | 2023-05-07I am a fan of Gearman to schedule and dispatch distributed jobs, Redis as a collaborative blackboard, and GlusterFS to share models across multiple systems and make bulk data available across the entire system (usually referenced in the blackboard as a pathname).
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Queuing related posts
- Implementing OTel Trace Context Propagation Through Message Brokers with Go
- NATS: First Impressions
- Sequential and parallel execution of long-running shell commands
- Interview with Sebastian Holstein, Founder of Qaze
- Revolutionizing Real-Time Alerts with AI, NATs and Streamlit
- Ruby 3.3
- A Developer's Journal: Simplifying the Twelve-Factor App
-
A note from our sponsor - WorkOS
workos.com | 25 Apr 2024
Index
What are some of the best open-source Queuing projects? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | NATS | 14,720 |
| 2 | Apache Pulsar | 13,744 |
| 3 | RabbitMQ | 11,590 |
| 4 | Disque | 7,991 |
| 5 | BeanstalkD | 6,472 |
| 6 | Chronicle Queue | 3,153 |
| 7 | pg-boss | 1,625 |
| 8 | Gearman | 724 |
| 9 | Siberite | 585 |
| 10 | Apache Qpid | 125 |
| 11 | horizon-exporter | 25 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com