pg-boss | starqueue | |
---|---|---|
12 | 4 | |
1,638 | 0 | |
- | - | |
4.4 | 10.0 | |
29 days ago | about 1 year ago | |
JavaScript | Python | |
MIT License | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
pg-boss
-
Choose Postgres Queue Technology
For running queues on Postgres with Node.js backend(s), I highly recommend https://github.com/timgit/pg-boss. I'm sure it has it scale limits. But if you're one of the 90% of the apps that never needs any kind of scale that a modern server can't easily handle then it's fantastic. You get transactional queueing of jobs, and it automatically handles syncing across multiple job processing servers using Postgres locks.
-
Build Your Own Personal Twitter Agent 🧠🐦⛓ with LangChain
Jobs use pg-boss, a postgres extension, to queue and run tasks under the hood.
-
SQL Maxis: Why We Ditched RabbitMQ and Replaced It with a Postgres Queue
If you don't want to roll your own, look into https://github.com/timgit/pg-boss
- How/do you handle queue type workflows?
-
Which tool/library well adopted to use Postgres as a message broker?
I saw this https://github.com/timgit/pg-boss but it's more for jobs than for message with multiple consumers (having their own progress offset).
-
How to schedule tasks in a Node.js app 🕙
The best I've used till now. Has all kind of features and really great when you have a postgres dB in your stack. https://github.com/timgit/pg-boss
-
Cluster friendly task scheduler for NodeJS
Check out these; - https://github.com/mitranim/posterus - https://github.com/timgit/pg-boss - https://github.com/FirebaseExtended/firebase-queue - https://www.npmjs.com/package/rabbit-queue
-
You don't need distributed systems.
You can use the simplest option than implement a new service. Keep in mind that every running system can be a job scheduler, you can just use nodejs worker threads, Redis, or even your DB as a job scheduler, check PGBoss for example.
-
Launch HN: Convoy (YC W22) – Open-source cloud-native webhooks service
Both! For context, we're currently using https://github.com/timgit/pg-boss as a task queue on top of postgres and it works great. No need to complicate things with Redis. I believe it's quite straightforward to implement a task queue on top of postgres using the SKIP LOCKED functionality.
- Devious SQL: Message Queuing Using Native PostgreSQL
starqueue
-
Choose Postgres Queue Technology
MS SQL server, Postgres and MySQL all support SKIP LOCKED, which means they are all suitable for running queues.
I built a complete implementation in Python designed to work the same as SQS but be more simple:
https://github.com/starqueue/starqueue
Alternatively if you just want to quickly hack something into your application, here is a complete solution in Python with retries:
import psycopg2
-
SQL Maxis: Why We Ditched RabbitMQ and Replaced It with a Postgres Queue
I wrote a message queue in Python called StarQueue.
It’s meant to be a simpler reimagining of Amazon SQS.
It has an HTTP API and behaves mostly like SQS.
I wrote it to support Postgres, Microsoft’s SQL server and so MySQL because they all support SKIP LOCKED.
At some point I turned it into a hosted service and only maintained the Postgres implementation though the MySQL and SQL server code is still in there.
It’s not an active project but the code is at https://github.com/starqueue/starqueue/
- Show Reddit: StarQueue - Postgres database backed message queue server for Python
-
Show HN: StarQueue database backed message queue server for Python
Hi folks,
This is a project I wrote and since I am doing nothing with it any more I thought I would publish the source code in case anyone finds it interesting.
https://github.com/starqueue/starqueue/
StarQueue is a message queue server written in Python.
It is designed to be a more simple copy of Amazon Simple Queue Server.
Clients access it via HTTP. The API is documented at https://github.com/starqueue/starqueue/tree/main/starqueueserver/website
The database is Postgres.
When I developed it initially, I included seamless support for Postgres, MySQL and Microsoft SQL server.
At some point in the development I gave up on all databases except Postgres, though I have left the MySQL and SQL server code in place.
I deployed StarQueue as an online service at one point (no longer online). This github repo is a copy of the source code for that service.
This project is not live and is archived, but I have posted it here in case anyone finds the source code interesting.
What are some alternatives?
worker - High performance Node.js/PostgreSQL job queue (also suitable for getting jobs generated by PostgreSQL triggers/functions out into a different work queue)
neoq - Queue-agnostic background job library for Go, with a pleasant API and powerful features.
celery - Distributed Task Queue (development branch)
kubeblocks - KubeBlocks is an open-source control plane that runs and manages databases, message queues and other data infrastructure on K8s.
django-postgres-queue - A task queue for django
BeanstalkD - Beanstalk is a simple, fast work queue.
RabbitMQ - Open source RabbitMQ: core server and tier 1 (built-in) plugins
tqs - Tiny Queue Service (Server)
Redis - Redis is an in-memory database that persists on disk. The data model is key-value, but many different kind of values are supported: Strings, Lists, Sets, Sorted Sets, Hashes, Streams, HyperLogLogs, Bitmaps.
kue - Kue is a priority job queue backed by redis, built for node.js.
arniesmtpbufferserver