Top 23 Pubsub Open-Source Projects
Dapr is a portable, event-driven, runtime for building distributed applications across cloud and edge.Project mention: Azure Container Apps - CI/CD deployments (Video Demo) | dev.to | 2022-01-11
Azure Container Apps is a container orchestration platform for deploying modern applications and microservices. With Azure Container Apps you can deploy your containerized apps without the need to take on the overhead of managing the infrastructure of the underlying systems. This service is still in preview, but provides you with the ability to use your preferred language or framework to build microservices with support for Distributed Application Runtime (Dapr). There's also scaling of your containers with Kubernetes Event-Driven Autoscaling (KEDA).
Simple, secure & standards compliant web server for the most demanding of applicationsProject mention: Book recommendations for Backend development concepts for a beginner | reddit.com/r/developersIndia | 2021-11-29
AFAIK, Websockets are built on HTTP. Have a look at https://github.com/uNetworking/uWebSockets.
Less time debugging, more time building. Scout APM allows you to find and fix performance issues with no hassle. Now with error monitoring and external services monitoring, Scout is a developer's best friend when it comes to application development.
Apache Pulsar - distributed pub-sub messaging systemProject mention: Simplify migrating from Kafka to Pulsar with Kafka Connect Support | dev.to | 2022-01-04
Use Kafka Connect Adaptor NAR https://github.com/apache/pulsar/tree/master/pulsar-io/kafka-connect-adaptor-nar as a starting point (for simplicity, I'll edit it directly) and add your Kafka Connector Sink to the list of the dependencies in pom.xml. Here’s what this would look like with the Kinesis Kafka connector sink:
ZeroMQ core engine in C++, implements ZMTP/3.1Project mention: Implement message handling in firmware | reddit.com/r/embedded | 2021-11-28
I just pulled this: https://github.com/zeromq/libzmq and it was like 80k lines of code.
🥊 Tiny 200 byte functional event emitter / pubsub.Project mention: Sibling component communication | reddit.com/r/vuejs | 2022-01-08
deepstream.io serverProject mention: Ask HN: Who Wants to Collaborate? | news.ycombinator.com | 2022-01-01
I'm working on a few projects, from one/two days to platforms.
The first is OS and is a simple nodeJS environment to deploy applications via lambda and express quickly. Sort of like nestJS except less decorators and more functional (https://vramework.io/). I already know of a few other colleagues that rolled their own propriety versions of this to support enterprise and cloud deployments so decided to OS it.
The other OS project is a strongly typed postgres/mysql driver. The idea is to generate typescript definitions directly from postgres (https://github.com/vramework/schemats) and then have a think layer ontop of pg-node that gives you strongly typed queries (https://github.com/vramework/postgres-typed).
An open-source project I spent a few years on the core team is https://deepstream.io/, a realtime-server that allows you to mix and match multiple streaming protocols (mqtt/websocket/others) and allow those clients to talk to each other using pub-sub and records. I'm not longer working for it but wanted to give it a shout out!
On a non OS project, I have been working on an immersive audio platform for a while now. The main goal is to allow users to pick and choose how audio books progress, and also have a live session mode which allows users to record their pulse / answer questions and a few other metrics and associate it with sentences. I pretty much built and deployed all of it but require some advice/brainstorming on how to proceed now. I built it to satisfy an itch when I was practicing shamanism during the first lockdown when I was in-between contracts / taking time off.
I also want to build a simple web-pages strategy game based around eco-education, but don't have the bandwidth . If anyone is interested in mixing together gamification and eco-village building might be a fun conversion to bounce ideas!
All the OS projects above were used to support my personal/a couple professional projects over the last few years.
Email in profile
Highly scalable realtime frameworkProject mention: Top WebSocket libraries for Node.js in 2022 | dev.to | 2022-01-07
At the time of writing, SocketCluster has almost 6k stars on GitHub and 7k downloads on npm weekly,
Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.
Scalable real-time messaging server in a language-agnostic way. Set up once and forever.Project mention: Woe be onto you for using a WebSocket | news.ycombinator.com | 2021-12-22
Every time I read criticism of WebSockets it reminds me about WebSuckets (https://speakerdeck.com/3rdeden/websuckets) presentation :)
I am the author of Centrifugo server (https://github.com/centrifugal/centrifugo) - where the main protocol is WebSocket. Agree with many points in post – and if there is a chance to build sth without replacing stateless HTTP to persistent WebSocket (or EventSource, HTTP-streaming, raw TCP etc) – then definitely better to go without persistent connections.
But there are many tasks where WebSockets simply shine – by providing a better UX, providing a more interactive content, instant information/feedback. This is important to keep - even if underlying stack is complicated enough. Not every system need to scale to many machines (ex. multiplayer games with limited number of players), corporate apps not really struggle from massive reconnect scenarios, and so on. So WebSockets are definitely fine for certain scenarios IMO.
I described some problems with WebSockets Centrifugo solves in this blog post - https://centrifugal.dev/blog/2020/11/12/scaling-websocket. I don't want to say there are no problems, I want to say that WebSockets are fine in general and we can do some things to deal with things mentioned in the OP's post.
Robust, Ubiquitous and Massively Scalable Messaging Platform (XMPP, MQTT, SIP Server)
μWebSockets for Node.js back-ends :metal:Project mention: Same algorithm is much faster in node.js than in Golang. Why? | reddit.com/r/golang | 2021-11-19
Calling out to much lower level code is also really simple in Node. (Go obviously has the bindings here too) but it’s how libraries like uWebSockets.js beat the pants off of any other Go websocket libraries.
A Microservice Toolkit from The New York Times (by nytimes)
High performance, distributed and low latency publish-subscribe platform.
A micro library providing Ruby objects with Publish-Subscribe capabilitiesProject mention: "I'm the CTO of a Growing Rails Startup" Ask Me Anything | reddit.com/r/rails | 2021-08-27
We follow the interactor pattern to store our business logic. So we mainly have skinny controllers, skinny models and then interactors. We also don't use ActiveRecord callbacks very much, we primarily use Wisper to broadcast events and then various domains can subscribe to the events they care about and respond accordingly.
A distributed MQTT message broker based on Erlang/OTP. Built for high quality & Industrial use cases.Project mention: Mosquitto: An open-source MQTT broker | news.ycombinator.com | 2021-09-29
The VerneMQ bugtracker scares me. Especially this one: https://github.com/vernemq/vernemq/issues/1663. I'm running one instance but I'm on the lookout for an alternative that can more reliably save messages if a subscription client goes down.
WebSocket and WAMP in Python for Twisted and asyncioProject mention: Jupyter data analytics on real-time trading data from crypto exchanges | reddit.com/r/ethereum | 2022-01-12
Lightweight, fault-tolerant message streams.Project mention: Processing billions of events in real time at Twitter | news.ycombinator.com | 2021-11-15
This is basically an ad for GCP right?
That said, it looks like Kafka is by far and away the way to handle persistent logs/events at scale. AFAIK a company here in Japan called LINE has all their messaging flowing through a large kafka cluster themselves.
Wonder if anyone is running large NATS Jetstream/Liftbridge or Pulsar (yahoo runs those) clusters. I guess Pulsar might be #2 in terms of adoption at large scale?
Distributed cache and in-memory key/value data store. It can be used both as an embedded Go library and as a language-independent service.Project mention: Olric | reddit.com/r/devopspro | 2021-12-11
Open source implementation of OPC UA (OPC Unified Architecture) aka IEC 62541 licensed under Mozilla Public License v2.0Project mention: Apache PLC4X announcing end of community support due to missing funding | news.ycombinator.com | 2022-01-12
There's quite a bit of OPC UA bashing across this project. So let me chime in to keep the "balance in the universe".
OPC UA is a protocol to interact with an object-oriented information model. Basically CORBA done right to use object-oriented principles and reuse software components in industrial automation.
Since OPC UA is a protocol, its performance depends mostly on the implementation. Some PLCs may be crappy. But that doesn't translate into bad performance overall. My experience goes to the exact contrary.
Full disclosure, I lead-develop and maintain an open source OPC UA implementation that sees quite a bit of use by the big guys in the automation domain. We use C for performance. And we do have funding from the industry.
But yes, it is hard to break into this world. Especially since solutions have to be maintained for 20+ years. A solo developer usually cannot ensure that this will still be usable some years down the line.
Your high performance web application C frameworkProject mention: C Deep | dev.to | 2021-02-27
facil.io - Mini-framework for web applications. Includes a fast HTTP and Websocket server, and also supports custom protocols. MIT
WebSocket & WAMP in Java for Android and Java 8
💨 A real time messaging system to build a scalable in-app notifications, multiplayer games, chat apps in web and mobile apps.Project mention: GitHub - Clivern/Beaver: 💨 A real time messaging system to build a scalable in-app notifications, multiplayer games, chat apps in web and mobile apps. | reddit.com/r/golang | 2021-10-16
Pusher server implementation compatible with Pusher client libraries.Project mention: Weekly "ask anything" thread | reddit.com/r/PHP | 2021-09-06
A rather common solution is to set up a dedicated service which will run real time connection with the client. Either node or something like elixir and shoot events into the 'blackbox' which later is responsible for pushing the data to the client. Good example is https://github.com/edgurgel/poxa.
🔬 Writing reliable & fault-tolerant microservices in Node.js https://hemerajs.github.io/hemera/
Pubsub related posts
Cheap temporary memory alternative for Memorystore for Memcached/Redis?
1 project | reddit.com/r/googlecloud | 16 Jan 2022
Sibling component communication
1 project | reddit.com/r/vuejs | 8 Jan 2022
Simplify migrating from Kafka to Pulsar with Kafka Connect Support
1 project | dev.to | 4 Jan 2022
Analyzing Real-Time Movie Reviews with Redpanda and Memgraph
2 projects | dev.to | 5 Dec 2021
How to pass data between pages in SvelteKit
1 project | reddit.com/r/sveltejs | 28 Nov 2021
Google Pub/Sub client library for R
2 projects | reddit.com/r/rstats | 8 Nov 2021
Ask HN: Approaches to load balancing web sockets?
1 project | news.ycombinator.com | 30 Oct 2021
What are some of the best open-source Pubsub projects? This list will help you:
Are you hiring? Post a new remote job listing for free.