liftbridge
nodejs-pubsub
liftbridge | nodejs-pubsub | |
---|---|---|
10 | 24 | |
2,536 | 512 | |
0.2% | 0.4% | |
6.3 | 8.4 | |
15 days ago | 3 days ago | |
Go | TypeScript | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
liftbridge
- Kafka alternatives
-
Understanding NATS.io concepts vs. Kafka - similarities and differences
Liftbridge (https://liftbridge.io/) is more or less the NATS Kafka versioning.
-
Gufo Liftbridge - the Python asyncio Liftbridge client
[Gufo Liftbridge](https://pypi.org/project/gufo-liftbridge/) is the Python asyncio Liftbridge client.
- What I Wish Someone Would Have Told Me About Using Rabbitmq
-
On Efficiently Partitioning a Topic in Apache Kafka
https://liftbridge.io/
Apache Pulsar might be worth a look, but it's actually more complex under the hood than Kafka, but has a lot of features built-in that either aren't in FOSS Kafka yet, like tiered storage, or won't be until Confluent doesn't dominate the PMC (like an integrated schema registry), or just can't be done very nicely, if at all, like decent multi-tenancy.
That said, it's a fast moving target, the code quality last I looked was patchy in places, ditto the documentation for both it and Bookkeeper, and the admin overhead is higher (managing bookies and brokers and Zookeepers vs. just brokers and ZK with Kafka, or when KRaft is production ready, just brokers).
-
Processing billions of events in real time at Twitter
This is basically an ad for GCP right?
That said, it looks like Kafka is by far and away the way to handle persistent logs/events at scale. AFAIK a company here in Japan called LINE has all their messaging flowing through a large kafka cluster themselves.
Wonder if anyone is running large NATS Jetstream[0]/Liftbridge[1] or Pulsar[2] (yahoo runs those) clusters. I guess Pulsar might be #2 in terms of adoption at large scale?
[0]: https://docs.nats.io/jetstream/jetstream
[1]: https://liftbridge.io/
[2]: https://pulsar.apache.org/
-
Most primitive lighweight alternative to Kafka?
Do you need the messages to be durable, if so you can have a look at Liftbridge: - https://liftbridge.io/ - https://github.com/liftbridge-io/liftbridge
-
ZooKeeper-free Kafka is out. First Demo
And if you want closer kafka semantics built on top of nats, check out liftbridge:
https://liftbridge.io/
- NATS, NATS Streaming & NATS JetStream + How to build a JetStream Cluster & Go Client
nodejs-pubsub
-
Event-Driven Architecture 101
Secondly, Go is incredibly easy to learn and in my opinion, maintain. This means that if you're a growing company and expect to onboard new teams and team members, having Go as a basis for your systems should mean that new engineers can get up to speed quickly. Below is a small sample application that can connect to Google PubSub, subscribe to a topic, send an event and then clean up. In total, its 82 lines of code including liberal line breaks. Even if you have never written or read a line of Go before, I hope you'll agree that it's quite clear and readable:
-
Kafka alternatives
Pub/Sub
-
Top 6 message queues for distributed architectures
Google Cloud Pub/Sub is a fully-managed, globally scalable and secure queue provided by Google Cloud for asynchronous processing messages. Cloud Pub/Sub has many of the same advantages and disadvantages as SQS due to also being cloud hosted. It has a free and paid tier.
-
Job Scheduling on Google Cloud Platform
Cloud Pub/Sub: A global messaging service for event-driven architectures
-
Messaging Patterns 101: A Comprehensive Guide for Software Developers
Google Cloud Pub/Sub (*https://cloud.google.com/pubsub*)
-
Effortlessly Scale Your Applications with FaaS: Learn How Functions as a Service Can Help You Grow and Thrive
Google Cloud Functions is a FaaS offering from Google Cloud Platform (GCP). It allows developers to run their code in response to events, such as changes in a database or the arrival of a message in a Pub/Sub topic. Like AWS Lambda, Google Cloud Functions can be used to build a variety of applications, including serverless websites, data processing pipelines, and real-time data streams.
-
Mixing GCloud and F#
that gets triggered when a Pub/Sub topic is fired (from the webhook function)
-
What is the best data storage solution for high-frequency (near real-time) updates
Maybe Pub/Sub from GCP?
-
Kafka on GKE cluster security guidelines
I'm curious - given your limited knowledge, is there a reason you're looking to self host this in your own cluster rather than using a managed service like https://cloud.google.com/find-a-partner/partner/confluent-inc?redirect= or just native Google PubSub https://cloud.google.com/pubsub/ ?
-
Moving to Google Cloud managed services, from a FinOps point of view
Pub/Sub, the GCP managed service for message queuing, has two levels of services on standard and one Lite. Standard is the high availability version of it and Lite could be a zonal or regional service with infrastructure managed by the client. Obviously, the model pricing will be very different with a x10 between Standard and zonal Lite. However, the model pricing is the same is based on throughput for message publishing, message storage costs and egress for message distribution. Here, we totally break the similarity with a VM model (except on storage). Everything is drived on volumetry and performance of inbound and outbound messages.
What are some alternatives?
jetstream - JetStream Utilities
twitch - Interact with Twitch's API, chat and subscribe to events via PubSub and EventSub.
Apache Kafka - Mirror of Apache Kafka
mitt - 🥊 Tiny 200 byte functional event emitter / pubsub.
NATS - High-Performance server for NATS.io, the cloud and edge native messaging system.
svelte-persisted-store - A Svelte store that persists to localStorage
KrakenD - Ultra performant API Gateway with middlewares. A project hosted at The Linux Foundation
RabbitMQ - Open source RabbitMQ: core server and tier 1 (built-in) plugins
glow - Glow is an easy-to-use distributed computation system written in Go, similar to Hadoop Map Reduce, Spark, Flink, Storm, etc. I am also working on another similar pure Go system, https://github.com/chrislusf/gleam , which is more flexible and more performant.
redpanda - Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
MySQL - MySQL Server, the world's most popular open source database, and MySQL Cluster, a real-time, open source transactional database.