NATS VS liftbridge

[1] https://github.com/nats-io/nats-server/issues/6832#issuecomm...

https://nats.io is easier to use than Kafka and already solves several of the points in this post I believe, like removing partitions, supporting key-based streams, and having flexible topic hierarchies.

Look at the contribution history, basically all active contributors work for Synadia: https://github.com/nats-io/nats-server/graphs/contributors

That's not a healthy / functioning open source community. Less than 30 people have made more than 10 commits; most of the 160 were "drive by" who fixed a single small thing.

NATS, with its lightweight and high-performance design, offers features well-suited for sandbox testing. We can leverage NATS queue groups, which function similarly to Kafka consumer groups.

In today’s dynamic cloud environments, managing Kubernetes resources across multiple clusters can be a complex task. Traditional methods often lack the agility and event-driven architecture needed to respond quickly to changes and automate resource provisioning. This article explores how Sveltos, in conjunction with NATS and JetStream, simplifies multi-cluster Kubernetes management through event-driven automation, streamlining operations and improving responsiveness.

hey thanks!

definitely interesting!

I do wonder if this really solves the author problem because by the looks of it , you just have to run meta command and it would run over each of the sub directory. While at the same time , I think I like it because this is what I think people refer to as "modular monolith"

Combining this with nats https://nats.io/ (hey if you don't want it to be over the network , you could use nats with the memory model of your application itself to reduce any overhead) and essentially just get yourself a really modular monolith in which you can then seperate things selectively (ahem , microservices) afterwards rather easily.

and now decided to write more complex things using https://nats.io and https://data-star.dev -

Liftbridge (https://liftbridge.io/) is more or less the NATS Kafka versioning.

[Gufo Liftbridge](https://pypi.org/project/gufo-liftbridge/) is the Python asyncio Liftbridge client.

https://liftbridge.io/

Apache Pulsar might be worth a look, but it's actually more complex under the hood than Kafka, but has a lot of features built-in that either aren't in FOSS Kafka yet, like tiered storage, or won't be until Confluent doesn't dominate the PMC (like an integrated schema registry), or just can't be done very nicely, if at all, like decent multi-tenancy.

That said, it's a fast moving target, the code quality last I looked was patchy in places, ditto the documentation for both it and Bookkeeper, and the admin overhead is higher (managing bookies and brokers and Zookeepers vs. just brokers and ZK with Kafka, or when KRaft is production ready, just brokers).

This is basically an ad for GCP right?

That said, it looks like Kafka is by far and away the way to handle persistent logs/events at scale. AFAIK a company here in Japan called LINE has all their messaging flowing through a large kafka cluster themselves.

Wonder if anyone is running large NATS Jetstream[0]/Liftbridge[1] or Pulsar[2] (yahoo runs those) clusters. I guess Pulsar might be #2 in terms of adoption at large scale?

[0]: https://docs.nats.io/jetstream/jetstream

[1]: https://liftbridge.io/

[2]: https://pulsar.apache.org/

Do you need the messages to be durable, if so you can have a look at Liftbridge: - https://liftbridge.io/ - https://github.com/liftbridge-io/liftbridge

And if you want closer kafka semantics built on top of nats, check out liftbridge:

https://liftbridge.io/