liftbridge VS styx

Liftbridge (https://liftbridge.io/) is more or less the NATS Kafka versioning.

[Gufo Liftbridge](https://pypi.org/project/gufo-liftbridge/) is the Python asyncio Liftbridge client.

https://liftbridge.io/

Apache Pulsar might be worth a look, but it's actually more complex under the hood than Kafka, but has a lot of features built-in that either aren't in FOSS Kafka yet, like tiered storage, or won't be until Confluent doesn't dominate the PMC (like an integrated schema registry), or just can't be done very nicely, if at all, like decent multi-tenancy.

That said, it's a fast moving target, the code quality last I looked was patchy in places, ditto the documentation for both it and Bookkeeper, and the admin overhead is higher (managing bookies and brokers and Zookeepers vs. just brokers and ZK with Kafka, or when KRaft is production ready, just brokers).

This is basically an ad for GCP right?

That said, it looks like Kafka is by far and away the way to handle persistent logs/events at scale. AFAIK a company here in Japan called LINE has all their messaging flowing through a large kafka cluster themselves.

Wonder if anyone is running large NATS Jetstream[0]/Liftbridge[1] or Pulsar[2] (yahoo runs those) clusters. I guess Pulsar might be #2 in terms of adoption at large scale?

[0]: https://docs.nats.io/jetstream/jetstream

[1]: https://liftbridge.io/

[2]: https://pulsar.apache.org/

Do you need the messages to be durable, if so you can have a look at Liftbridge: - https://liftbridge.io/ - https://github.com/liftbridge-io/liftbridge

And if you want closer kafka semantics built on top of nats, check out liftbridge:

https://liftbridge.io/

Built something similar in Go with a friend a few years ago.

https://github.com/thibauts/styx

Here is an event stream abstraction that has very strong semantics (exactly once in most cases) with simple usage examples, native HTTP and websockets APIs, strong atomicity and durability guarantees [1].

What it doesn’t have is clustering and a father that’s != null at marketing =]

[1] https://github.com/thibauts/styx

Hi Alexander ! I'm a big fan of the work you do at vectorizedio and of your blog posts ! I strongly believe there's a need for much simpler event streaming and room for improvement performance-wise.

I took a different path by divorcing from the Kafka protocol and experimenting with what I believe is a simpler to model approach to reliable event processing [1].

[1] https://github.com/dataptive/styx/blob/master/docs/howto/rel...