jetstream VS Apache Kafka

Centrifugo can use Nats as PUB/SUB broker. I also tried to use Jetstream to implement Centrifugo history/recovery semantics – unfortunately Jetstream does not fit us very well as streams must be created over a separate call to admin API, and it's not possible to iterate/paginate over Jetstream streams to be suitable to current Centrifugo message history and recovery semantics (I asked about support for this – unfortunately the issue was closed and required API not implemented - https://github.com/nats-io/jetstream/issues/266, suggested solution is far from optimal for our use case).

Apache Kafka — a distributed event streaming platform implementing a variant of the Raft consensus protocol (written in Java, integrated with Scala);

Spring does this during autoconfiguration. For example this and this. When the user adds a configuration then it gets to overwrite the default from the template. I am looking for something similar, perhaps simpler approach.

Source and documentation on GitHub

Code is still in trunk though. https://github.com/apache/kafka/tree/trunk/connect/file/src/main/java/org/apache/kafka/connect/file

I briefly described the algorithm in this article and there is a wonderful article from Kafka that goes into more depth in their general purpose implementation. My implementation is specialized and over optimized in comparison, e.g. by using bit manipulation to avoid more expensive division/modulus instructions. Tokio rewrote their timerwheel after I showed them mine, borrowing some ideas but also staying more general. Hope that helps!

Apache Kafka