styx VS kafkacat

Built something similar in Go with a friend a few years ago.

https://github.com/thibauts/styx

Here is an event stream abstraction that has very strong semantics (exactly once in most cases) with simple usage examples, native HTTP and websockets APIs, strong atomicity and durability guarantees [1].

What it doesn’t have is clustering and a father that’s != null at marketing =]

[1] https://github.com/thibauts/styx

Hi Alexander ! I'm a big fan of the work you do at vectorizedio and of your blog posts ! I strongly believe there's a need for much simpler event streaming and room for improvement performance-wise.

I took a different path by divorcing from the Kafka protocol and experimenting with what I believe is a simpler to model approach to reliable event processing [1].

[1] https://github.com/dataptive/styx/blob/master/docs/howto/rel...

To communicate with Kafka, you can use Kafkacat, a command-line tool that allows to produce and consume Kafka messages using a very simple syntax. It also allows you to view the topics' metadata.

Feels like you could get pretty far with kafkacat and a SQLite database.

This is the second release version of Kafcat. Kafcat is a Rust fully async rewrite of kafkacat.

Consume from the Kafka topic my.topic with kafkacat:

From the code above, you can see that to process the headers, simply use the ConsumerRecord.headers() method to return the headers. In our example above, we’re printing the headers out to the console for demonstration purposes. Once you have access to the headers, you can process them as needed. For reading headers from the command line, KIP-431 adds support for optionally printing headers from the ConsoleConsumer, which will be available in the Apache Kafka 2.7.0 release.You can also use kafkacat to view headers from the command line. Here’s an example command:

Note: In the example above, we have used kafkacat to consume the topics. The option -o-1 is used to only consume the latest message