rust-rdkafka VS genawaiter

I use this library a lot and it's got some nice touches for how to handle wrapping a C library: https://github.com/fede1024/rust-rdkafka

For this example, we're using rdkafka to build producers and consumers, because it allows us to specify custom headers for each record.

kafka

You might find https://crates.io/crates/rdkafka helpful

I am playing with tokio and rust-rdkafka library, following the examples like this one: https://github.com/fede1024/rust-rdkafka/blob/6fb2c37/examples/asynchronous_processing.rs

The source for the current version of the library can be found on Github. I had to increase the major version because I needed to break the API in order to support all formats supported by the current Schema Registry version. I also added the possibility to set an API key, so it can be used with Confluent Cloud, the cloud offering from Confluent. As part of the latest major refactoring it's also supporting async. This might improve performance of your app, and is also the default for the major Kafka client, more information about why you would want to use async can be found in the async book. The schemas retrieved from the Schema Registry are cached. This way the schema is only retrieved once for each id, and reused for other messages with the same id.

What's wrong with rust-rdkafka?

This is a two-part series to help you get started with Rust and Kafka. We will be using the rust-rdkafka crate which itself is based on librdkafka (C library).

Yes, Letlang is translated to Rust and the runtime is implemented in Rust, using tokio and genawaiter. The compiler itself is also built in Rust.

(note that genawaiter itself doesn't support no_std environments, but there's a merge request for that.)

> `async` to make fake generators.

Genawaiter[0] is one of them.

[0]: https://github.com/whatisaphone/genawaiter

Are you aware of the genawaiter crate?

A warning though: genawaiter doesn't seem to be maintained. The last commit is 2 years old, and issues are not active (I opened one that I find somewhat critical: https://github.com/whatisaphone/genawaiter/issues/35).

There's e.g. https://github.com/whatisaphone/genawaiter, but you can also use yield directly (https://doc.rust-lang.org/beta/unstable-book/language-features/generators.html).

If you're interested in something that can be used right now, on stable, you should take a look at the genawaiter crate, it reimplements generators with async.