rust-rdkafka VS tonic

I use this library a lot and it's got some nice touches for how to handle wrapping a C library: https://github.com/fede1024/rust-rdkafka

For this example, we're using rdkafka to build producers and consumers, because it allows us to specify custom headers for each record.

kafka

You might find https://crates.io/crates/rdkafka helpful

I am playing with tokio and rust-rdkafka library, following the examples like this one: https://github.com/fede1024/rust-rdkafka/blob/6fb2c37/examples/asynchronous_processing.rs

The source for the current version of the library can be found on Github. I had to increase the major version because I needed to break the API in order to support all formats supported by the current Schema Registry version. I also added the possibility to set an API key, so it can be used with Confluent Cloud, the cloud offering from Confluent. As part of the latest major refactoring it's also supporting async. This might improve performance of your app, and is also the default for the major Kafka client, more information about why you would want to use async can be found in the async book. The schemas retrieved from the Schema Registry are cached. This way the schema is only retrieved once for each id, and reused for other messages with the same id.

What's wrong with rust-rdkafka?

This is a two-part series to help you get started with Rust and Kafka. We will be using the rust-rdkafka crate which itself is based on librdkafka (C library).

Use tonic-build directly from Rust.

} // Omit the remaining code and refer to the example in Tonic: https://github.com/hyperium/tonic/blob/master/examples/src/multiplex/server.rs ```

If I’m not misunderstanding, you could both decode the gRPC protobuf AND write to delta lake in Rust. Tonic, Delta-rs.

Original post: https://github.com/hyperium/tonic/issues/1405. Cross-post here in case the problem is not specific to tonic.

Tonic

Just to double check Is this the correct repo?

Wasm isn't involved much actually. Basically, the frontend asks the backend to perform a gRPC call on its behalf using Tauri events. They are like named channels on which you can send any serde-compatible value. But the backend is a normal Rust program so there are no constraints there. I use prost-reflect to encode/decode Protobuf messages according to Protobuf descriptors loaded at runtime and make the actual requests using tonic from the tokio ecosystem. prost-reflect is necessary because, normally, tonic expects the Protobuf descriptor to be known at compile time so it can make some code generation behind the scenes.

You could have a look at gRPC i.e. https://github.com/hyperium/tonic

Looking at this example code: https://github.com/hyperium/tonic/blob/master/examples/src/tower/server.rs

The remaining gap is remote actors, since you still need some kind of serialization between them, and take your pick of standards for that one such as gRPC using Tonic.