arewegameyet VS rust-rdkafka

Why Rust? Because I wanted to use far more concurrency than usually seen in games to overcome the fundamental performance problem in metaverse clients. The graphics stack works well enough to do this.[1] But it needs to get faster and do a few more things. Basically, I need three.js for Rust. I sort of have it, but the Rust crate has been abandoned, has bugs, and now I have to maintain it.

For a while, there was much enthusiasm for Rust game dev.[2] But it kind of ran down.

[1] https://video.hardlimit.com/w/7usCE3v2RrWK6nuoSr4NHJ

[2] https://arewegameyet.rs

arewegameyet

"While the ecosystem is still very young, you can find enough libraries and game engines to sink your teeth into doing some slightly experimental gamedev."

For practical uses of Rust? Whatever you want to program. People use Rust for game development, GUIs, web dev, and more. Anything where abstraction, speed, concurrency, memory safety, etc. are important, Rust will probably be a good fit.

Are we game yet? "Almost. We have the blocks, bring your own glue"

Not seeing anything else that's close to photo realistic. I'm hitting the tough bugs first all too often. More than half my time has been spent on ecosystem problems.

I also know of https://arewegameyet.rs/

Yes — and in fact, Firefox's implementation has been the go-to graphics API for folks trying to make Rust gamedev happen for a long time now. Bevy Engine's renderer is built on it, for example.

The Rust Kafka client library I used internally can be explored from this repository. The Lambda code and CDK IaC can be found here.

I use this library a lot and it's got some nice touches for how to handle wrapping a C library: https://github.com/fede1024/rust-rdkafka

For this example, we're using rdkafka to build producers and consumers, because it allows us to specify custom headers for each record.

kafka

You might find https://crates.io/crates/rdkafka helpful

I am playing with tokio and rust-rdkafka library, following the examples like this one: https://github.com/fede1024/rust-rdkafka/blob/6fb2c37/examples/asynchronous_processing.rs

The source for the current version of the library can be found on Github. I had to increase the major version because I needed to break the API in order to support all formats supported by the current Schema Registry version. I also added the possibility to set an API key, so it can be used with Confluent Cloud, the cloud offering from Confluent. As part of the latest major refactoring it's also supporting async. This might improve performance of your app, and is also the default for the major Kafka client, more information about why you would want to use async can be found in the async book. The schemas retrieved from the Schema Registry are cached. This way the schema is only retrieved once for each id, and reused for other messages with the same id.

What's wrong with rust-rdkafka?

This is a two-part series to help you get started with Rust and Kafka. We will be using the rust-rdkafka crate which itself is based on librdkafka (C library).