flume VS trust-dns

I also like flume, it has impressive performance (although not the best). More importantly, it's written only with safe rust. https://github.com/zesterer/flume

The most commonly suggested replacement for mspc is crossbeam-channel; flume is also relatively popular.

It's not officially deprecated, but the alternatives on crates.io are considered better. flume and crossbeam-channel feature less unsafe code and offer better performance. Benchmarks.

it's slow (checkout flume's benchmarks for example)

For concurrency/parallelism, you launch at most 2 * CPU Cores, PIN them and use a fast broker to spread the task (like a ring buffer or an MPSC). But you keep linear scan, tight loops, SIMD friendly data, on each. You are not switching context that much, and instead, bet you will process the batch fast. (CPUs are fast today!)

First, I was becoming more and more confident that the channel library Daxpedda and I fell in love with, flume, was misbehaving, but I couldn't seem to reproduce it outside of the massive PliantDB codebase. I finally called up Daxpedda on Discord and screen shared my debugging session, showing him how the tests succeeded if I retained a channel. If I allowed the sender to drop after successfully sending, sometimes the tests would fail. He agreed, something was odd. It took me a while, but I finally whittled it down to about 30 lines of code and reported the issue. In an amazingly quick fashion, the maintainer fixed the issue and released an update. And for the record, I still fully love and recommend this library if you're mixing async and non-async code using channels. It's a wonderful implementation.

Sort of? Here's the closest thing you'll probably find: https://crates.io/crates/flume

My first try was with flume, which uses a single producer, multi-consumer setup, and while it compiles and runs, it's magnitudes slower than the serial version. Here's a psuedocode rendition:

IO is not a part of the async runtime contract (I don't know if this is good or bad), and Tokio & futures famously have different `Async{Read,Write}` traits. I once had to do this [0] to adapt between them.

This means that any crate that uses IO will be bound to a limited number of Runtimes. Everything being Tokio-only is pretty bad (though Tokio itself is great), but here we are...

[0] https://github.com/bluejekyll/trust-dns/pull/1373#issuecomme...

You might be interested in Trust DNS - "A Rust based DNS client, server, and Resolver, built to be safe and secure from the ground up."

It looks like you need to reach for a separate crate for that: https://github.com/bluejekyll/trust-dns/blob/7dcb7b983f5407d95d93b800af13caeee975aaa8/crates/async-std-resolver/src/lib.rs

This is not true; you can use the async resolver with other executors: https://github.com/bluejekyll/trust-dns/blob/main/crates/resolver/src/async_resolver.rs

You could run a local DNS server using something like https://github.com/bluejekyll/trust-dns . Or, you could install an NSS module to resolve names via the innernet client.

My point is that "implementation that doesn't use unsafe" is not necessarily always slower than "implementation that does use unsafe". Often people assume that this is the case, and it isn't. tinyvec currently beats smallvec in more than a few benchmarks. Not all, but some. And this sometimes visible to users. The point is that if you want speed, you don't necessarily need to give up any safety at all. Most differences in performance are due to the amount of effort or expertise that has been spent on the codebase, not the amount of unsafe in it.