flume VS crossbeam

I also like flume, it has impressive performance (although not the best). More importantly, it's written only with safe rust. https://github.com/zesterer/flume

The most commonly suggested replacement for mspc is crossbeam-channel; flume is also relatively popular.

It's not officially deprecated, but the alternatives on crates.io are considered better. flume and crossbeam-channel feature less unsafe code and offer better performance. Benchmarks.

it's slow (checkout flume's benchmarks for example)

For concurrency/parallelism, you launch at most 2 * CPU Cores, PIN them and use a fast broker to spread the task (like a ring buffer or an MPSC). But you keep linear scan, tight loops, SIMD friendly data, on each. You are not switching context that much, and instead, bet you will process the batch fast. (CPUs are fast today!)

First, I was becoming more and more confident that the channel library Daxpedda and I fell in love with, flume, was misbehaving, but I couldn't seem to reproduce it outside of the massive PliantDB codebase. I finally called up Daxpedda on Discord and screen shared my debugging session, showing him how the tests succeeded if I retained a channel. If I allowed the sender to drop after successfully sending, sometimes the tests would fail. He agreed, something was odd. It took me a while, but I finally whittled it down to about 30 lines of code and reported the issue. In an amazingly quick fashion, the maintainer fixed the issue and released an update. And for the record, I still fully love and recommend this library if you're mixing async and non-async code using channels. It's a wonderful implementation.

Sort of? Here's the closest thing you'll probably find: https://crates.io/crates/flume

My first try was with flume, which uses a single producer, multi-consumer setup, and while it compiles and runs, it's magnitudes slower than the serial version. Here's a psuedocode rendition:

Shooting from the hip, crossbeam might be a good candidate for understanding the thread safety aspects of Rust. I kind of feel like this is probably "too big" of a project if you're just learning, but I can't think of something smaller off the top of my head that would be suitable.

The crossbeam crate offers a powerful alternative to standard channels with support for the Select operation, timeouts, and more.

Deadlocks and leaks are easy as other languages.

Ouch, didn’t know about https://github.com/crossbeam-rs/crossbeam/issues/821, thanks for pointing that out, that’s a big update for me!

The last option I can think of is using two threads (like above) and epoch GC instead of a lock (i.e. using crossbeam-epoch). But I don't have enough experience with this to say anything about it.

You should definitely look at the Rayon crate. After I figured that out, my productivity vs multi threaded C was exponentially better (literally did many days of equivalent C work in less than an hour). Rayon is focused on data processing, other multi threaded stuff can benefit from other crates like crossbeam (Channels, scoped threads), and tokio (async networking).