postage-rs VS flume

Here are a few things that I learned from writing postage that might be helpful: - Poll::Pending can cause subtle deadlock bugs. If a channel endpoint is going to return Poll::Pending, it needs to check conditions, register for notification, and then re-check those conditions before returning Pending. Otherwise, the endpoints could deadlock due to concurrent interactions between the channel state and the notifier. Here's an example fix on an mpsc channel: https://github.com/austinjones/postage-rs/commit/9d2ba3f83dcbbcef83c684462f5efc5b3e7681cc - The single-threaded cases around sender/receiver polls can be covered in unit tests. Tests can verify the poll result, as well as waker interactions. Here's an example: https://github.com/austinjones/postage-rs/blob/main/src/channels/mpsc.rs#L242

I ran into the same thing with Stream - had tons of code that merged/mapped/filtered channel receivers. I ended up deciding to write an async channel library called postage to replace that functionality.

In case anyone is late to the party, I published v0.4.0 with: - a dispatch channel (mpmc queue) - borrow_mut for the watch sender (so you can mutate the value stored in the channel) - .blocking_send() and .blocking_recv() - And a CI matrix, removing the 'beta' tag.

I'm working on postage, an async channel library. Postage provides mpsc/broadcast/watch/oneshot channels, as well as Sink and Stream combinators.

The repository seems abandoned; or maybe complete?

At work we use flume, which is another capable multi-producer, multi-consumer async-capable channel [1]. It's great for shuffling data between threads, as well as between async tasks, and between threads and async tasks. Basically any time you want to pieces of code to exchange data or signals without pesky shared state.

1: https://github.com/zesterer/flume

I also like flume, it has impressive performance (although not the best). More importantly, it's written only with safe rust. https://github.com/zesterer/flume

The most commonly suggested replacement for mspc is crossbeam-channel; flume is also relatively popular.

It's not officially deprecated, but the alternatives on crates.io are considered better. flume and crossbeam-channel feature less unsafe code and offer better performance. Benchmarks.

it's slow (checkout flume's benchmarks for example)

I would have done the same. I think, and I might be wrong, but the only other alternative, besides anything unsafe, would be to pass mutex back, but I am not sure this would be faster. Btw, I have not done testing, but you might want to look at Flume for your mpsc channels: https://github.com/zesterer/flume Flume, seems to be very fast mpsc implementation. I am planning to evaluate it for logging system.

For concurrency/parallelism, you launch at most 2 * CPU Cores, PIN them and use a fast broker to spread the task (like a ring buffer or an MPSC). But you keep linear scan, tight loops, SIMD friendly data, on each. You are not switching context that much, and instead, bet you will process the batch fast. (CPUs are fast today!)

https://crates.io/crates/flume and https://crates.io/crates/crossbeam-channel provide MPMC channels.