futures-rs VS memberlist

So this is actually better than true fairness (true fairness would lead to deadlock if a sender is forgotten). It is a pity that the there does not seem to be resources that document this design. There is this old thread where Carl provides some background, but I found it personally a bit hard to follow.

Is this still true after it switched to using FuturesOrdered?

Done.

You can use Streams, which are the async version of Iterators; They aren't stable yet, so you'll have to use a crate such as futures.

futures

For traits, like AsyncRead and AsyncWrite, go with the futures crate.

Here is the PR: https://github.com/rust-lang/futures-rs/pull/2551

Yield = wake the `waker_ref`. Avoiding the yield would be clone().wake().

That said, "poll immediately" isn't actually a thing nor was it ever a thing except in incorrect implementations.

Rust lacks an implementation of ReactiveX. futures/futures-signals seems to be the the ecosystem equivalent but I'm sure there'd be a lot of interest in an actual implementation.

The opposite problem exists as well. Many types are actually unwind safe, but do not get the autotrait. In that case authors would have to manually declare them UnwindSafe. Because this is rarely done, having an API with a trait bound T: UnwindSafe is rarely viable in terms of ergonomics. It now obliges client code to wrap all calls to your API in AssertUnwindSafe which, if they use types from third party libraries, obliges them to assert this is fine. example

Also, we plan to give users more control in the future. See https://github.com/rust-lang/futures-rs/issues/2207, https://github.com/rust-lang/futures-rs/issues/2295, etc. for this

I use lightweight https://github.com/hashicorp/memberlist to build initial cluster skeleton (find out what are nodes constituting it)

Does https://github.com/hashicorp/memberlist count? It’s a gossip-based eventual consistency protocol based on SWIM.

An equivalent of golang's memberlist would be awesome.

I don't know the engineer that did it but I know what happened. There's a library that a bunch of foundational Amazon services use called DFDD. It uses a gossip protocol to handle service discovery and health checking. The open source equivalent is Hashicorp memberlist. To remove a node from the cluster, you have to send a command to an arbitrary node that says a node is dead.

serf uses memberlist which uses the SWIM failure detection protocol (https://www.cs.cornell.edu/projects/Quicksilver/public_pdfs/SWIM.pdf) with the Lifeguard extensions (https://arxiv.org/pdf/1707.00788.pdf).

https://github.com/hashicorp/memberlist

Fairly idiomatic/clean

This algorithm assumes you're doing that separately using some sort of membership protocol (e.g. SWIM), and you pass in the membership list that you are keeping up to date with that protocol. If you're curious about those, HashiCorp memberlist and HashiCorp Serf are really good to look at. I believe Consul uses those libraries under the hood.