bzip2 VS memberlist

Sometimes rebuilding something that already exists can be very useful! For example, a bzip2 compressor/decompressor already exists, but a memory-safe one doesn't. The maintainer has started porting to Rust, but ultimately didn't have the time. If you rebuild bzip2 in Rust, that will be a very valuable project!

Bzip2 upstream started an incremental Rust port, but it seems the upstream maintainers are busy with other projects. Help would be welcome.

I use lightweight https://github.com/hashicorp/memberlist to build initial cluster skeleton (find out what are nodes constituting it)

Does https://github.com/hashicorp/memberlist count? It’s a gossip-based eventual consistency protocol based on SWIM.

An equivalent of golang's memberlist would be awesome.

I don't know the engineer that did it but I know what happened. There's a library that a bunch of foundational Amazon services use called DFDD. It uses a gossip protocol to handle service discovery and health checking. The open source equivalent is Hashicorp memberlist. To remove a node from the cluster, you have to send a command to an arbitrary node that says a node is dead.

serf uses memberlist which uses the SWIM failure detection protocol (https://www.cs.cornell.edu/projects/Quicksilver/public_pdfs/SWIM.pdf) with the Lifeguard extensions (https://arxiv.org/pdf/1707.00788.pdf).

https://github.com/hashicorp/memberlist

Fairly idiomatic/clean

This algorithm assumes you're doing that separately using some sort of membership protocol (e.g. SWIM), and you pass in the membership list that you are keeping up to date with that protocol. If you're curious about those, HashiCorp memberlist and HashiCorp Serf are really good to look at. I believe Consul uses those libraries under the hood.