bittorrent-dht VS specs

I'm not talking about the consensus protocol of the blockchain itself, but of the p2p algorithms underlying it, e.g. using Kademlia for service discovery and message routing. I'm asking why a distributed system would choose something like Consul (which uses Raft, and requires a coordinator node) instead of running a decentralized protocol like Kademlia (which has no coordinator nodes) within their distributed single-tenant environment.

I did a bit more research last night, and discovered that Bitfinex actually does something like this internally (anyone know if this is up to date?) [0] — they built a service discovery mesh by storing arbitrary data on a DHT implementing BEP44 (using webtorrent/bittorrent-dht [1]).

This seems pretty cool to me, and IMO any modern distributed system should consider running decentralized protocols to benefit from their robustness properties. Deploying a node to a decentralized protocol requires no coordination or orchestration, aside from it simply joining the network. Scaling a service is as simple as joining a node to the network and announcing its availability of an implementation of that service.

At first glance, this looks like a competitive advantage, because it decouples the operational and maintenance costs of the network from the size of the network.

So I'm wondering if there is a consistent tradeoff in exchange for this robustness — are decentralized applications more complex to implement but simpler to operate? Is latency of decentralized protocols (e.g. average number of hops to lookup item in a DHT) untenably higher than that of distributed protocols (e.g. one hop once to get instructions from coordinator, then one hop to lookup item in distributed KV)? Does a central coordinator eliminate some kind of principle agent problem, resulting in e.g. a more balanced usage of the hashing keyspace?

Decentralization emerged because distributed solutions fail in untrusted environments — but this doesn't mean that decentralized solutions fail in trusted environments. So why not consider more decentralized protocols to scale internal systems?

[0] https://github.com/bitfinexcom/grenache

[1] https://github.com/webtorrent/bittorrent-dht

Yes the database is fixed. I would like to make it updateable using web2web or mutable torrents (BEP44) which the WebTorrent DHT supports.

The beauty of ipfs is the transport protocols are completely modular. They do a pretty good job supporting a lot of variety a separating concerns via https://github.com/libp2p/specs

For p2p networking, I'd say things are pretty interesting and boring at the same time. (Read: https://github.com/libp2p/specs if you're interested and decide for yourself)

At the bottom is the link to the more technical specification: https://github.com/libp2p/specs/blob/master/kad-dht/README.m...

The problems you described are specific to implementations, not the protocol itself. I have read all of the QUIC specs in full (since I'm working on an implementation) and have seen nothing in any of them that mandates a centralised certificate infrastructure (caveat: I have not read the HTTP/3 spec, perhaps you point out the relevant section if its in there). Of course, the most common use case requires this, but in that respect it's no different to HTTPS.

IPFS uses QUIC as one of its supported transport protocols, and this works in the most common implementation, Kubo [1]. The spec for the QUIC transport used in IPFS [2] indicates the same certificate trust policy as for the TLS protocol [3]. The latter, in turn, relies on peer-to-peer authentication with automatically-generated self-signed certificates and the use of an additional extension.

IPFS is particularly well suited to the use case of personal websites you've mentioned, as it's specifically designed to operate without any form of centralisation.

[1] https://github.com/ipfs/kubo.

[2] https://github.com/libp2p/specs/tree/master/quic

[3] https://github.com/libp2p/specs/blob/master/tls/tls.md

Yes, there is already a Rust version (https://github.com/libp2p/rust-libp2p) that behaves well at this level but I think we can reach a higher level of performance on this point with Zig. Also, if you look at the long term roadmap of libp2p (https://github.com/libp2p/specs/blob/master/ROADMAP.md), the mobile devices and IoT integrations for example are part of the considerations.

A standalone daemon that provides libp2p circuit relay services, for both protocol versions v1 and v2.

Interestingly, that section also links to one about relay connections, which seems to be closely related to the original question: https://github.com/libp2p/specs/blob/master/relay/circuit-v2.md

Hi HN,

during December 2022, we are running a measurement campaign to investigate decentralized NAT hole punching success rates using the libp2p DCUtR protocol [0]. Ubiquitous peer-to-peer connectivity is still a big challenge. If successful, NAT Hole Punching can be a game-changer for decentralised applications and networks!

For that we are searching for participants who would run a lean client on their machines that performs hole punches with other peers and then reports back the results to our server. We explained the measurement methodology in this video [1] and the linked repository above.

Running such a client certainly has privacy implications which are documented here [2]. Most importantly, we record public IP addresses, successful NAT port mappings, and the login router page (to draw conclusions about which routers work better than others).

Optionally, you can also sign up here [3] and provide additional information about your personal network and receive a personal API key so that we can link your data to your information. Obviously, this has stronger privacy implications - but this is totally optional.

The most frictionless way to participate is to head to the releases page [4] and download a client that suits your platform and needs. No sign-up required.

[0] https://github.com/libp2p/specs/blob/master/relay/DCUtR.md

Rendezvous point: The rendezvous protocol is a lightweight mechanism for generalized peer discovery. It allows for the discovery of peers in a decentralized fashion. We operate a community rendezvous point through which swap providers can make themselves known to users, and through which users can find swap providers with whom they want to swap.(/dns4/discover.unstoppableswap.net/tcp/8888/p2p/12D3KooWA6cnqJpVnreBVnoro8midDL9Lpzmg8oJPoAGi7YYaamE)

It's not regular pubsub, it's "peer to peer pubsub". It's a pubsub, but p2p, anyone can join, subscribe, publish. The libp2p project has an implementation of this https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md