any-sync VS rdv

Co-founder of Anytype is here [1]. Excited to share the multi-player release is live!

Last time I shared how Anytype can be used to build encrypted, local-first spaces around your interests [2]. Today, these spaces can be shared and co-created together: with your family members, small communities, teams, neighbors and people with similar interests. Examples: https://gallery.any.coop/Collaboration.

When we were testing collaboration internally for the first time, it felt like magic - we as a team were editing the same page, even after switching off the Internet - changes happened in an instant and all of them were synced via wi-fi.

Unlike cloud applications, in Anytype the encryption keys are stored locally, so only creators have access to them, so they own the space with its data and social graph. Every change is cryptographically signed, spaces can be created offline and can sync in peer to peer ways in local networks. This is enabled by open source protocol AnySync that supports high-performant collaboration over encrypted data and creators’ controlled keys. All our code is open on Github - https://github.com/anyproto/any-sync.

This first version of collaboration is very basic - it’s an alpha. It’s far from polished. We will focus on making it complete by adding notifications, public spaces, comments, and many other essential features in the coming months.

Why we’ve built it? Why anytype is the way it is?

Haha that’s currently a way to explain to people who are used to hosting/sync based cloud solutions that anything that is (a) local p2p and (b) doesn’t need to upload-before-download is much faster. It’s also faster than WebRTC based solutions which there are dozens, WebRTC kind of sucks for large stuff.

That said, the next version will have multi connection tcp striping, which is a lot faster than any single tcp solution in many cases, especially over long distances, similar to some ftp/usenet clients. (Spoiler there will be online p2p transfers. See https://github.com/betamos/rdv if curious)

This is an incredible resource on NAT traversal in practice. I used it as a reference while building a server-assisted p2p-over-TCP[1] system. The post covers UDP but all of the theory applies to TCP as well – you just need some SO_REUSEPORT dual dial/listen magic so the code will look a bit different.

The post alludes to this, but I think NAT traversal is an inaccurate and confusing term. I like “hole punching” a lot more as a general term. It gets the message across without the myopic connotation that it’s all about NATs – it isn’t.

[1]: https://github.com/betamos/rdv

> In trying to come up with a WebRTC solution, I think I've settled on defeat. The local network syncing works like a charm, but I haven't had any luck in trying to get around a TURN server by using an API endpoint to provide routing data. It would be nice if there were other prebuilts than COTURN (something deployable to a deno/node server would be ideal),

The bad news is that if you want something that works in all instances, you need a relay of _some_ sort, because p2p isn't possible/feasible in all cases. Bittorrent (for instance) works around that limitation by simply having many-to-many peering and relying on large numbers, to be reliable. But that doesn't work for 1:1.

The good news is that maintaining a relay (or TURN, with WebRTC), need not be expensive. Yes, you need a server, and perhaps some IP-based rate limiting, but that can handle a LOT of connections and small data.

I created https://github.com/betamos/rdv for this purpose, an extremely light-weight alternative to WebRTC, but for TCP only (BYO identity, auth and encryption). The p2p success rate is, anecdotally, very high. However, you cannot use it from a web browser.

Feel free to reach out (see profile), happy to chat about p2p whether or not you use this project.