Nebula VS tinc

(I am a Nebula maintainer.) We recently merged support for gVisor-based services, although it's very new, and I don't know of much experimentation that's been done with it yet: https://github.com/slackhq/nebula/pull/965

Nebula, originally from Slack[0].

Wireguard rightly gets a lot of attention, but Nebula is a really simple and easy to deploy mesh network that is often overlooked.

It does lack a management GUI and that stuff is very much DIY.

[0] https://github.com/slackhq/nebula

Fair enough about the android mobile client... My use case only involves meshing linux appliances across various networks so we only need the nebula core binaries which are under MIT license

https://github.com/slackhq/nebula/blob/master/LICENSE

That's not at all confusing with Slack's Nebula. https://github.com/slackhq/nebula

Sounds like a bunch of your pain points are just related to needing an online CA or ICA. But, looking through the Nebula docs I don't know that it supports things like CRL addresses where you could host the CRL, or OCSP responders. Someone got support for an OCSP responder but never submitted a PR with completed code: https://github.com/slackhq/nebula/issues/72

Nebula is a scalable, cross-platform overlay networking tool focused on performance, simplicity, and security. This portable tool is equally adapted for linking a small number of computers or scaling to connect tens of thousands. It integrates encryption, security groups, certificates, and tunneling into a powerful, cohesive connectivity solution. Thanks for the recommendation go to jmeador42.

Replying to my own comment as I can no longer edit it:

The folks over at Slack had an interesting discussion regarding the the "battle of the VPNs" article published by Netmaker I sourced in my parent comment:

https://github.com/slackhq/nebula/discussions/911

Interesting. I thought recognized the logo, apparently seems to be a commercial support offering of https://github.com/slackhq/nebula and they support the "nebula" iOS app. I had been using for nebula/defined in the past.

Nebula: Is super easy to get running. It uses an interesting angle, working on the service and not just the device level. Unfortunately their NAT support seems to be still quite problematic and I am not going to maintain all those forwarded ports manually. There is a PR to support PCP but even if that ever gets applied I am not sure how well that will play with older routers. While it should be battle proven at slack, the community seems to be not that active. It still has the in-house tool that just got released.

The catch is that I want this to be reliable and fault tolerant, so if some of the game servers in the network go down, the remaining online servers should still always be able to receive broadcasts from any other online server. The servers can also be in multiple geographic locations and I am planning on using a mesh overlay network like Nebula to connect them. Essentially each pair of online servers will likely have a secure link between them that goes directly through the underlying network.

But both Nebula and tinc max out at around 1 Gbit/s on my Hetzner servers, thus not using most of my 10 Gbit/s connectivity. This is because they cap out at 100% of 1 CPU. The Nebula issue about that was closed due to "inactivity" [2].

I also observed that when Nebula operates at 100% CPU usage, you get lots of package loss. This causes software that expects reasonable timings on ~0.2ms links to fail (e.g. consensus software like Consul, or Ceph). This in turn led to flakiness / intermittent outages.

I had to resolve to move the big data pushing softwares like Ceph outside of the VPN to get 10 Gbit/s speed for those, and to avoid downtimes due to the packet loss.

Such software like Ceph has its own encryption, but I don't trust it, and that mistrust was recently proven right again [3].

So I'm currently looking to move the Ceph into WireGuard.

Summary: For small-data use, tinc and Nebula are fine, but if you start to push real data, they break.

[1]: https://github.com/gsliepen/tinc/issues/218

[2]: https://github.com/slackhq/nebula/issues/637

[3]: https://github.com/google/security-research/security/advisor...

From a purely networking perspective, there are far better solutions than tailscale.

Have a look at full mesh VPNs like:

https://github.com/cjdelisle/cjdns

https://github.com/yggdrasil-network/yggdrasil-go

https://github.com/gsliepen/tinc

https://github.com/costela/wesher

These build actual mesh networks where every node is equal and can serve as a router for other nodes to resolve difficult network topologies (where some nodes might not be connected to the internet, but do have connections to other nodes with an internet connection).

Sending data through multiple routers is also possible. They also deal with nodes disappearing and change routes accordingly.

tailscale (and similar solutions like netbird) still use a bunch of "proxy servers" for that. You can set them up on intermediate nodes, but that have to be dealt with manually (and you get two kinds of nodes).

Two other options are Tinc https://tinc-vpn.org/ or Nebula https://www.defined.net/nebula/

And there is Tinc; the OG overlay network. I don't have experience with this. Seemed a bit of a pain to setup. https://tinc-vpn.org

For what its worth I have used the open source Tinc VPN [1] for mesh multihop routing for ages. It is nowhere near as fast as Wireguard but I could envision Tinc incorporating support for Wireguard if the author were so inclined. Like you mentioned Tinc does not mesh with other VPN's AFAIK.

[1] - https://tinc-vpn.org/

This is actually very simple in concept and is just as simple or even simpler to do with tinc (https://tinc-vpn.org).

Since I can use tinc in bridge mode, I can run tinc on the upstream server and on a local machine which then provides access to several physical machines without running extra software on each of those machines, which is particularly useful for machines that are resource limited, like my Macintosh LC II and LC III+:

http://elsie.zia.io/

It'd be nice if it weren't so difficult to get public addresses.

I clicked expected some broken analogy between https://tinc-vpn.org/ and the Catan board game, but instead it is a Catan implementation. Fair enough.

will generate a real-time network graph using the Graphviz DOT language. It's a cool feature that I find quite useful.

[0] https://tinc-vpn.org/