mininet VS dettrace

mininet [1] can be used to emulate networks on a single computer. It is used to teach networking concepts.

1. https://github.com/mininet/mininet

You can learn routing even on a host in Linux with iptables as well as Linux network namespaces via mini net http://mininet.org/ etc. once you’re comfortable there you can start looking at docker networking and also frr as well as gns3 or eve-ng running virtualized routers.

Or instead of using Docker, you can also get a Mininet VM, (optionally) install a desktop environment on it and create any manner of network configurations to tinker with, and use Wireshark to analyse traffic.

You can learn BGP with mininet: https://mininet.org/

You can simulate arbitrarily large networks and internetworks with this, provided you have the hardware to run a large enough number of virtual appliances, but they are pretty lightweight.

Ubuntu https://github.com/mininet/mininet/releases/

Note that this is a follow-on project from the earlier Dettrace system, which was applied mainly to reproducible builds (as in the academic paper, https://dl.acm.org/doi/10.1145/3373376.3378519, and presented to the Debian Reproducible Builds summit):

- https://github.com/dettrace/dettrace

And one cool part of it is this Rust program instrumentation layer:

- https://github.com/facebookexperimental/reverie

It's good for building OS-emulator style projects or tracing tools.

We've started looking into eBPF a bit - IIUC eBPF by itself doesn't give us the ability to service or arbitrarily manipulate the traced process's syscalls.

We have recently learned of an interesting technique that dettrace [1] uses of combining seccomp with an eBPF filter and ptrace. Instead of generating a ptrace-stop for every syscall (as we do now, using PTRACE_SYSEMU), they use a seccomp policy with an eBPF filter, s.t. a ptrace-stop is only generated for syscalls that violate the policy, allowing them to emulate the result of those syscalls. syscalls that don't violate the policy are allowed to execute natively, saving a lot of overhead.

[1]: https://github.com/dettrace/dettrace

This works great for them since they want to emulate a relatively small subset of syscalls. In our case we want to emulate most syscalls, so it's not as clear-cut of a win. We have found though that if we use an LD_PRELOAD'd shim in the target process to intercept syscalls and then service them via IPC, that's substantially faster than catching them with ptrace. That runs back into the problems with LD_PRELOAD in general of there being various ways of missing syscalls. but, we may be able to use that technique along with ptrace+seccomp+ebpf to intercept any syscalls that we'd otherwise miss. The seccomp technique would allow us to exempt the syscalls that our shim itself is making to do the IPC.