Our great sponsors
-
shadow
Shadow is a discrete-event network simulator that directly executes real application code, enabling you to simulate distributed systems with thousands of network-connected processes in realistic and scalable private network experiments using your laptop, desktop, or server running Linux. (by shadow)
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
testground
🧪 A platform for testing, benchmarking, and simulating distributed and p2p systems at scale.
-
-
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
For anyone wondering how this might actually work, "Shadow intercepts a selective set of system calls to enable seamless integration of an application to the simulated environment."
Or more expansively: "Plug-ins are shared library shims that are linked to real applications. Shadow dynamically loads these libraries to natively execute the application code. Shadow intercepts a selective set of system calls to enable seamless integration of an application to the simulated environment. In this way, the application may be unaware that it is running in the simulator and will function as if it was running in a standard UNIX environment." https://github.com/shadow/shadow/blob/main/docs/0-Design-Ove...
Any reason this type of work couldn't be done with mininet (http://mininet.org/)?
I can see this was developed for NRL research, so not trying to say it's a knock off or anything. Seems to have been developed a few years later, but close to the same time, one of those things like Deep Impact and Armageddon both coming out at the same time.
Anyway, having used mininet quite a bit back in the day in grad school, this kind of thing is great. Not sure exactly the extent of networking you can simulate with this toolset, but mininet lets you put entire simulated autonomous systems on a single laptop and you can mess around with BGP policies. It's really cool, stuff you would normally need to work for AT&T with access to millions of miles of fiber to have a chance to do.
A quick look (by searching for bgp) suggests that this handles the upper layers, but doesn't try to simulate some of the deeper layers of a network such as bgp. Which something that emulators like core can do [0].
0. https://github.com/coreemu/core.
related, but "higher level" project: project:https://github.com/testground/testground
The in-repo docs have a general walk-through for getting started: https://github.com/shadow/shadow/tree/main/docs.
Shadow's primary use-case is simulating the Tor network. Shadow's tor plugin has a step-by-step for getting a tor network simulation in particular running: https://github.com/shadow/shadow-plugin-tor/wiki
We've started looking into eBPF a bit - IIUC eBPF by itself doesn't give us the ability to service or arbitrarily manipulate the traced process's syscalls.
We have recently learned of an interesting technique that dettrace [1] uses of combining seccomp with an eBPF filter and ptrace. Instead of generating a ptrace-stop for every syscall (as we do now, using PTRACE_SYSEMU), they use a seccomp policy with an eBPF filter, s.t. a ptrace-stop is only generated for syscalls that violate the policy, allowing them to emulate the result of those syscalls. syscalls that don't violate the policy are allowed to execute natively, saving a lot of overhead.
[1]: https://github.com/dettrace/dettrace
This works great for them since they want to emulate a relatively small subset of syscalls. In our case we want to emulate most syscalls, so it's not as clear-cut of a win. We have found though that if we use an LD_PRELOAD'd shim in the target process to intercept syscalls and then service them via IPC, that's substantially faster than catching them with ptrace. That runs back into the problems with LD_PRELOAD in general of there being various ways of missing syscalls. but, we may be able to use that technique along with ptrace+seccomp+ebpf to intercept any syscalls that we'd otherwise miss. The seccomp technique would allow us to exempt the syscalls that our shim itself is making to do the IPC.