sysbox
systemd
Our great sponsors
sysbox | systemd | |
---|---|---|
22 | 510 | |
2,517 | 12,457 | |
3.5% | 1.8% | |
8.6 | 10.0 | |
9 days ago | about 6 hours ago | |
Shell | C | |
Apache License 2.0 | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
sysbox
-
Podman Desktop: A Free OSS Alternative to Docker Desktop
You are probably referring to Sysbox (https://github.com/nestybox/sysbox), which I believe will meet your requirements (systemd, inner containers, security, etc).
Btw, Sysbox is already supported in Docker-Desktop (business tier only), so you can easily do what you want with this instruction:
$ docker run -it --rm -e SYSBOX_SYSCONT_MODE=TRUE nestybox/ubuntu-focal-systemd-docker:latest bash
Disclaimer: I'm Sysbox's co-creator and currently working for Docker.
- Sysbox: VM-Like Containers
- What companies are using golang and have source code in github?
-
SELinux is unmanageable; just turn it off if it gets in your way
One project in this space that looked quite promising to me is sysbox[0]. I've used them once for a gitlab runner set-up similar to what is described in their blog[1].
It's currently working great and I have not had any major crashes/incidents for at least the past 8 months.
[0]: https://github.com/nestybox/sysbox
[1]: https://blog.nestybox.com/2020/10/21/gitlab-dind.html
-
Jenkins in Docker: Running Docker in a Jenkins container
Today, things are very different. Docker-in-Docker has a more secure and safe approach with rootless containers and freemium tools like sysbox. Tools like sysbox let you run Docker-in-Docker without the -privileged flag and optimizes specific scenarios, like running multiple nodes of a Kubernetes cluster as ordinary containers.
-
Run untrusted code in sandbox
Right now I am going with sysbox rootless containers. https://github.com/nestybox/sysbox
-
Real-world stories of how we’ve compromised CI/CD pipelines
We’ve been using Sysbox (https://github.com/nestybox/sysbox) for our Buildkite based CI/CD setup, allows docker-in-docker without privileged containers. Paired with careful IAM/STS design we’ve ended up with isolated job containers with their own IAM roles limited to least-privilege.
-
Individual Docker Desktops vs hosting on a server?
A good alternative to the VM approach is to use Kubernetes + Sysbox (a next-gen "runc", free, open-source).
- Sysbox now works on K8s v1.21
-
Does running a container with privileged mode turn on allow code to escape into the Host ?
But nowadays there is an option to run such software in containers securely. It's called Sysbox, and it's a new "runc" (the piece of software that creates the containers). I am one of the developers, so I am biased, but I think you'll find it helpful.
systemd
-
Linux fu: getting started with systemd
https://github.com/systemd/systemd/issues/32028#issuecomment...
There are some very compelling arguments made there if you care to read them
-
Ubuntu 24.04 (and Debian) removed libsystemd from SSH server dependencies
Maybe it was because you weren't pointing out anything new?
There was a pull request to stop linking libzma to systemd before the attack even took place
https://github.com/systemd/systemd/pull/31550
This was likely one of many things that pushed the attackers to work faster, and forced them into making mistakes.
-
Systemd minimizing required dependencies for libsystemd
The PR for changing compression libraries to use dlopen() was opened several weeks before the xz-utils backdoor was revealed.
https://github.com/systemd/systemd/pull/31550
- Going in circles without a real-time clock
-
The xz sshd backdoor rabbithole goes quite a bit deeper
I find this the most plausible explanation by far:
* The highly professional outfit simply did not see teknoraver's commit to remove liblzma as standard dependency of systemd build scripts coming.
* The race was on between their compromised code and that commit. They had to win it, with as large a window as possible.
* This caused serious errors.
* The performance regression is __not__ big. It's lucky Andres caught it at all. It's also not necessarily all that simple to remove it. It's not simply a bug in a loop or some such.
* The payload of the 'hack' contains fairly easy ways for the xz hackers to update the payload. They actually used it to remove a real issue where their hackery causes issues with valgrind that might lead to discovering it, and they also used it to release 5.6.1 which rewrites significant chunks; I've as yet not read, nor know of any analysis, as to why they changed so much.
Extra info for those who don't know:
https://github.com/systemd/systemd/commit/3fc72d54132151c131...
That's a commit that changes how liblzma is a dependency of systemd. Not because the author of this commit knew anything was wrong with it. But, pretty much entirely by accident (although removing deps was part of the point of that commit), almost entirely eliminates the value of all those 2 years of hard work.
And that was with the finish line in sight for the xz hackers: On 24 feb 2024, the xz hackers release liblzma 5.6.0 which is the first fully operational compromised version. __12 days later systemd merges a commit that means it won't work__.
So now the race is on. Can they get 5.6.0 integrated into stable releases of major OSes _before_ teknoraver's commit that removes liblzma's status as direct dep of systemd?
I find it plausible that they knew about teknoraver's commit _just before_ Feb 24th 2024 (when liblzma v5.6.0 was released, the first backdoored release), and rushed to release ASAP, before doing the testing you describe. Buoyed by their efforts to add ways to update the payload which they indeed used - March 8th (after teknoraver's commit was accepted) it was used to fix the valgrind issue.
So, no, I don't find this weird, and I don't think the amateurish aspects should be taken as some sort of indication that parts of the outfit were amateuristic. As long as it's plausible that the amateuristic aspects were simply due to time pressure, it sounds like a really bad idea to make assumptions in this regard.
-
Excellent succinct breakdown of the xz mess, from an OpenBSD developer
sshd is started by systemd.
systemd has several ways of starting programs and waiting until they're "ready" before starting other programs that depend on them: Type=oneshot, simple, exec, forking, dbus, notify, ...
A while back, several distro maintainers found problems with using Type=exec (?) and chose Type=notify instead. When sshd is ready, it notifies systemd. How you do notification is you send a datagram to systemd's unix domain socket. That's about 10 lines of C code. But to make life even simpler, systemd also provides the one-line sd_notify() call, which is in libsystemd.so. This library is so other programmers can easily integrate with systemd.
So the distro maintainers patched sshd to use the sd_notify() function from libsystemd.so
What else is in libsystemd.so? That's right, systemd also does logging. All the logging functions are in there, so user programs can do logging the systemd way. You can even _read_ logs, using the functions in libsystemd.so. For example, sd_journal_open_files().
By the way... systemd supports the environment variable SYSTEMD_JOURNAL_COMPRESS which can be LZ4, XZ or ZSTD, to allow systemd log files to be compressed.
So, if you're a client program, that needs to read systemd logs, you'll call sd_journal_open_files() in libsystemd.so, which may then need liblz4, liblzma or libzstd functions.
These compression libraries could be dynamically loaded, should sd_journal_open_files() need them - which is what https://github.com/systemd/systemd/pull/31550 submitted on the 29th February this year did. But clearly that's not in common use. No, right now, most libsystemd.so libraries have headers saying "you'll need to load liblz4.so, liblzma.so and libzstd before you can load me!", so liblzma.so gets loaded for the logging functions that sshd doesn't use, so the distro maintainers of sshd can add 1 line instead of 10 to notify systemd that sshd is ready.
-
Reflections on Distrusting xz
They just added an example to the documentation[0] of how to implement the sd_notify protocol without linking to libsystemd, so a little bit of discarding systemd (or at least parts of it) does seem to be part of the solution.
[0] https://github.com/systemd/systemd/pull/32030/files
-
Timeline of the xz open source attack
I think this analysis is more interesting if you consider these two events in particular:
2024-02-29: On GitHub, @teknoraver sends pull request to stop linking liblzma into libsystemd.[1]
2024-03-20: The attacker is now a co-contributor for a patchset proposed to the Linux kernel, with the patchset adding the attacker as a maintainer and mirroring activity with xz-utils.
A theory is that the attacker saw the sshd/libsystemd/xz-utils vector as closing soon with libsystemd removing its dependency on xz-utils. When building a Linux kernel image, the resulting image is compressed by default with gzip [3], but can also be optionally compressed using xz-utils (amongst other compression utilities). There's a lot of distributions of Linux which have chosen xz-utils as the method used to compress kernel images, particularly embedded Linux distributions.[4] xz-utils is even the recommended mode of compression if a small kernel build image is desired.[5] If the attacker can execute code during the process of building a new kernel image, they can cause even more catastrophic impacts than targeting sshd. Targeting sshd was always going to be limited due to targets not exposing sshd over accessible networks, or implementing passive optical taps and real time behavioural analysis, or receiving real time alerts from servers indicative of unusual activity or data transfers. Targeting the Linux kernel would have far worse consequences possible, particularly if the attacker was targeting embedded systems (such as military transport vehicles [6]) where the chance of detection is reduced due to lack of eyeballs looking over it.
[1] https://github.com/systemd/systemd/pull/31550
[2] https://lkml.org/lkml/2024/3/20/1004
[3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
[4] https://github.com/search?q=CONFIG_KERNEL_XZ%3Dy&type=code
[5] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
[6] https://linuxdevices.org/large-military-truck-runs-embedded-...
-
What we know about the xz Utils backdoor that almost infected the world
systemd merged a change to using dlopen for compression libraries recently https://github.com/systemd/systemd/pull/31550 which is a safer linking method in that sense.
-
XZ: A Microcosm of the interactions in Open Source projects
1) Debian includes this downstream patch, also.
2) A potential explanation for "why now" is that systemd DID prevent these dependencies from loading automatically in a patch one month ago, and the patches enabling the backdoor merged a few days later. It could be a total coincidence or it could be that the attacker was trying to catch the window before it was closed on them https://github.com/systemd/systemd/pull/31550#issuecomment-1...
What are some alternatives?
kata-containers - Kata Containers is an open source project and community working to build a standard implementation of lightweight Virtual Machines (VMs) that feel and perform like containers, but provide the workload isolation and security advantages of VMs. https://katacontainers.io/
openrc - The OpenRC init system
containerd - An open and reliable container runtime
tini - A tiny but valid `init` for containers
dind - Docker in Docker
inotify-tools - inotify-tools is a C library and a set of command-line programs providing a simple interface to inotify.
gvisor - Application Kernel for Containers
s6 - The s6 supervision suite.
gatekeeper - 🐊 Gatekeeper - Policy Controller for Kubernetes
earlyoom - earlyoom - Early OOM Daemon for Linux
snekbox - Easy, safe evaluation of arbitrary Python code
supervisor - Supervisor process control system for Unix (supervisord)