systemd
s6
Our great sponsors
systemd | s6 | |
---|---|---|
510 | 22 | |
12,432 | 726 | |
1.8% | - | |
10.0 | 7.4 | |
about 23 hours ago | 1 day ago | |
C | C | |
GNU General Public License v3.0 only | ISC License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
systemd
-
Linux fu: getting started with systemd
https://github.com/systemd/systemd/issues/32028#issuecomment...
There are some very compelling arguments made there if you care to read them
-
Ubuntu 24.04 (and Debian) removed libsystemd from SSH server dependencies
Maybe it was because you weren't pointing out anything new?
There was a pull request to stop linking libzma to systemd before the attack even took place
https://github.com/systemd/systemd/pull/31550
This was likely one of many things that pushed the attackers to work faster, and forced them into making mistakes.
-
Systemd minimizing required dependencies for libsystemd
The PR for changing compression libraries to use dlopen() was opened several weeks before the xz-utils backdoor was revealed.
- Going in circles without a real-time clock
-
The xz sshd backdoor rabbithole goes quite a bit deeper
I find this the most plausible explanation by far:
* The highly professional outfit simply did not see teknoraver's commit to remove liblzma as standard dependency of systemd build scripts coming.
* The race was on between their compromised code and that commit. They had to win it, with as large a window as possible.
* This caused serious errors.
* The performance regression is __not__ big. It's lucky Andres caught it at all. It's also not necessarily all that simple to remove it. It's not simply a bug in a loop or some such.
* The payload of the 'hack' contains fairly easy ways for the xz hackers to update the payload. They actually used it to remove a real issue where their hackery causes issues with valgrind that might lead to discovering it, and they also used it to release 5.6.1 which rewrites significant chunks; I've as yet not read, nor know of any analysis, as to why they changed so much.
Extra info for those who don't know:
https://github.com/systemd/systemd/commit/3fc72d54132151c131...
That's a commit that changes how liblzma is a dependency of systemd. Not because the author of this commit knew anything was wrong with it. But, pretty much entirely by accident (although removing deps was part of the point of that commit), almost entirely eliminates the value of all those 2 years of hard work.
And that was with the finish line in sight for the xz hackers: On 24 feb 2024, the xz hackers release liblzma 5.6.0 which is the first fully operational compromised version. __12 days later systemd merges a commit that means it won't work__.
So now the race is on. Can they get 5.6.0 integrated into stable releases of major OSes _before_ teknoraver's commit that removes liblzma's status as direct dep of systemd?
I find it plausible that they knew about teknoraver's commit _just before_ Feb 24th 2024 (when liblzma v5.6.0 was released, the first backdoored release), and rushed to release ASAP, before doing the testing you describe. Buoyed by their efforts to add ways to update the payload which they indeed used - March 8th (after teknoraver's commit was accepted) it was used to fix the valgrind issue.
So, no, I don't find this weird, and I don't think the amateurish aspects should be taken as some sort of indication that parts of the outfit were amateuristic. As long as it's plausible that the amateuristic aspects were simply due to time pressure, it sounds like a really bad idea to make assumptions in this regard.
-
Excellent succinct breakdown of the xz mess, from an OpenBSD developer
sshd is started by systemd.
systemd has several ways of starting programs and waiting until they're "ready" before starting other programs that depend on them: Type=oneshot, simple, exec, forking, dbus, notify, ...
A while back, several distro maintainers found problems with using Type=exec (?) and chose Type=notify instead. When sshd is ready, it notifies systemd. How you do notification is you send a datagram to systemd's unix domain socket. That's about 10 lines of C code. But to make life even simpler, systemd also provides the one-line sd_notify() call, which is in libsystemd.so. This library is so other programmers can easily integrate with systemd.
So the distro maintainers patched sshd to use the sd_notify() function from libsystemd.so
What else is in libsystemd.so? That's right, systemd also does logging. All the logging functions are in there, so user programs can do logging the systemd way. You can even _read_ logs, using the functions in libsystemd.so. For example, sd_journal_open_files().
By the way... systemd supports the environment variable SYSTEMD_JOURNAL_COMPRESS which can be LZ4, XZ or ZSTD, to allow systemd log files to be compressed.
So, if you're a client program, that needs to read systemd logs, you'll call sd_journal_open_files() in libsystemd.so, which may then need liblz4, liblzma or libzstd functions.
These compression libraries could be dynamically loaded, should sd_journal_open_files() need them - which is what https://github.com/systemd/systemd/pull/31550 submitted on the 29th February this year did. But clearly that's not in common use. No, right now, most libsystemd.so libraries have headers saying "you'll need to load liblz4.so, liblzma.so and libzstd before you can load me!", so liblzma.so gets loaded for the logging functions that sshd doesn't use, so the distro maintainers of sshd can add 1 line instead of 10 to notify systemd that sshd is ready.
-
Reflections on Distrusting xz
They just added an example to the documentation[0] of how to implement the sd_notify protocol without linking to libsystemd, so a little bit of discarding systemd (or at least parts of it) does seem to be part of the solution.
-
Timeline of the xz open source attack
I think this analysis is more interesting if you consider these two events in particular:
2024-02-29: On GitHub, @teknoraver sends pull request to stop linking liblzma into libsystemd.[1]
2024-03-20: The attacker is now a co-contributor for a patchset proposed to the Linux kernel, with the patchset adding the attacker as a maintainer and mirroring activity with xz-utils.
A theory is that the attacker saw the sshd/libsystemd/xz-utils vector as closing soon with libsystemd removing its dependency on xz-utils. When building a Linux kernel image, the resulting image is compressed by default with gzip [3], but can also be optionally compressed using xz-utils (amongst other compression utilities). There's a lot of distributions of Linux which have chosen xz-utils as the method used to compress kernel images, particularly embedded Linux distributions.[4] xz-utils is even the recommended mode of compression if a small kernel build image is desired.[5] If the attacker can execute code during the process of building a new kernel image, they can cause even more catastrophic impacts than targeting sshd. Targeting sshd was always going to be limited due to targets not exposing sshd over accessible networks, or implementing passive optical taps and real time behavioural analysis, or receiving real time alerts from servers indicative of unusual activity or data transfers. Targeting the Linux kernel would have far worse consequences possible, particularly if the attacker was targeting embedded systems (such as military transport vehicles [6]) where the chance of detection is reduced due to lack of eyeballs looking over it.
[1] https://github.com/systemd/systemd/pull/31550
[2] https://lkml.org/lkml/2024/3/20/1004
[3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
[4] https://github.com/search?q=CONFIG_KERNEL_XZ%3Dy&type=code
[5] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
[6] https://linuxdevices.org/large-military-truck-runs-embedded-...
-
What we know about the xz Utils backdoor that almost infected the world
systemd merged a change to using dlopen for compression libraries recently https://github.com/systemd/systemd/pull/31550 which is a safer linking method in that sense.
-
XZ: A Microcosm of the interactions in Open Source projects
1) Debian includes this downstream patch, also.
2) A potential explanation for "why now" is that systemd DID prevent these dependencies from loading automatically in a patch one month ago, and the patches enabling the backdoor merged a few days later. It could be a total coincidence or it could be that the attacker was trying to catch the window before it was closed on them https://github.com/systemd/systemd/pull/31550#issuecomment-1...
s6
- OpenRC is a dependency-based init system for Unix-like systems
-
are there any good reasons for me to avoid systemd
A (rare) good critique to systemd can be found here. Written by the developer behind s6, which happens to be scheduled to replace OpenRC on Alpine Linux. For completeness-sake, some of the main reasons why Alpine doesn't prefer systemd do not apply on most other distros.
-
A discussion about the Ultimate Linux Desktop
It got mass-adopted while being imperfect, so that's to be expected. Thankfully its inception and the criticism that followed have paved the way for the likes of dinit and s6.
-
Which do you use systemd or openrc? Why do you use what you use?
this page and this page, both by Laurent Bercot, creator of s6.
-
init software: What's the difference?
Of the two I have experience with, runit is simpler and thus easier to get the hang of than s6-rc/s6. Though the s6 (not s6-rc) docs at the author's site contain a lot of info (including apologetics and rationales) that applies almost equally well to runit
-
What do you guys think about this?
systemd: Yes; it's awaiting its "PipeWire". Thankfully, the likes of s6 and dinit are very promising. Though I can actually appreciate that systemd is addressed. As ultimately it helps in raising awareness that will benefit whatever software will replace it eventually.
-
The (GNU/)Linux rabbit hole has been a negative influence on my mental state
Arguably this is less troublesome to solve compared to the other concerns. As we're inevitably waiting for the system supervision suite that will be to systemd what PipeWire has been to PulseAudio. I'm very optimistic about this as both s6 and Dinit are shaping up lovely.
- Systemd 252 Released
-
Trouble with s6 services
Using the s6-service add command I added a service called "libvertd" when I ment to put "libvirtd". Now when I run s6-db-reload it spits out a error message saying "undefined service name libvertd". But I cant remove it using s6-service remove libvertd because that just spits out a generic help message and doesn't change anything. I also couldn't find documentation on https://skarnet.org/software/s6/ or https://skarnet.org/software/s6-rc/ as the command just *doesn't* exist on those pages. (also no man or info page).
- Alpine Linux is reducing dependencies on Busybox
What are some alternatives?
openrc - The OpenRC init system
dinit - Service monitoring / "init" system
tini - A tiny but valid `init` for containers
init - KISS Linux - Init Framework
inotify-tools - inotify-tools is a C library and a set of command-line programs providing a simple interface to inotify.
earlyoom - earlyoom - Early OOM Daemon for Linux
jsonnet - Jsonnet - The data templating language
supervisor - Supervisor process control system for Unix (supervisord)
hummingbird - Hummingbird init system for Linux based operating systems.
dracut - dracut the event driven initramfs infrastructure
systemE - 🤣 A lightweight systemd replacement written in Emacs lisp 🤣