InitWare VS systemd

Fear not. They get to be part of the future too.

> Yes indeed, I should've expanded to requiring user namespaces and other kernel magic I can't expect from any random box i wanna work on.

That's fair, do have to make sure to avoid to modules that do user systemd services.

Longer term, though, I am hoping https://github.com/InitWare/InitWare will help with the userland part. And I hope to personally help with things like

https://lists.freebsd.org/archives/freebsd-arch/2022-January...

https://lore.kernel.org/lkml/f8457e20-c3cc-6e56-96a4-3090d7d...

to get us more sane cross-platform system calls.

I know, but support is still in FreeBSD. My big long term plan is:

1. Work on FreeBSD cross in Nixpkgs, because I need a way to pin forks and run nice tests without going insane. (We already have NetBSD cross.)

2. Rig up a booting image that uses https://github.com/InitWare/InitWare, the fork of systemd.

3. Add support to CloudABI in initware.

4. Bang on drum for other OSes and upstream systemd to implement this stuff we can can good portable abstractions -- I think this is our best shot to get "portable containers".

I haven't bothered to have a beef with systemd, but some of us have discussed https://github.com/InitWare/InitWare to support non-Linux kernels. That would be really fun.

I'm the first to admit that I'm ignorant of the facts here, but seeing that a systemd fork ran on OpenBSD for the first time two months ago does not give me confidence that it's "an option" in the sense that you can trust it to work well.

And to be pedantic (this is an OpenBSD thread, after all), it's not "systemd", it's a fork of systemd called "InitWare", and the GitHub repo describes it as "alpha software".

Someone also pointed out in the discussion you linked that it doesn't seem to include journald. Here's a relevant PR: https://github.com/InitWare/InitWare/pull/27

The project GitHub is found at https://github.com/InitWare/InitWare

You may have heard of InitWare https://github.com/InitWare/InitWare. Discussions, note that one of the titles is misleading:

As per the article mentions of resolving AAAA IPv6 records, a feature was just merged into Linux's systemd-resolved last week.

https://github.com/systemd/systemd/pull/34165

So you can do.

RefuseRecordTypes=AAAA

So there, your DNS won't ever query for IPv6 AAAA records.

(in fact you can refuse any DNS questions like MX, SRV, TXT as well with:

RefuseRecordTypes=MX SRV TXT)

Note - I started adding that feature because some applications were "misbehaving" and querying AAAA records when IPv6 stack was disabled.

On Linux, a privileged process can change the memory address which the kernel (/proc filesystem) reads argv/etc from... prctl(PR_SET_MM) with the PR_SET_MM_ARG_START/PR_SET_MM_ARG_END arguments. Likewise, with PR_SET_MM_ENV_START/PR_SET_MM_ENV_END.

The API is ugly, and since it needs CAP_SYS_RESOURCE many programs can't use it... but systemd does: https://github.com/systemd/systemd/blob/2635b5dc4a96157c2575...

This shouldn't cause the kind of race conditions we are talking about here, since it isn't changing a single arg, it is changing the whole argv all at once. However, the fact that PR_SET_MM_ARG_START/PR_SET_MM_ARG_END are two separate prctl syscalls potentially introduces a different race condition. If Linux would only provide a prctl to set both at once, that would fix that. The reason it was done this way, is the API was originally designed for checkpoint-restore, in which case the process will be effectively suspended while these calls are made.

The article doesn't even mention this: systemd is not well written software, and far from behaving robust and consistently. Take some time, look through the currently open issues on Github, see for yourself whether you consider this stable software. Machine hanging indefinitely during poweroff/reboot is one of the classics.

[1] https://github.com/systemd/systemd/issues

Idk, when systemd became the main thing I hated it too. But mostly because it was different and I didn't know how to use it. But then I learned and you know what? I agreed. This took awhile though and I had to see the problems they are solving. Otherwise it looks really bloaty and confusing. Like why have things like nspawn? Why use systemd jobs instead of using cron? Why use systemd-homed instead of useradd?

Well a big part of it is security and flexibility.

I write systemd services now instead of cron jobs. With a cron job I can't make private tmps[0]. Cron won't run a job if the computer is off during the job time. Cron can't stagger services. Cron can't wait for other services first. Cron can't be given limited CPU, memory, or other resource limitations.

Nspawn exists to make things highly portable. Chroot on steroids is used for a reason. Being able to containerize things, placing specific capabilities and all that. This is all part of a systemd job anyways. It really makes it a lot easier to give a job the minimum privileges. So often nspawn is a better fit than things like docker.

Same goes for homed. You can do things like setting timezones unique to users. But there's so much more like how it can better handle encryption. And you can do really cool things like move your home directory from one machine to another. This might seem like a non-issue to most people but it isn't. That whole paradigm where your keyboard is just an interface to a machine (i.e. a terminal, and I don't mean the cli. There's a reason that's called a terminal "emulator"). This is a really useful thing when you work on servers.

Look, there's a reason the distros switched over. It's not collective madness.

[0] https://www.redhat.com/en/blog/new-red-hat-enterprise-linux-...

https://systemd.io/

We all know that systemd and dbus are beast of their own, but what I didn't know was that just a simple change on our own service files would lead me to hours of exploration, some documentations that are not properly documentated (don't judge, we all have done that.), and to my surprise, the answer again was very simple.

https://github.com/systemd/systemd/issues/6129#issuecomment-...

"core: add RestartSteps= and RestartSecMax= to dynamically increase interval between auto restarts"

I ran into this issue on an embedded system before it was raised / fixed:

https://github.com/systemd/systemd/issues/8398

FWIW, I'm generally a fan of systemd. That's a pretty minor issue in the grand scheme of things and it's not really "mysterious" -- the behaviour was consistent but didn't line up with the documentation.

In my experience, almost every time I hit an issue with systemd, it's when I'm interacting with it at a (relatively) low level. e.g. Monitoring / controlling units via D-Bus. There is theoretically enough documentation to do that, but it's often incomplete or ambiguous and you're left trying to figure out systemd's behaviour via experimentation.

But as I said, this isn't _really_ a dig against systemd. Even if monitoring service state transitions via D-Bus is finnicky, it's a heck of a lot easier than the alternatives with busybox init or whatever.

> Primarily append-based, hence robust to corruption

It's so robust, it doesn't even let you modify the journal if you want to (e.g. https://github.com/systemd/systemd/issues/20673).

> Support for in-line compression

Mind that journald only supports compressing single lines, but not the whole journal (https://github.com/systemd/systemd/issues/31358), which is pretty limiting.

https://github.com/systemd/systemd/pull/33383

Or that time that systemd mounted efivarfs as rw making it possible to brick your motherboard

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

Or the time back around 2012ish when an update caused the dhcp lease renewal portion to get wedged and took many thousands of servers offline until the service could be manually restarted or the machine rebooted.

Or there was that time when OpenSSH was compromised because they pulled in systemd components to handle notifications: