glibc_version_header VS steam-runtime

One major headache with trying to run precompiled binaries on Linux is that if they were compiled using a newer version of glibc than the target machine, they won't be able to run. Back while working on Factorio, I was trying to get around this problem with endless Docker containers, but coworker Wheybags came up with a much solution to this, which is simply to, at compile time, link to the oldest compatible version of glibc: https://github.com/wheybags/glibc_version_header

If what you're doing works for you, great, but in case it stops working at some point (or if for some reason you need to build on a current-gen distro version), you could also consider using this:

https://github.com/wheybags/glibc_version_header

It's a set of autogenerated headers that use symbol aliasing to allow you to build against your current version of glibc, but link to the proper older versioned symbols such that it will run on whatever oldest version of glibc you select.

There are other approaches like https://github.com/wheybags/glibc_version_header or sysroots with older glibc, e.g. https://wiki.gentoo.org/wiki/Crossdev - you don't need your whole XP, just the the system libs to link against.

Sure, having a nice SDK where you can just specify the minimum vesion you want to support would be nice but who do you expect to develop such an SDK? GNU/glibc maintainers? They would rather you ship as source. Red Hat / SUSE / Canonical? They want you to target only their distro. Valve? They decided its easier to just provide an unchaning set of libraries since they need to support existing games that got things wrong anyway and already have a distribution platform to distribute such a base system along with the games without bundling it into every single one.

A few links gathered from a quick google search as a primer:

http://stevehanov.ca/blog/?id=97

https://www.evanjones.ca/portable-linux-binaries.html

https://insanecoding.blogspot.com/2012/07/creating-portable-...

https://rpg.hamsterrepublic.com/ohrrpgce/Portable_GNU-Linux_...

https://github.com/wheybags/glibc_version_header

In other words: there are a lot of steps and a lot of gotchyas to doing this that you're glossing over. Linux userland libraries are generally designed with the intention that an army of third-party maintainers will integrate all of this desperately developed software together and place it in a repo. Naturally every distribution wants to do things a little differently too, and they have a habit of changing it up every couple years. When you try to step out of that mold things unsurprisingly become more difficult. Whereas Windows, Mac, Android, etc. have been designed since the beginning not to require that sort of thing and it is consequently a much, much more straightforward process.

I'm curious why, since you seem to believe the process is so straight-forward, you think it is that so few people distribute a simple binary? Why were Flatpak and AppImage invented?

> Binaries compiled against today's glibc can fail to run on a machine that hasn't been updated since last week because they rely on a new / different symbol.

Note, however, that it is a Glibc bug (modulo Drepper’s temper) if the reverse happens: Glibc symbol versioning ensures that binaries depending on an old Glibc (only) will run on a new one. So the proper way to build a maximally-compatible Linux executable would be to build a cross toolchain targeting an old Glibc and compile your code with it. Unfortunately, the build system is hell and old Glibcs doesn’t compile without backported patches, so while I did try to follow in the footsteps of a couple of people[1–4], I did not succeed.

Mass-rebuilds still happen with other ecosystems, though. GHC-compiled Haskell libraries are fine-grained and not ABI-stable across compiler versions, so my Arch box regularly gets hit with a deluge of teensy library updates, and Arch is currently undergoing a massive Python rebuild (blocking all other Python package updates) behind the scenes as well.

[1]: https://github.com/wheybags/glibc_version_header (hack but easy and will probably work most of the time)

> It turns out that unless the game is explicitly marked (by Valve reviewers), Steam Deck will use the Windows build + Proton even if a Linux version is available.

I found this which sounds like it's not the default, but is in fact a result of compatibility testing:

> If your game has gone through Steam Deck compatibility testing and the testers reported that the native Linux version didn't work (because of #579), then it might have been flagged to run the Windows binaries via Proton by default, instead of the native Linux version.

per https://github.com/ValveSoftware/steam-runtime/issues/585

> Where is it written that steam-run will magically execute most binaries without patching them?

Somewhere in here: https://github.com/ValveSoftware/steam-runtime

:p

But I do get what you're saying. Once Flakes are default, I hope people start a proper push to clear up documentation and streamline the development process. The end-result is amazing, and the perfect OS/packaging system for my needs. The means of getting there... need a lot of work. I'm along for the ride either way.

I think they have something a bit like a container built into Steam: https://github.com/ValveSoftware/steam-runtime

this is for steamworks API, my understanding is there's a separate SDK for consuming Linux dependencies like glibc. Like Soldier runtime, Sniper runtime, and so on. Am I wrong in thinking these are two separate SDKs? here's the link to the other SDK I'm talking about: https://github.com/ValveSoftware/steam-runtime

I'm not sure we can distribute a flatpak or an appimage through Steam. They have their own controlled environment called Steam Runtime (https://github.com/ValveSoftware/steam-runtime) in which I should compile to be sure it runs everywhere (very similar to what I am doing). Last time, I look at this, it wasn't very clear and they supported only old versions of GCC. But it seems the documentation improved and now that I succeeded in building a modern version of GCC in my own container, maybe I could do that in theirs.

I believe it's usable outside of Steam: https://github.com/ValveSoftware/steam-runtime though the instructions are not particularly clear. There's also a link to the APT repo they use as a reference: https://repo.steampowered.com/steamrt/

Looking at the Flatpak version, if you want to use Proton versions 5.13 or newer with Steam in Flatpak, you need to install Flatpak from backports https://github.com/ValveSoftware/steam-runtime/issues/294 . Using Flatpak saves having to install i386 if that matters to you.

> Game developers would be fine to target a single distro like Ubuntu 22.04.

Valve has its own container-only Linux distribution, called "Soldier Runtime" (https://github.com/ValveSoftware/steam-runtime); especially for games distributed on Steam, it probably makes more sense to target that distribution instead of Ubuntu.