omnios-build VS manylinux

key features at OmniOS Community Edition

OS Challenge: Try out an Illumos (OpenSolaris) image in a VM (there are Solaris kernel options available in Proxmox). If you want something fun, install OmniOS and create a Linux zone on it. So cool to run Linux on Solaris. https://omnios.org/

While UTM is much better at resource usage, it's not perfect. Virtualbox seems to emulate more of a system instead of passing it through to the hypervisor and so is better for running more unusual OSes. I have two VMs there running Illumos which I have yet to figure out how to boot in UTM. A problem that I think is related to UTM's greater use of the hypervisor is that you can't suspend and resume VMs that use it instead of being emulated, so I've still got a few VMs hanging around in Virtualbox which spend most of their time suspended. Finally, what stops me from using UTM at work is that you can't use it as a Vagrant provider. This is incredibly annoying, as the lack of a decent virtualization application makes the otherwise very nice M1 Macs nothing more than pretty toys. I expect that this glaring lack will be fixed within the next couple of years.

Can anyone recommend a good place to start learning OmniOS? Coming from Debain world. Other than the documentation on omnios.org, I only found a few blog posts here and there...

OpenIndiana is pretty neat. It has quite a few cool features like Zones, DTrace and Crossbow. ZFS is another big feature but you have already said that you don't really care about that. Hardware support is kinda lacking, but improving. Also it's rolling release so if you're like me and don't like that, OmniOS might be a better option; it's another illumos based OS.

I use manylinux containers as the OS for compilation. It tries to ensure as much cross-os / libc / etc.. as much as possible for precompiled libraries. https://github.com/pypa/manylinux

Just to clarify for anyone who isn't aware, the "compiling issues", at least historically, have been that that Alpine uses musl, and PyPI's manylinux wheels are built against old glibc versions. So stuff like numpy that would trivially and quickly install from whl on glibc distros (like a bare-bones Ubuntu image) trigger compilations and the installation of build-only dependencies on Alpine.

That said, it looks like as of late-2021, at least some projects are offering musllinux wheels as well, per the discussion here: https://github.com/pypa/manylinux/issues/37 (not numpy, though: https://pypi.org/project/numpy/#files)

It's very hard. Incompatible glibc ABIs make this nigh impossible, there's a reason Steam installs a vcredistributable.dll for pretty much every game on Windows.

Look no further than the hoops you need jump through to distribute a Linux binary on PyPI [1]. Despite tons of engineering effort, and tons of hoop jumping from packagers, getting a non-trivial binary to run across all distros is still considered functionally impossible.

[1]: https://github.com/pypa/manylinux

As a user, if you build every python package from source, it's ok. But if you a maintainer of an OSS project and you need to publish binary packages for it, then you will hit the trouble. Binaries built on Ubuntu 20.04 can only support Ubuntu 20.04 and newer. So you'd better to choose an older Linux release to target broader users. Now most python packages choose CentOS 6 or 7. See https://github.com/pypa/manylinux/issues/1012 for more details. They need help!

I recently learned that Clang supports this kind of cross-compiling out of the box. https://mcilloni.ovh/2021/02/09/cxx-cross-clang/

The main difference is that Clang does not ship with headers/libraries for different platforms, as Zig appears to do. You need to give Clang a "sysroot" -- a path that has the headers/libraries for the platform you want to compile for.

If you create a bunch of sysroots for various architectures, you can do some pretty "easy" cross-compiling with just a single compiler binary. Docker can be a nice way of packaging up these sysroots (especially combined with Docker images like manylinux: https://github.com/pypa/manylinux). Gone are the days when you had to build a separate GCC cross-compiler for each platform you want to target.

Now you come and use manylinux to build. (https://github.com/pypa/manylinux) so you are based on the CentOS 7 toolchain (at best if you use manylinux2014) or Debian 9 toolchain (if you use manylinux_2_24).

I think the generally accepted way to do that would be a container image running a relatively old distribution. This is exactly what python packages do when they need to distribute binary packages on linux [0]. You are supposed to compile the package in a container (or VM) that runs CentOS 7 (or older if you want broader support), although now the baseline is moving gradually to Debian 9.

[0]: https://github.com/pypa/manylinux