Bundling binary tools in Python wheels

1. cibuildwheel

   1 11 2,238 9.4 Python

   🎡 Build Python wheels for all the platforms with minimal configuration.

   

   > size of wheels you can upload is constrained by PyPi

   I feel PyPi is pretty generous with their limits. You can even request more once you hit the ceiling, i think it’s around 60MB [1]. There are some wheels that are crazy large, tensorflow-gpu [2] are around 500MB each. I think there’s discussions out there to try and find ways of alleviating this problem on PyPi.

   > difficult to support multiple versions across multiple operating systems, unless you provide a source distribution, which is then…

   This can be a problem but I’ve found that recently the problem has improved quite a lot. You can create manylinux wheels for both x86, x64, and arm64 which cover pretty a lot of the Linux distributions using glibc. A musllinux tag was recently added to cover musl based distributions like Alpine. MacOS wheels support both x64, arm64, and can even be a universal2 wheel. Windows is still purely x86 or x64 for now but I’ve seen some people work on arm64 support support in CPython and once that’s in I’m sure PyPi won’t be too far around. There are also some great tools like cibuildwheel [3] that make building and testing these wheels pretty simple.

   > Still a nightmare on Windows

   I’m actually curious what is a nightmare about Windows. I found that Windows is probably the easiest of all the platforms to build and upload wheels for. You aren’t limited to a tiny subset of system libs, like you are on Linux, and building them is mostly the same process. Probably the hardest thing is ensuring you have tue correct vs build kit installed but that’s not insurmountable.

   [1] https://pypi.org/help/#file-size-limit

   [2] https://pypi.org/project/tensorflow-gpu/#files

   [3] https://github.com/pypa/cibuildwheel

2. SaaSHub

   www.saashub.com featured

   SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

   

3. postgresql-wheel

   2 4 81 0.0 Python

   A Python wheel containing PostgreSQL

   

   This is also the basis for the postgresql-wheel package, a Python wheel that contains an entire local PostgreSQL binary installation:

   https://github.com/michelp/postgresql-wheel

   Out of sheer laziness I used CFFI to encapsulate the build, even though there is no dynamic linking involved. There's probably a simpler way but this approach has worked very well for disposable database tests.

4. sigstore-python

   3 4 319 9.4 Python

   A Sigstore client written in Python

   

   You're right, both the infrastructure and metadata for cryptographic signatures on Python packages (both wheels and sdists) isn't quite there yet.

   At the moment, we're working towards the "e2e" scheme you've described by adding support for Sigstore[1] certificates and signatures, which will allow any number of identities (including email addresses and individual GitHub release workflows) to sign for packages. The integrity/availability of those signing artifacts will in turn be enforced through TUF, like you mentioned.

   You can follow some of the related Sigstore-in-Python work here[2], and the ongoing Warehouse (PyPI) TUF work here[3]. We're also working on adding OpenID Connect token consumption[4] to Warehouse itself, meaning that you'll be able to bootstrap from a trusted GitHub workflow to a PyPI release token without needing to share any secrets.

   [1]: https://www.sigstore.dev/

   [2]: https://github.com/sigstore/sigstore-python

   [3]: https://github.com/pypa/warehouse/pull/10870

   [4]: https://github.com/pypa/warehouse/pull/11272

5. warehouse

   4 320 4,057 9.7 Python

   The Python Package Index

   

   You're right, both the infrastructure and metadata for cryptographic signatures on Python packages (both wheels and sdists) isn't quite there yet.

   At the moment, we're working towards the "e2e" scheme you've described by adding support for Sigstore[1] certificates and signatures, which will allow any number of identities (including email addresses and individual GitHub release workflows) to sign for packages. The integrity/availability of those signing artifacts will in turn be enforced through TUF, like you mentioned.

   You can follow some of the related Sigstore-in-Python work here[2], and the ongoing Warehouse (PyPI) TUF work here[3]. We're also working on adding OpenID Connect token consumption[4] to Warehouse itself, meaning that you'll be able to bootstrap from a trusted GitHub workflow to a PyPI release token without needing to share any secrets.

   [1]: https://www.sigstore.dev/

   [2]: https://github.com/sigstore/sigstore-python

   [3]: https://github.com/pypa/warehouse/pull/10870

   [4]: https://github.com/pypa/warehouse/pull/11272

6. auditwheel

   5 4 525 8.4 Python

   Auditing and relabeling cross-distribution Linux wheels.

   

   In a similar vein: Python projects can use `auditwheel` to automatically relocate (fixup RPATHs) and vendor their "system" dependencies, such as a specific version of `zlib` or `libffi`[1].

   [1]: https://github.com/pypa/auditwheel

Suggest a related project