Our great sponsors
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
I thought very hard about this problem as I've developed Virgil [https://github.com/titzer/virgil] over the years. Bootstrapping any system, not the least of which a new programming language is a hard problem.
Virgil has an (almost) hermetic build. The compiler binaries for a stable version are checked into the repository. At any given revision, that stable compiler can compile the source code in the repo to produce a new compiler binary, for any of the supported platforms. That stable binary is therefore a cross-compiler. There are stable binaries for each of the supported stable platforms (x86-darwin, x86-linux, JVM), and there are more platforms that are in the works (x86-64-linux, wasm), but don't have stable binaries.
What do you need to run one of the stable binaries?
JVM: any Java 5 compliant JVM
> I feel like this really only makes sense if pyca/cryptography had planned on adding the Rust dependency from the very beginning (or from very early on). Is there any indication that was the case?
I am sure this idea surfaced several times in IRC or possibly in the mailing lists. Certainly, the authors have been toying with handling ASN.1 in rust since 2015 [1], which I guess will be the next logical step.
I do agree that this is mostly a political stance. pyca/cryptography is a wrapper sandwiched between a gigantic runtime written in C (CPython/PyPy) and a gigantic library written in C (openssl).
The addition of Rust as dependency enables the inclusion of just 90 lines of Rust [2] where the only part that really couldn't be implemented in pure Python is a line copied from OpenSSL [3] (i.e. it was already available), and which is purely algebraic, therefore not mitigating any real memory issue at all (the reason to use rust in the first place).
The change in this wrapper (pyca/cryptographic) does not move the needle of security in any significant way, and it is really only meant to send the signal that adding Rust in all other Python packages and especially in the runtime itself will now come at no (political) cost.
[1] https://github.com/alex/rust-asn1
[2] https://github.com/pyca/cryptography/blob/main/src/rust/src/...
[3] https://github.com/openssl/openssl/blob/OpenSSL_1_1_1i/inclu...
Alright, let's do some digging...
On 2013-03-21, urllib3 added an optional dependency to pyopenssl for SNI support on python2 - https://github.com/urllib3/urllib3/pull/156
On 2013-12-29, pyopenssl switched from opentls to cryptography - https://github.com/pyca/pyopenssl/commit/6037d073
On 2016-07-19, urllib3 started to depend on a new pyopenssl version that requires cryptography - https://github.com/urllib3/urllib3/commit/c5f393ae3
On 2016-11-15, requests started to depend on a new urllib3 version that now indirectly requires cryptography - https://github.com/psf/requests/commit/99fa7bec
On 2018-01-30, portage started to enable the +rsync-verify USE flag by default, which relies on the gemato python library maintained by mgorny himself, and gemato depended on requests. So 5-6 levels of indirection at this point? I lost count.
On 2020-01-01, python2 was sunset. A painful year to remember, and a painful migration to forget. And just when the year was about to end...
On 2020-12-22, cryptography started to integrate rust in the build process, and all hell broke loose - https://github.com/pyca/cryptography/commit/c84d6ee0
Ultimately, I think mgorny only has himself to blame here, by injecting his own library into the critical path of gentoo, without carefully taking care of its direct and indirect dependencies. (But of course it is also fair game to blame it on the 2to3 migration)
In comparison, few months before this, the librsvg package went through a similar change where it started to depend on rust, and it was swift and painless without much drama - https://bugs.gentoo.org/739820 and https://wiki.gentoo.org/wiki/Project:GNOME/3.36-notes