sigstore-python VS pip

You could use `pip-compile` if you want full pinning. That's what we do on another project -- we use GitHub Actions with `pip-compile` to provide a fully frozen copy of the dependency tree for users who'd like that[1].

In the context of `pip-audit`, that makes a little less sense: most of our dependencies are semantically versioned, and we'd rather users receive patches and fixes to our subdependencies automatically, rather than having to wait for us to release a corresponding fix version. Similarly, we expect users to install `pip-audit` into pre-existing virtual environments, meaning that excessive pinning will produce overly conservative dependency conflict errors.

[1]: https://github.com/sigstore/sigstore-python/tree/main/instal...

The conflicting advice is a serious problem.

I hope you'll forgive me for adding one additional piece of advice: for many Python packages, the only packaging metadata you need is `pyproject.toml`. You don't even need `setup.py` anymore, so long as you're using a build backend that supports editable installs with `pyproject.toml`.

Here's an example of a Python package that does everything in `pyproject.toml`[1]. You should be able to copy that into any of your projects, edit it to match your metadata, and everything will work exactly as if you have a `setup.cfg` or `setup.py`.

[1]: https://github.com/sigstore/sigstore-python

You're right, both the infrastructure and metadata for cryptographic signatures on Python packages (both wheels and sdists) isn't quite there yet.

At the moment, we're working towards the "e2e" scheme you've described by adding support for Sigstore[1] certificates and signatures, which will allow any number of identities (including email addresses and individual GitHub release workflows) to sign for packages. The integrity/availability of those signing artifacts will in turn be enforced through TUF, like you mentioned.

You can follow some of the related Sigstore-in-Python work here[2], and the ongoing Warehouse (PyPI) TUF work here[3]. We're also working on adding OpenID Connect token consumption[4] to Warehouse itself, meaning that you'll be able to bootstrap from a trusted GitHub workflow to a PyPI release token without needing to share any secrets.

[1]: https://www.sigstore.dev/

[2]: https://github.com/sigstore/sigstore-python

[3]: https://github.com/pypa/warehouse/pull/10870

[4]: https://github.com/pypa/warehouse/pull/11272

Whenever you are working on a Python project that has external dependencies installed with pip, it is strongly recommended to first create a virtual environment.

Unfortunately that feature is easy to break: https://github.com/pypa/pip/issues/9644

I think I did my part https://github.com/pypa/pip/issues/6409

$ sudo pip install y$ sudo pip install yaml WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip. Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue. To avoid this problem you can invoke Python with '-m pip' instead of running pip directly. ERROR: Could not find a version that satisfies the requirement yaml (from versions: none) ERROR: No matching distribution found for yaml

What are you implying will happen?

Using the build-in tools, you can save the exact versions of dependencies (i.e. a lock file) using "pip freeze >dependencies.txt". This should give you the exact same set of packages in two years' time.

If you want to be even more sure, you can also store hashes in the lock file. This has to be generated by a separate tools at the moment [1][2] but can be consumed by the built-in tools [3], so "pip install -r requirements.txt" is still all you need in two years' time.

[1] https://github.com/pypa/pip/issues/4732

[2] https://pip-tools.readthedocs.io/en/latest/#using-hashes

[3] https://pip.pypa.io/en/stable/topics/secure-installs/#hash-c...

Here’s the issue, https://github.com/pypa/pip/issues/11664. I think the idea would be to have some file/json description of environment that could be passed to pip to allow it to fully cross compile. They are open to supporting it just needs contributor to be found to implement it and go through review/discussion.

To be fair the only alternative is fixing Python, and even then you still would have to wait a good 5 years at least for all the old Python versions to dwindle.

It doesn't look like the fixing effort is progressing very quickly: https://github.com/pypa/pip/issues/8606

To their credit, at least they didn't close it "works as intended" which I imagine a lot of projects would.

Another good benchmark to trying to resolve apache-airflow[all]==1.10.13 using the state of PyPi on 2020-12-02, I give instructions here on how to reproduce that workflow: https://github.com/pypa/pip/issues/11836. Including a benchmark how how many extra packages your resolver should visit.