sigstore-python VS globus-group-manager

You could use `pip-compile` if you want full pinning. That's what we do on another project -- we use GitHub Actions with `pip-compile` to provide a fully frozen copy of the dependency tree for users who'd like that[1].

In the context of `pip-audit`, that makes a little less sense: most of our dependencies are semantically versioned, and we'd rather users receive patches and fixes to our subdependencies automatically, rather than having to wait for us to release a corresponding fix version. Similarly, we expect users to install `pip-audit` into pre-existing virtual environments, meaning that excessive pinning will produce overly conservative dependency conflict errors.

[1]: https://github.com/sigstore/sigstore-python/tree/main/instal...

The conflicting advice is a serious problem.

I hope you'll forgive me for adding one additional piece of advice: for many Python packages, the only packaging metadata you need is `pyproject.toml`. You don't even need `setup.py` anymore, so long as you're using a build backend that supports editable installs with `pyproject.toml`.

Here's an example of a Python package that does everything in `pyproject.toml`[1]. You should be able to copy that into any of your projects, edit it to match your metadata, and everything will work exactly as if you have a `setup.cfg` or `setup.py`.

[1]: https://github.com/sigstore/sigstore-python

You're right, both the infrastructure and metadata for cryptographic signatures on Python packages (both wheels and sdists) isn't quite there yet.

At the moment, we're working towards the "e2e" scheme you've described by adding support for Sigstore[1] certificates and signatures, which will allow any number of identities (including email addresses and individual GitHub release workflows) to sign for packages. The integrity/availability of those signing artifacts will in turn be enforced through TUF, like you mentioned.

You can follow some of the related Sigstore-in-Python work here[2], and the ongoing Warehouse (PyPI) TUF work here[3]. We're also working on adding OpenID Connect token consumption[4] to Warehouse itself, meaning that you'll be able to bootstrap from a trusted GitHub workflow to a PyPI release token without needing to share any secrets.

[1]: https://www.sigstore.dev/

[2]: https://github.com/sigstore/sigstore-python

[3]: https://github.com/pypa/warehouse/pull/10870

[4]: https://github.com/pypa/warehouse/pull/11272

It has only recently sunk in for me that, with the latest versions, Pip is not a tool for maintaining an environment and list of dependencies (where you'd have a lockfile, for example). Instead, it is an interface to…

1. Download packages from PyPi (or a different repository that provides the same interface)

2. Read the pyproject.toml file to find the build backend to use, and then install that.

3. Call the build backend to actually do the installation. This can be Poetry, Setuptools, Flit, or something else.

Pipenv is another such interface.

PEP 517 (https://peps.python.org/pep-0517/) created pyproject.toml and defined the API that build backends follow. That API includes a way for the build backend to tell Pip (or Pipenv, etc.) what dependencies to install. For systems that have a lockfile (like Pipenv), that could be a list of packages with explicit versions.

PIP 660 extended PEP 517, defining a standard way to have "editable installs". In other words, a way to support `pip install -e package`, or even `pip install -e .`.

The above is all my understanding, which is not at all authoritative!

It's worth checking out the list of packaging-related PEPs: https://peps.python.org/topic/packaging/ It's worth reading PEPs 518 and 621.

As an example, here's a Python package that uses Poetry: https://github.com/globus/globus-timer-cli It works perfectly fine with Pip, because Pip sees (via `pyproject.toml`) that it needs to pull in Poetry, and call it to do the actual install.

Here's an older repo of mine, where I've just started to learn about the transition: https://github.com/stanford-rc/mais-apis-python/blob/main/py... In this case, I could delete `pyproject.toml` entirely, Pip would see the `setup.py` file, and understand to use Setuptools. This was me when I was just starting to learn about the transition.

Finally, here's a newer repo of mine, where I've ditched setup.py (and _almost_ ditched setup.cfg) entirely: https://github.com/stanford-rc/globus-group-manager I'm still using Setuptools, but all of the metadata and requirements are included in the `pyproject.toml` file.

It's definitely been a rocky transition, but it's really looking (to me, at least) like we're at (or near) the point where I can just use `pip install …` and it'll work, regardless of the build backend in use!

I very much appreciate Dependabot! I like how it can pick up dependencies in interesting places.

For example, the Globus @ Stanford web site (https://globus.stanford.edu) uses GitHub Pages (repo at https://github.com/stanford-rc/globus.stanford.edu). I have a Gemfile in the repo: When I want to test changes locally, I use Bundler to install everything I need, and to launch Jekyll. Even though the Gemfile isn't used 'in production', Dependabot still warns me, so that I don't run older, vulnerable software on my laptop.

At the same time, I can't be sure if Dependabot is picking up dependencies for my Python project.

In my latest project (https://github.com/stanford-rc/globus-group-manager), I'm using pyproject.toml to hold all of the Python dependencies for the project, something that Setuptools is now supporting experimentally (woot!). I've configured Dependabot, and it has picked up my repo's `pyproject.toml` file, but I can't tell if it has actually cataloged my Python dependencies.

Looking around the web also does not give me a clear answer. For example, https://github.com/dependabot/feedback/issues/57 is titled "pyproject.toml support", but it refers specifically to Poetry (and indeed, Poetry v1 is listed as supported at https://docs.github.com/en/code-security/dependabot/dependab...). But Setuptools is not.

https://github.community/t//2576 asks about Setuptools support, and has been pretty dormant. I thought setup.cfg was supported after https://github.com/dependabot/dependabot-core/pull/3423, but another project of mine (https://github.com/stanford-rc/mais-apis-python/network/depe...) doesn't show anything for setup.cfg.