pipdeptree VS pip-tools

The major drawback is that it comes with a high number of external dependencies that take time to download. Here’s a pipdeptree for ipdb dependencies:

Yes you can use `pip freeze` though you get dependencies of dependencies. pipdeptree may help to remove those.

Use dependency pinning tools like Poetry (https://github.com/python-poetry/poetry) or pip-tools (https://github.com/jazzband/pip-tools) to lock all transitive dependencies, not just top-level ones. Before upgrading any ML library, audit the release notes for breaking changes to default parameters: Scikit-Learn maintains a detailed changelog at https://scikit-learn.org/stable/whats\_new.html. For critical pipelines, add a pre-commit hook that checks for unpinned dependencies or unrecognized default parameters. We reduced our dependency-related incidents by 92% after implementing this practice.

PEP723 is also supported by pipx and hatch, even though uv is (deservedly) getting all the attention these days.

Others like pip-tools have support in the roadmap (https://github.com/jazzband/pip-tools/issues/2027)

If you’ve been managing Python projects long enough, you’ve probably dealt with a mess of tools: pip, pip-tools, poetry, virtualenv, conda, maybe even pdm.

I don’t know if this is how pipreqs works, but I’d be concerned about a typo in an import that inadvertently installs a library.

I’ve found pip-tools [1] to be a nice middle ground between `pip freeze` and something heavier like poetry. It’s especially helpful in showing you where a library came from, when a library installs other libraries it depends upon.

One could still have a typo in their requirements.in file with pip-tools, but changes there are much less frequent than imports which could be a daily occurrence in some codebases.

[1] https://github.com/jazzband/pip-tools

> Why is the "requirements.txt" file a stupid flat listing of all transitive dependencies with pinned versions? It makes it harder to change library versions even if there are no true conflicts.

My friend, here is what you seek: https://github.com/jazzband/pip-tools

requirements.txt is flat because it's really the output of `pip freeze`. It's supposed to completely and exactly rebuild the environment. Unfortunately it's far too flexible and people abuse it by putting in only direct dependencies etc.

If you're writing packages, you don't need a requirements.txt at all, by the way. Package dependencies (only direct dependencies) live in pyproject.toml with the rest of the package config. requirements.txt (and pip tools) are only for when you want to freeze the whole environment, like for a server deployment.

For all my projects I found myself regenerating manual lock files using complex shell commands with pip-compile to get a reproducible environments across devices using a custom pre-install-command. I finally decided that instead of hacking together the same solution on all my projects I would build a plugin that handles this complexity for me.

Instead of venv, we are using pip-tools in this starter kit. pip-tools take things further in dependency management. Check out what pip-tools does in their official GitHub repo. In short, it helps your project find the best match for the dependent packages. For example, you might need two packages A and B in your project that requires same package C under the hood. But A requires any version of C from 1.0.1 to 1.0.10 and B requires any version of C from 1.0.7 to 1.0.15. Pip tools will automatically compile the version of 'C' that suits for both of your packages.

I've created a small project called just-pip-tools that combines pip-tools and just to manage Python dependencies in a layered approach. This isn't a magic bullet; it's a set of files you can adapt to your needs.

For small projects I recommend pip-tools. Just write packet list in requirements.in and pip-compile compile a requirements.txt with comments.