pip VS devpi

Whenever you are working on a Python project that has external dependencies installed with pip, it is strongly recommended to first create a virtual environment.

Unfortunately that feature is easy to break: https://github.com/pypa/pip/issues/9644

I think I did my part https://github.com/pypa/pip/issues/6409

$ sudo pip install y$ sudo pip install yaml WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip. Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue. To avoid this problem you can invoke Python with '-m pip' instead of running pip directly. ERROR: Could not find a version that satisfies the requirement yaml (from versions: none) ERROR: No matching distribution found for yaml

What are you implying will happen?

Using the build-in tools, you can save the exact versions of dependencies (i.e. a lock file) using "pip freeze >dependencies.txt". This should give you the exact same set of packages in two years' time.

If you want to be even more sure, you can also store hashes in the lock file. This has to be generated by a separate tools at the moment [1][2] but can be consumed by the built-in tools [3], so "pip install -r requirements.txt" is still all you need in two years' time.

[1] https://github.com/pypa/pip/issues/4732

[2] https://pip-tools.readthedocs.io/en/latest/#using-hashes

[3] https://pip.pypa.io/en/stable/topics/secure-installs/#hash-c...

Here’s the issue, https://github.com/pypa/pip/issues/11664. I think the idea would be to have some file/json description of environment that could be passed to pip to allow it to fully cross compile. They are open to supporting it just needs contributor to be found to implement it and go through review/discussion.

To be fair the only alternative is fixing Python, and even then you still would have to wait a good 5 years at least for all the old Python versions to dwindle.

It doesn't look like the fixing effort is progressing very quickly: https://github.com/pypa/pip/issues/8606

To their credit, at least they didn't close it "works as intended" which I imagine a lot of projects would.

Another good benchmark to trying to resolve apache-airflow[all]==1.10.13 using the state of PyPi on 2020-12-02, I give instructions here on how to reproduce that workflow: https://github.com/pypa/pip/issues/11836. Including a benchmark how how many extra packages your resolver should visit.

There are cases where you want the flexibility of installing a python package via pip without having it available to the open public. This article will focus on using devpi to provide a self-hosted pip compatible python package server. Ubuntu will be used for the OS as it's a fairly common Linux distribution and easily available on Windows Linux Subsystem.

When not providing all dependencies yourself, you might suffer from people deleting the packages you depend on (IMHO a very rare scenario). If it is really that critical (hint: usually it isn't), create a local mirror of Pypi (full or only the packages you need). Devpi, Artifactory, etc. can do that or you just dump the necessary files into Cloud storage, so you have a backup.

devpi acts as a caching proxy for PyPI and takes a bit less setup than this. Plus, you can use it for storing your own packages in a separate index.

https://github.com/devpi/devpi

There are several open-source solutions for provisioning your own, personal PyPI server — for example, this can be done using pypiserver or devpi. However, configuring these services takes time and effort, and it costs money to deploy them. Instead, we’ll use Packagr, a cloud-hosted python package server that allows you to provision your own private Python package repository. It also supports NPM packages and even Docker registries and is very easy to set up. You can get started by creating a free trial account on Packagr — when you’ve created your account, you’ll see this:

devpi-server will probably for the bill

> One should probably run their own package server like https://github.com/pypiserver/pypiserver

Never used pypiserver but I’ve had a good experience with https://github.com/devpi/devpi