geospatial-data-lake VS devpi

One option is to trust on first use, checksum the installation script and at least casually verify the diff each time the checksum changes[1].

Pros:

- Protects against simple hijacking.

- Reproducible as long as the installer doesn't also call out to a moving target, such as example.com/releases/latest.

Cons:

- Build breaks as soon as the installer is bumped. If it's bumped often (or just before an important release) this can cause pain.

- TOFU may not be acceptable, but of course you could review the code thoroughly before even the first use.

[1] https://github.com/linz/geostore/blob/b3cd162605109da8a3a688...

I'd recommend a project from work, Geostore[1]. Highlights:

- 100% test coverage (with some typical exceptions like `if __name__ == "__main__":` blocks)

- Randomises test sequence and inputs reproducibly

- Passes Pylint with max McCabe complexity of 6

- Passes `mypy --strict`

- Formatted using Black and isort

[1] https://github.com/linz/geostore

The current work project[1] has all of these: Pyenv, Poetry, Pytest, pytest-cov with 100% branch coverage, pre-commit, Pylint rather than Flake8, Black, mypy (with a stricter configuration than recommended here), and finally isort. These are all super helpful.

There's also a simpler template repo[2] with almost all of these.

[1] https://github.com/linz/geostore/

[2] https://github.com/linz/template-python-hello-world

As someone who's used CDK for a few months and never handcoded CF, that sounds completely correct. If you're comfortable with Python, here's a simple but non-trivial architecture you can check out: https://github.com/linz/geospatial-data-lake/blob/master/app....

There are cases where you want the flexibility of installing a python package via pip without having it available to the open public. This article will focus on using devpi to provide a self-hosted pip compatible python package server. Ubuntu will be used for the OS as it's a fairly common Linux distribution and easily available on Windows Linux Subsystem.

When not providing all dependencies yourself, you might suffer from people deleting the packages you depend on (IMHO a very rare scenario). If it is really that critical (hint: usually it isn't), create a local mirror of Pypi (full or only the packages you need). Devpi, Artifactory, etc. can do that or you just dump the necessary files into Cloud storage, so you have a backup.

devpi acts as a caching proxy for PyPI and takes a bit less setup than this. Plus, you can use it for storing your own packages in a separate index.

https://github.com/devpi/devpi

There are several open-source solutions for provisioning your own, personal PyPI server — for example, this can be done using pypiserver or devpi. However, configuring these services takes time and effort, and it costs money to deploy them. Instead, we’ll use Packagr, a cloud-hosted python package server that allows you to provision your own private Python package repository. It also supports NPM packages and even Docker registries and is very easy to set up. You can get started by creating a free trial account on Packagr — when you’ve created your account, you’ll see this:

devpi-server will probably for the bill

> One should probably run their own package server like https://github.com/pypiserver/pypiserver

Never used pypiserver but I’ve had a good experience with https://github.com/devpi/devpi