Cooperative Package Management for Python

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured
  • PDM

    A modern Python package and dependency manager supporting the latest PEP standards

  • It's a good safeguard, and it's going in the direction of the other initiatives to make python package management default behavior saner.

    PEP 852 is the another one to follow up: https://www.python.org/dev/peps/pep-0582/

    It basically uses the concept of node_modules, making python interpreters local any local __pypackages__ directory. There are 2 differences though:

    - unlike JS, python can only have one version of one lib

    - but since having several versions of python often matters, you may have several __pypackages__/X.Y sub dirs to catter to each of them

    It does also force you to use "-m" to use commands, which is the best practice anyway. I hope it will make jupyter fix "-m" on windows for them because that's a blocker for beginners.

    If you are not already using "-m", start now. It solves a lot of different problems with running python cli programs.

    E.G: instead of running "black" or "pylint", do "python -m black" or "python -m pylint". Or course you may want to chose a specific version of python, so "python3.8 -m black" for unix, or "py -3.8 -m black" on windows.

    To test out __pypackages__, give a try to the pdm project: https://github.com/pdm-project/pdm

    At last, some other tools that I wish people knew more about that solves packaging issues:

    - pyflow (https://github.com/David-OConnor/pyflow): it's a package manager like poetry, but it also install whatever python you want like pyenv. Except it provides the binary, no need to compile anything. It's a young project, but I wish it succeeds because it's really a great concept.

    - shiv (shiv.readthedocs.io/): it leverage the concept of zipapp, meaning the ability that python has to execute python inside a zip file. It's a successor to pex. Basically it lets you bundle your code + all deps from virtualenv inside a zip, like a Java .war file. You can then run the resulting zip, a .pyz file, like if it was a regular .py file. It will unzip on the first run automatically. It makes deployment almost as easy as with golang.

    - nuitka (shiv.readthedocs.io/): take your code and all dependancies, turn them into C, and compiles it. Although it does require a bit of setup, since it needs headers and a compiler, it results reliably in a standalone compiled executable that will run on the same architecture with no need for anything else. Also it will speed up your Python program, up to 4 times.

  • pyflow

    An installation and dependency system for Python (by David-OConnor)

  • It's a good safeguard, and it's going in the direction of the other initiatives to make python package management default behavior saner.

    PEP 852 is the another one to follow up: https://www.python.org/dev/peps/pep-0582/

    It basically uses the concept of node_modules, making python interpreters local any local __pypackages__ directory. There are 2 differences though:

    - unlike JS, python can only have one version of one lib

    - but since having several versions of python often matters, you may have several __pypackages__/X.Y sub dirs to catter to each of them

    It does also force you to use "-m" to use commands, which is the best practice anyway. I hope it will make jupyter fix "-m" on windows for them because that's a blocker for beginners.

    If you are not already using "-m", start now. It solves a lot of different problems with running python cli programs.

    E.G: instead of running "black" or "pylint", do "python -m black" or "python -m pylint". Or course you may want to chose a specific version of python, so "python3.8 -m black" for unix, or "py -3.8 -m black" on windows.

    To test out __pypackages__, give a try to the pdm project: https://github.com/pdm-project/pdm

    At last, some other tools that I wish people knew more about that solves packaging issues:

    - pyflow (https://github.com/David-OConnor/pyflow): it's a package manager like poetry, but it also install whatever python you want like pyenv. Except it provides the binary, no need to compile anything. It's a young project, but I wish it succeeds because it's really a great concept.

    - shiv (shiv.readthedocs.io/): it leverage the concept of zipapp, meaning the ability that python has to execute python inside a zip file. It's a successor to pex. Basically it lets you bundle your code + all deps from virtualenv inside a zip, like a Java .war file. You can then run the resulting zip, a .pyz file, like if it was a regular .py file. It will unzip on the first run automatically. It makes deployment almost as easy as with golang.

    - nuitka (shiv.readthedocs.io/): take your code and all dependancies, turn them into C, and compiles it. Although it does require a bit of setup, since it needs headers and a compiler, it results reliably in a standalone compiled executable that will run on the same architecture with no need for anything else. Also it will speed up your Python program, up to 4 times.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • Nuitka

    Nuitka is a Python compiler written in Python. It's fully compatible with Python 2.6, 2.7, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, 3.10, and 3.11. You feed it your Python app, it does a lot of clever things, and spits out an executable or extension module.

  • g-sorcery

    Framework for automated ebuild generators

  • FWIW it's worth for portage (Gentoo) there is g-sorcery[0], which can create ebuilds for Emacs (m/elpa) and python packages automatically. Similarly there is also cargo-ebuild[1] which can create ebuilds for rust programs/libraries, including a list of all dependencies with hashes.

    I've successfully used cargo-ebuild in the past to create ebuilds automatically, it's a breeze. I'd be surprised if similar tools didn't exist for deb/rpm based distros.

    [0]:https://github.com/jauhien/g-sorcery

    [1]: https://github.com/cardoe/cargo-ebuild

  • cargo-ebuild

    Discontinued cargo extension that can generate ebuilds using the in-tree eclasses

  • FWIW it's worth for portage (Gentoo) there is g-sorcery[0], which can create ebuilds for Emacs (m/elpa) and python packages automatically. Similarly there is also cargo-ebuild[1] which can create ebuilds for rust programs/libraries, including a list of all dependencies with hashes.

    I've successfully used cargo-ebuild in the past to create ebuilds automatically, it's a breeze. I'd be surprised if similar tools didn't exist for deb/rpm based distros.

    [0]:https://github.com/jauhien/g-sorcery

    [1]: https://github.com/cardoe/cargo-ebuild

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • just-pip-tools: An example of managing python dependencies as layered lock files with just and pip-tools

    3 projects | /r/Python | 18 Oct 2023
  • An opinionated Python boilerplate

    6 projects | /r/Python | 12 Mar 2023
  • Run python scripts before compilation using Cargo?

    4 projects | /r/rust | 27 Dec 2022
  • Just 1.0 released!

    9 projects | /r/rust | 22 Feb 2022
  • Python Best Practices for a New Project in 2021

    22 projects | news.ycombinator.com | 5 Jul 2021