rye VS Poetry

I tried out rye + uv on a recent greenfield project. They are awesome tools and I'm really excited about their potential.

For me, rye (+ uv underneath) has perhaps the perfect workflow for an open source Python project. So I'm definitely using rye for that from now in -- instead of, say, poetry -- or hatchling directly, following the PyPA boilerplate[1].

You have a way of doing local development against any Python interpreter version. You have a way of tweaking dependencies. It all works atop "standard" PyPA infrastructure like pyproject.toml. You have a single command to build[1] project artifacts, like wheels. And you have a single command to publish new artifact versions to PyPI[2].

I think if you're doing local development on a project that is not meant to be published to PyPI, like a private Django project, then whether to use rye becomes more of a debate. For example, for a Django project I'm working on, I decided to just use uv directly, along with a Makefile. This is because during development of a Django project, I preferred to just use a plain requirements.txt (really, requirements.in) file, avoid the sync/lock workflow that rye imposes, and avoid the need to use something like rye run. And rye's ability to package didn't solve a problem since the Django project wasn't being deployed via a PyPA packaging mechanism.

But this is probably also because the Python interpreter/venv management problem, for me, is already handled by pyenv. I think if you're not already a pyenv user, rye is even more appealing because it handles "all" of the Python issues -- interpreters, requirements/dependencies, and packaging/publishing. (As well as a number of other standard issues besides, like testing, linting, and formatting.) But, in my case, I could hand venv management to uv, and then make dependency management part of a larger Makefile for my Django project, including custom linting, testing, and deployment steps. I wrote a little bit about my high level thoughts on Python packaging and dependency management, though this post was written before rye and uv were out.[4]

I'll also say, I found a little bug in how rye (+ hatch) interacted with my local git setup, and reported it to the rye team, and they helped me get to the bottom of it rather quickly.[5]

[1]: https://packaging.python.org/en/latest/tutorials/packaging-p...

[2]: https://rye-up.com/guide/commands/build/

[3]: https://rye-up.com/guide/commands/publish/

[4]: https://amontalenti.com/2022/10/09/python-packaging-and-zig

[5]: https://github.com/astral-sh/rye/issues/793

I've been using Rye[0] lately, which has been pretty good. It's really just a wrapper around a bunch of underlying tools - it's nice to not have to worry about those and let Rye do it's thing.

All that being said, the creator of Rye is 100% cognizant of that XKCD comic, this [1] is a nice read.

I'm not super well versed in Python tooling at all. I've had to work a lot in Python in the past 6+ months, and I become super confused when I tried making a Python project in my spare time.

I settled on Rye because it just seemed to be the easiest to use.

[0]: https://rye-up.com/

I think Rye actually does handle this mostly correctly (as the sibling comment said). I got through some of it here: https://github.com/mitsuhiko/rye/issues/671. I think actually it's very close to what I actually want (maybe not what Armin wants with multiversion).

Rye[1] is an all in one manager for python projects. Including the python versions and virtualenv, pip etc etc... It seperates tool deps from app deps. Its all configured through a pyproject.toml config file.

Its still new but works well. I'm transiting to it from an unholy mess of pyenv, pip installs and other manual hacks.

If you're starting a new python project that is more than just a straightforward script I'd use Rye from the get go.

[1]https://rye-up.com/

Your first comment irked me because it adds zero value to the discussion. You lazily threw out XKCD 927 which the Rye author explicitly mentioned themselves.

If you click into their link "Should Rye Exist" [1] you'll see that XKCD 927 is literally the first sentence and full width image.

[1] https://github.com/mitsuhiko/rye/discussions/6

Try Rye.

Since this is a discussion on dependency management in Python - does anyone use rye [0] regularly now? I'm interested in using it but want a little more social validation before I try - some issues with package managers only appear after you've invested considerable time.

[0]: https://rye-up.com/

The short term solution is "relieving the packaging pain" link in the article.

The long term solution is described in the "What a solution could look like?" section of https://www.bitecode.dev/p/why-is-the-python-installation-pr...

The community is buzzing with attempts to fix those issues this year, so I’m hopping those posts will become obsolete one day.

Flask’s author is attempting something interesting with rye: https://github.com/mitsuhiko/rye

Trio’s author is drafting a spec for the equivalent of wheels, but for the whole python interpreter: https://github.com/njsmith/posy/blob/main/pybi/README.md

Not advocating to use them right now, but the fact is bootstrapping Python is finally acknowledged as one major cause of packaging issues and a priority to solve.

I'm building a package manager for Python (Rye) in Rust and it is modeled after cargo and rustup. It like rustup manages itself. This means it has commands such as rye self update which downloads the latest version and swaps itself out. Likewise there is rye self uninstall which uninstalls rye itself.

You can manage dependencies in Python with the package manager pip, which comes pre-installed with Python. Pip allows you to install and uninstall Python packages, and it uses a requirements.txt file to keep track of which packages your project depends on. However, pip does not have robust dependency resolution features or isolate dependencies for different projects; this is where tools like pipenv and poetry come in. These tools create a virtual environment for each project, separating the project's dependencies from the system-wide Python environment and other projects.

Poetry provides packaging and dependency management for Python. If you haven't already, install poetry via pip:

Poetry

The Semantify repository provides an example Astro.js project. Ensure you have poetry installed, then build the project from the root of the repository:

Has anyone else been paying attention to how hilariously hard it is to package PyTorch in poetry?

https://github.com/python-poetry/poetry/issues/6409

Based on this comment 5 days ago[0], it's working? I'm not sure didn't dig in too far but based on that comment it seems fair to say that it's not fully Poetry's fault because torch removed hashes (which poetry needs to be effective) for a while only recently adding it back in.

Not sure where I would stand if I fully investigated it tho.

[0] https://github.com/python-poetry/poetry/issues/6409#issuecom...

We will be running this project in Python 3.10 on Mac/Linux, and we will use Poetry to manage our dependencies. Later, we will bundle our app into a container using docker for deployment.

Here are the two main packaging issues I run into, specifically when using Poetry:

1) Lack of support for building extension modules (as mentioned by the article). There is a workaround using an undocumented feature [0], which I've tried, but ultimately decided it was not the right approach. I still use Poetry, but build the extension as a separate step in CI, rather than kludging it into Poetry.

2) Lack of support for offline installs [1], e.g. being able to download the dependencies, copy them to another machine, and perform the install from the downloaded dependencies (similar to using "pip --no-index --find-links=."). Again, you can work around this (by using "poetry export --with-credentials" and "pip download" for fetching the dependencies, then firing up pypiserver [2] to run a local PyPI server on the offline machine), but ideally this would all be a first class feature of Poetry, similar to how it is in pip.

I don't have the capacity to create Pull Requests for addressing these issues with Poetry, and I'm very grateful for the maintainers and those who do contribute. Instead, on the linked issues I share my notes on the matter, in the hope that it may at least help others and potentially get us closer to a solution.

Regardless, I'm sticking with Poetry for now. Though to be fair, the only other Python packaging tools I've used extensively are Pipenv and pip/setuptools. It's time consuming to thoroughly try out these other packaging tools, and is generally lower priority than developing features/fixing bugs, so it's helpful to read about the author's experience with these other tools, such as PDM and Hatch.

[0] https://github.com/python-poetry/poetry/issues/2740

[1] https://github.com/python-poetry/poetry/issues/2184

[2] https://pypi.org/project/pypiserver/

We believe that poetry is currently the best tool for this purpose, besides of being the most popular one at the moment. This is why we will use poetry to manage the dependencies of our project throughout this series of posts. Poetry allows you to declare the libraries your project depends on, and it will manage (install/update) them for you. Poetry also allows you to package your project into a distributable format and publish it to a repository, such as PyPI. We strongly recommend you to learn more about this tool by reading the official documentation.

I started using poetry. The problem is poetry will not install if there is dependency conflict and there is no way to ignore: github