pyenv-virtualenv VS mu1

First of all, install the Python virtual environment from these links: 1 and 2. I developed my GPT-based API in Python version 3.8.18. Pick any Python versions >= 3.7.

git clone https://github.com/pyenv/pyenv-virtualenv.git $(pyenv root)/plugins/pyenv-virtualenv

Install pyenv-virtualenv plugin from repository (in never versions it's included in pyenv.run script I think)

> trying to build a lifeboat for Twitter, Python works, but then modules require builds that break.

> Alternatively, any good resources for the above?

There are many, _unbelievably many_ writeups and tools for Python building and packaging. Some of them are really neat! But paralysis of choice is real. So is the reality that many of the new/fully integrated/cutting edge tools, however superior they may be, just won't get long term support to catch on and stay relevant.

When getting started with Python, I very personally like to choose from a few simple options (others are likely to pipe up with their own, and that's great; mine aren't The One Right Way, just some fairly cold/mainstream takes).

1. First pick what stack you'll be using to develop and test software. In Python this is sadly often going to be different from the stack you'll use to deploy/run it in production, but here we are. There are two sub-choices to be made here:

1.a. How will you be running the _python interpreter_ in dev/test? "I just want to use the Python that came with my laptop" is fine to a point, but breaks down a lot sooner than folks expect (again, the reasons for this are variously reasonable and stupid, but here we are). Personally, I like pyenv (https://github.com/pyenv/pyenv) here. It's a simple tool that builds interpreters on your system and provides shell aliases to adjust pathing so they can optionally be used. At the opposite extreme from pyenv, some folks choose Python-in-Docker here (pros: reproducible, makes deployment environments very consistent with dev; cons: IDE/quick build-and-run automations get tricker). There are some other tools that wrap/automate the same stuff that pyenv does.

1.b. How will you be isolating your project's dependencies? "I want to install dependencies globally" breaks down (or worse, breaks your laptop!) pretty quickly, yes it's a bummer. There are three options here: if you really eschew automations/wrappers/thick tools in general, you can do this yourself (i.e. via "pip install --local", optionally in a dedicated development workstation user account); you can use venv (https://docs.python.org/3/library/venv.html stdlib version of virtualenv, yes the names suck and confusing, here we are etc. etc.), which is widely standardized upon and manually use "pip install" while inside your virtualenv, and you can optionally integrate your virtualenv with pyenv so "inside your virtualenv" is easy to achieve via pyenv-virtualenv (https://github.com/pyenv/pyenv-virtualenv); or you can say "hell with this, I want maximum convenience via a wrapper that manages my whole project" and use Poetry (https://python-poetry.org/). There's no right point on that spectrum, it's up to you to decide where you fall on the "I want an integrated experience and to start prototyping quickly" versus "I want to reduce customizations/wrappers/tooling layers" spectrum.

2. Then, pick how you'll be developing said software: what frameworks or tools you'll be using. A Twitter lifeboat sounds like a webapp, so you'll likely want a web framework. Python has a spectrum of those of varying "thickness"/batteries-included-ness. At the minimum of thickness are tools like Flask (https://flask.palletsprojects.com/en/2.2.x/) and Sanic (like Flask, but with a bias towards performance at the cost of using async and some newer Python programming techniques which tend, in Python, to be harder than the traditional Flask approach: https://sanic.dev). At the maximum of thickness are things like Django/Pyramid. With the minimally-thick frameworks you'll end up plugging together other libraries for things like e.g. database access or web content serving/templating, with the maximally-thick approach that is included but opinionated. Same as before: no right answers, but be clear on the axis (or axes) along with you're choosing.

3. Choose how you'll be deploying/running the software, maybe after prototyping for awhile. This isn't "lock yourself into a cloud provider/hosting platform", but rather a choice about what tools you use with the hosting environment. Docker is pretty uncontentious here, if you want a generic way to run your Python app on many environments. So is "configure Linux instances to run equivalent Python/package versions to your dev/test environment". If you choose the latter, be aware that (and this is very important/often not discussed) many tools that the Python community suggests for local development or testing are very unsuitable for managing production environments (e.g. a tool based around shell state mutation is going to be extremely inconvenient to productionize).

Yeah, that's a lot of choices, but in general there are some pretty obvious/uncontentious paths there. Pyenv-for-interpreters/Poetry-for-packaging-and-project-management/Flask-for-web-serving/Docker-for-production is not going to surprise anyone or break any assumptions. Docker/raw-venv/Django is going to be just as easy to Google your way through.

Again, no one obvious right way (ha!) but plenty of valid options!

Not sure if that's what you were after. If you want a "just show me how to get started"-type writeup rather than an overview on the choices involved, I'm sure folks here or some quick googling will turn up many!

For this, I will use pyenv and the pyenv-virtualenv tools.

I would suggest using pyenv and the pyenv-virtualenv plugin to manage various python versions and virtualenvs

To help, check out pyenv and pyenv-virtualenv (or pyenv with Poetry if you want the new hotness). You essentially can install multiple python versions and create a virtualenv that uses a specific python version. So you could have `myapp-3.7` and `myapp-3.10` each isolated with their own package versions, etc.

Recently moved to Windows from Linux and was looking for a replacement for pyenv which I was using to manage multiple versions of Python. Found pyenv-win but it was missing the pyenv-virtualenv plugin which can be used to create virtualenvs for different Python versions. Frustrated with the lack of options, I decided to create my own CLI called pyenv-win-venv to do the same thing. I created it only for my personal use but later decided to open source it so it has some of the basic features of pyenv-virtualenv and I hope it is useful to other users of pyenv-win.

Pyenv lets you install different versions of python on your system simultaneously, without breaking a thing. You can switch between any of them in one command. You can also bind a specific version to a directory, which will activate every time you enter it. There is an extension that adds support for virtual environments.

I got sick of juggling code that migrated from one category to the other, so I wrote a little script that deals with chopping up a large source file into multiple TUs before feeding them to the compiler.

https://github.com/akkartik/mu1/blob/master/build2

More details: https://news.ycombinator.com/item?id=33574154#33575045

This really speaks to me. Modern software is too hard to assemble from source. If you're shipping sources, every moving part you add increases the odds of something going wrong on other people's computers.

It's worth having some skepticism of tools. By making some operations easy, tools encourage them. Build systems make it easy to bloat software. Package managers make it easy to bloat dependencies. This dynamic explains why Python in particular has such a terrible package management story. It's been around longer than Node or Rust, so if they seem better -- wait 10 years!

For many of my side projects I try to minimize moving parts for anyone (usually the '1' is literally true) who tries them out. I work in Unix, and one thing I built is a portable shell script that acts like a build system while being much more transparent about what it does: https://codeberg.org/akkartik/basic-build

When I use this script my build instructions are more verbose, but I think that's a good thing. They're more explicit for newcomers, and they also impose costs that nudge me to keep my programs minimalist.

You can see this build system evolve to add partial builds and parallel builds in one of my projects:

https://github.com/akkartik/mu1/blob/master/build0

https://github.com/akkartik/mu1/blob/master/build1

https://github.com/akkartik/mu1/blob/master/build2

https://github.com/akkartik/mu1/blob/master/build3

https://github.com/akkartik/mu1/blob/master/build4

Each of these does the same thing for this one repo -- build it -- but adding successively more bells and whistles.

I think providing just the most advanced version, build4, would do my users a disservice. It's also the most likely to break, where build0 is rock solid. If my builds do break for someone, they can poke around and downgrade to a simpler version.

Totally agreed!

Here's a prototype from a few years ago where I tried to make this easier: https://github.com/akkartik/mu1#readme (read just the first few paragraphs)

I still think the full answer lies in this direction.