mamba VS pip-tools

If you want to learn this stuff as a computer engineer, you can read the code here [0]. I find the math quite helpful.

[0]: https://github.com/state-spaces/mamba

This model was discussed recently: https://news.ycombinator.com/item?id=38522428 It's a new kind of ML model architecture that can be used instead of a transformer in LLMs.

See also the original repo from the paper: https://github.com/state-spaces/mamba

The library is compatible with any auto-regressive model, not transformers. To prove our point we integrated Mamba, a new state-space model architecture, to the library. Try it out!

You might have come across the paper Mamba paper in the last days, which was the first attempt at scaling up state space models to 2.8B parameters to work on language data.

Hey everyone! Many of you might have come across the Mamba paper a few days ago, which introduced an LLM based on a state space model architecture. The Mamba architecture is quite useful as its complexity scales subquadratically with input length and is therefore way more efficient than transformer models: https://github.com/state-spaces/mamba

I got really excited about the paper, so I decided to fine-tune the model on a chat dataset. It turns that this actually worked quite well! The model is quite suitable for casual chatting, which honestly surprised me given that it only has 2.8B parameters and the base model was only trained on the Pile. It's quite exciting that we might have a serious candidate for an architecture that could dethrone transformers.

You can find both my fine-tuning and inference code here: https://github.com/havenhq/mamba-chat

> Why is the "requirements.txt" file a stupid flat listing of all transitive dependencies with pinned versions? It makes it harder to change library versions even if there are no true conflicts.

My friend, here is what you seek: https://github.com/jazzband/pip-tools

requirements.txt is flat because it's really the output of `pip freeze`. It's supposed to completely and exactly rebuild the environment. Unfortunately it's far too flexible and people abuse it by putting in only direct dependencies etc.

If you're writing packages, you don't need a requirements.txt at all, by the way. Package dependencies (only direct dependencies) live in pyproject.toml with the rest of the package config. requirements.txt (and pip tools) are only for when you want to freeze the whole environment, like for a server deployment.

For all my projects I found myself regenerating manual lock files using complex shell commands with pip-compile to get a reproducible environments across devices using a custom pre-install-command. I finally decided that instead of hacking together the same solution on all my projects I would build a plugin that handles this complexity for me.

Instead of venv, we are using pip-tools in this starter kit. pip-tools take things further in dependency management. Check out what pip-tools does in their official GitHub repo. In short, it helps your project find the best match for the dependent packages. For example, you might need two packages A and B in your project that requires same package C under the hood. But A requires any version of C from 1.0.1 to 1.0.10 and B requires any version of C from 1.0.7 to 1.0.15. Pip tools will automatically compile the version of 'C' that suits for both of your packages.

I've created a small project called just-pip-tools that combines pip-tools and just to manage Python dependencies in a layered approach. This isn't a magic bullet; it's a set of files you can adapt to your needs.

pip-tools

I recommend generating a lockfile to document this information, as you might do with pip-tools.

Check out pip-tools [1] which does exactly that, albeit in a slightly more polished way.

[1]: https://github.com/jazzband/pip-tools

I've been using pip-compile from https://github.com/jazzband/pip-tools for this use case; a standard project Makefile defines "make update" which pip-compiles the current requirements, and "make install" installs the frozen requirements list.

This way I can install the same bill of materials every time

I like pip-tools for this use case—it lets you write a separate requirements file with “loose” dependencies, which then gets compiled to a requirements.txt with pinned version numbers (it essentially becomes your “lock” file). It also supports layered dependencies, which allows you to create “dev dependencies”.