miniforge VS mamba

Nope they don't use venv for any of the oobabooga# variants nor is it recommended for the git version. I'm using https://github.com/conda-forge/miniforge#mambaforge-pypy3 (better version of the recommended conda) for the git variant. The oobabooga* variant uses micro/miniconda (I suck with names) which you can easily drop into with cmd_?something? and does it all internally. Like whoever built that whole environment setup for the _windows/linux/mac did a great job.

wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-aarch64.sh .

Mambaforge!

Just to further clarify: you don't need mamba to avoid the Anaconda distribution. The place you get mambaforge also supplies (and originally supplied) miniforge, which is miniconda with conda-forge set as the default channel. All the *forge installers do in this regard is automatically set conda-forge as the default (and only) channel, which is something one can do manually with miniconda.

I'd recommend using miniforge if you're comfortable with CLIs, otherwise https://www.anaconda.com/.

Tip 1: To use less of your Codespaces resources start with a smaller image like Miniconda or Miniforge and install only what you need.

miniforge, no need to deal with conda environments anymore. https://github.com/conda-forge/miniforge

> how the recall can grow unbounded with no tradeoff

this? https://github.com/state-spaces/mamba/issues/175

If you want to learn this stuff as a computer engineer, you can read the code here [0]. I find the math quite helpful.

[0]: https://github.com/state-spaces/mamba

This model was discussed recently: https://news.ycombinator.com/item?id=38522428 It's a new kind of ML model architecture that can be used instead of a transformer in LLMs.

See also the original repo from the paper: https://github.com/state-spaces/mamba

[2] - https://github.com/state-spaces/mamba

Out of curiosity, does anyone feel as though there's any benefit to linking to reddit when we can link to whatever the link is? I for one do not click the link and read discussion on reddit - if I wanted that sort of discussion, I would browse there, not HN.

The library is compatible with any auto-regressive model, not transformers. To prove our point we integrated Mamba, a new state-space model architecture, to the library. Try it out!

I ran the NanoGPT of Karparthy replacing Self-Attention with Mamba on his TinyShakespeare Dataset and within 5 minutes it started spitting out the following:

You might have come across the paper Mamba paper in the last days, which was the first attempt at scaling up state space models to 2.8B parameters to work on language data.