pyre-check VS mamba

Did you come across pyre in your search? MIT license and pretty fast.

https://github.com/facebook/pyre-check

Pyre is a performant type-checker developed by Facebook. Pyre can analyse codebases with millions of lines of code incrementally – providing instantaneous feedback to developers as they write code.

Pyre from Meta, pyright from Microsoft and PyType from Google provide additional assistance. They can 'infer' types based on code flow and existing types within the code.

Have you seen Pyre[0]? Not Rust, OCaml, and pretty fast. Made by a team at Meta and open sourced on GitHub. If you use python-lsp, I wrote an extension[1] to enable integration (though I haven't tested it recently, been programming in rust; it is mostly a "for me" extension).

0: https://pyre-check.org/

1: https://github.com/cricalix/python-lsp-pyre

Internally we use Pyre for Python type checking: https://github.com/facebook/pyre-check

Before type hinting, work had intense rules and linters enforcing docstrings with types. Now, type hints and automatic pyre runs take care of all the heavy lifting.

Python now has an optional type system and if you add one of them such as mypy or pyre to your CI process and you can configure GitHub to refuse the pull request until types are added you can make it somewhat strongly typed.

If you have a preexisting codebase I believe the way you can convert it is to add the types that you know on commits and eventually you will have enough types that adding the missing ones should be easy. For the missing ones Any is a good choice.

https://pyre-check.org and https://github.com/python/mypy are popular.

If you want to learn this stuff as a computer engineer, you can read the code here [0]. I find the math quite helpful.

[0]: https://github.com/state-spaces/mamba

This model was discussed recently: https://news.ycombinator.com/item?id=38522428 It's a new kind of ML model architecture that can be used instead of a transformer in LLMs.

See also the original repo from the paper: https://github.com/state-spaces/mamba

The library is compatible with any auto-regressive model, not transformers. To prove our point we integrated Mamba, a new state-space model architecture, to the library. Try it out!

You might have come across the paper Mamba paper in the last days, which was the first attempt at scaling up state space models to 2.8B parameters to work on language data.

Hey everyone! Many of you might have come across the Mamba paper a few days ago, which introduced an LLM based on a state space model architecture. The Mamba architecture is quite useful as its complexity scales subquadratically with input length and is therefore way more efficient than transformer models: https://github.com/state-spaces/mamba

I got really excited about the paper, so I decided to fine-tune the model on a chat dataset. It turns that this actually worked quite well! The model is quite suitable for casual chatting, which honestly surprised me given that it only has 2.8B parameters and the base model was only trained on the Pile. It's quite exciting that we might have a serious candidate for an architecture that could dethrone transformers.

You can find both my fine-tuning and inference code here: https://github.com/havenhq/mamba-chat