ferrite
llama2.f90
ferrite | llama2.f90 | |
---|---|---|
3 | 7 | |
13 | 16 | |
- | - | |
6.1 | 10.0 | |
6 months ago | 6 months ago | |
Fortran | Fortran | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
ferrite
-
hae – like grep but with natural language queries
This is interesting, I'd like to see how fast it runs. A challenge even with fast implementations of models is there is always some overhead starting up.
I also want to mention this is exactly the kind of use case I had in mind for Ferrite, a dependency free Fortran implementation of sentence transformers: https://github.com/rbitr/ferrite The idea is a very simple script to run transformer inference on a CPU that can be incorporated into a project without pulling in a million dependencies.
I am happy to see though that this is a C++ project and appears not to require pytorch and HF transformers to use!
-
GPU Embedding with GGML
I'll plug the project I've started work on along the same lines - a simplified CPU focused embedding model (right now with distillbert) that's coded as a single file with no dependencies and no abstraction. https://github.com/rbitr/ferrite
- Ferrite – Simple, lightweight transformers in Fortran
llama2.f90
-
GPU Embedding with GGML
I'm mostly a python programmer, but I find a lot of the ML frameworks are overkill for what they actually do, especially for inference. Fortran is pretty close to numpy - it handles arrays natively including slicing and matmul instinsics, you don't have to worry about memory etc. But it compiles into something fast and lightweight much more easily than python. It's nothing you couldn't do in C but I think Fortran is better suited for linear algebra.
See also https://github.com/rbitr/llama2.f90 which is basically the same thing but for running llama models and has 16-bit and 4-bit options and a lot more optimization.
- GitHub - rbitr/llama2.f90: LLaMA2 model in Fortran
- LLaMA2 Model Inference in Fortran
-
Is Mojo the Fortran for AI Programming, or More? – The Next Platform
I clicked on this because of Fortran and it had nothing to do with it. It was more an advertisement for Mojo.
If anyone is interested in Fortran for AI, I am working on a Fortran LLM project: https://github.com/rbitr/llama2.f90
- Show HN: Llama2.f90 – Toy LLaMA2 model inference in Fortran
- Ask HN: Who wants to be hired? (September 2023)
What are some alternatives?
ollama - Get up and running with Llama 3, Mistral, Gemma, and other large language models.
cv
hae - Like grep but with natural language queries
CutBox - CutBox makes your macOS pasteboard awesome.
mariocesar - I’m a software developer, trying to write, create and mostly being a great father
andreigatej.dev-new - My portfolio website. Also a way to share my journey.
Resume - Add latex resume here in case online latex generators blow up
curriculum-vitae - My personal resume/CV
resume
aword - Deploy private GPT-powered AI tools for your business docs
Resume
bgpfeeder - A BGPv4 daemon to distribute an infrequently-changing set of static routes