cformers
gpt4all.cpp
| cformers | gpt4all.cpp | |
|---|---|---|
| 4 | 2 | |
| 315 | 510 | |
| 0.6% | - | |
| 6.7 | 6.5 | |
| 5 months ago | about 1 year ago | |
| C | C | |
| MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
cformers
-
[P] rwkv.cpp: FP16 & INT4 inference on CPU for RWKV language model
it's a combination of things, and removing python from the loop isn't essential to achieving most of these performance gains. the main trick is quantizing the weights and compiling the model. concrete example that builds on top of ggml with python APIs: https://github.com/NolanoOrg/cformers
- Cformers 🚀 - "Transformers with a C-backend for lightning-fast CPU inference". | Nolano
-
FauxPilot – an open-source GitHub Copilot server
We will add quantized CodeGen for fast inference on CPUs up on cformers (https://github.com/NolanoOrg/cformers/) by later today.
gpt4all.cpp
What are some alternatives?
llama.cpp - LLM inference in C/C++
alpaca.cpp - Locally run an Instruction-Tuned Chat-Style LLM
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
RWKV-LM - RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
rwkv.cpp - INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
CodeGen - CodeGen is a family of open-source model for program synthesis. Trained on TPU-v4. Competitive with OpenAI Codex.
llm - An ecosystem of Rust libraries for working with large language models
fauxpilot - FauxPilot - an open-source alternative to GitHub Copilot server