alpaca.cpp
LocalAI
| alpaca.cpp | LocalAI | |
|---|---|---|
| 94 | 83 | |
| 9,878 | 19,862 | |
| - | 8.3% | |
| 9.4 | 9.9 | |
| about 1 year ago | 3 days ago | |
| C | C++ | |
| MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
alpaca.cpp
-
LLaMA Now Goes Faster on CPUs
Where's the 30B-in-6GB claim? ^FGB in your GH link finds [0] which is neither by jart nor by ggerganov but by another user who promptly gets told to look at [1] where Justine denies that claim.
[0] https://github.com/antimatter15/alpaca.cpp/issues/182 -
Is there potential to short NVDA?
You can just download the language model, dude!!! Everyone doesn’t need to make their own and the open source models literally get better every day.
- [Oobabooga] Alpaca.cpp est extrêmement simple à travailler.
-
Hollywood’s Screenwriters Are Right to Fear AI
Alpaca
-
Square Enix’s AI Tech Demo Is a Staggering Failure
Square could have also trained a more specific data source for their NLP, very similar to Alpaca. Alpaca was trained from interactions from a larger dataset. So while it isn't as smart, it's still able to understand instructions and act upon them.
- [Singularity] Ich bin Alpaka 13B - Frag mich alles
-
Alpaca Vs. Final Jeopardy
The model I found was in 8 parts. The alpaca.cpp chat client (chat.cpp) needs to be modified to run the 8 part model, documented here: https://github.com/antimatter15/alpaca.cpp/issues/149
-
LocalAI: OpenAI compatible API to run LLM models locally on consumer grade hardware!
try the instructions on this github repo https://github.com/antimatter15/alpaca.cpp, its not the best one but I was able to run this model on my linux machine with 16GB memory, I think its a good starting point.
-
What educational materials do you think would be most useful during/after collapse?
Doesn't run offline. If you're running something without a beefy-ish GPU, there's https://github.com/antimatter15/alpaca.cpp .
-
ChatGPT Reignited My Passion For Coding
Ye, atm. toying with alpaca 7B/13B in a local install.
LocalAI
- LocalAI: Self-hosted OpenAI alternative reaches 2.14.0
- Drop-In Replacement for ChatGPT API
- Voxos.ai – An Open-Source Desktop Voice Assistant
- Ask HN: Set Up Local LLM
- FLaNK Stack Weekly 11 Dec 2023
- Is there any open source app to load a model and expose API like OpenAI?
-
What do you use to run your models?
If you're running this as a server, I would recommend LocalAI https://github.com/mudler/LocalAI
-
OpenAI Switch Kit: Swap OpenAI with any open-source model
LocalAI can do that: https://github.com/mudler/LocalAI
https://localai.io/features/openai-functions/
-
"ChatGPT romanesc"
De inspirație, LocalAI, un replacement la OpenAI. E deja hot pe GitHub.
-
Local LLM's to run on old iMac / Hardware
Your hardware should be fine for inferencing, as long as you don't bother trying to get the GPU working.
My $0.02 would be to try getting LocalAI running on your machine with OpenCL/CLBlas acceleration for your CPU. If you're running other things, you could limit the inferencing process to 2 or 3 threads. That should get it working; I've been able to inference even 13b models on cheap Rockchip SOCs. Your CPU should be fine, even if it's a little outdated.
LocalAI: https://github.com/mudler/LocalAI
Some decent models to start with:
TinyLlama (extremely small/fast): https://huggingface.co/TheBloke/TinyLlama-1.1B-Chat-v0.3-GGU...
Dolphin Mistral (larger size, better responses: https://huggingface.co/TheBloke/dolphin-2.1-mistral-7B-GGUF
What are some alternatives?
gpt4all - gpt4all: run open-source LLMs anywhere
llama.cpp - LLM inference in C/C++
ollama - Get up and running with Llama 3, Mistral, Gemma, and other large language models.
coral-pi-rest-server - Perform inferencing of tensorflow-lite models on an RPi with acceleration from Coral USB stick
llama-cpp-python - Python bindings for llama.cpp
ggml - Tensor library for machine learning
private-gpt - Interact with your documents using the power of GPT, 100% privately, no data leaks
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
alpaca-lora - Instruct-tune LLaMA on consumer hardware
FastChat - An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.