local.ai
LocalAI
| local.ai | LocalAI | |
|---|---|---|
| 4 | 82 | |
| 520 | 19,593 | |
| - | 7.1% | |
| 8.7 | 9.9 | |
| 7 months ago | 7 days ago | |
| TypeScript | C++ | |
| GNU General Public License v3.0 only | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
local.ai
-
Why does GPT4all respond so slowly on my machine?
I tried to launch gpt4all on my laptop with 16gb ram and Ryzen 7 4700u. Gpt4all doesn't work properly. It uses igpu at 100% level instead of using cpu. And it can't manage to load any model, i can't type any question in it's window. Faraday.dev, secondbrain.sh, localai.app, lmstudio.ai, rwkv runner, LoLLMs WebUI, kobold cpp: all these apps run normally. Only gpt4all and oobabooga fail to run.
-
All AI Models, from 3B to 13B running at ~0.5 tokens/s, what could be causing this?
Sidenote: can you try out localai.app and see if it's faster than oobabooga on your end? (It's all CPU inferencing as well, but just curious if there's any speed gain).
-
Releasing local.ai - an LLM local playground with minimal setup
Thank you. When I first started the project and got the domain localai.app, I had no idea LocalAI was a thing. A friend of mine forwarded me a link to that project mid May, and I was like dang it, let's just add a dot and call it a day (for now...) - local "dot" ai vs LocalAI lol
LocalAI
- Drop-In Replacement for ChatGPT API
- Voxos.ai – An Open-Source Desktop Voice Assistant
- Ask HN: Set Up Local LLM
- FLaNK Stack Weekly 11 Dec 2023
- Is there any open source app to load a model and expose API like OpenAI?
-
What do you use to run your models?
If you're running this as a server, I would recommend LocalAI https://github.com/mudler/LocalAI
-
OpenAI Switch Kit: Swap OpenAI with any open-source model
LocalAI can do that: https://github.com/mudler/LocalAI
https://localai.io/features/openai-functions/
-
"ChatGPT romanesc"
De inspirație, LocalAI, un replacement la OpenAI. E deja hot pe GitHub.
-
Local LLM's to run on old iMac / Hardware
Your hardware should be fine for inferencing, as long as you don't bother trying to get the GPU working.
My $0.02 would be to try getting LocalAI running on your machine with OpenCL/CLBlas acceleration for your CPU. If you're running other things, you could limit the inferencing process to 2 or 3 threads. That should get it working; I've been able to inference even 13b models on cheap Rockchip SOCs. Your CPU should be fine, even if it's a little outdated.
LocalAI: https://github.com/mudler/LocalAI
Some decent models to start with:
TinyLlama (extremely small/fast): https://huggingface.co/TheBloke/TinyLlama-1.1B-Chat-v0.3-GGU...
Dolphin Mistral (larger size, better responses: https://huggingface.co/TheBloke/dolphin-2.1-mistral-7B-GGUF
-
Retrieval Augmented Generation in Go
Neither of this really requires OpenAI. You can do it with locally-running models via something like https://github.com/mudler/LocalAI
What are some alternatives?
chatgpt-api-example - Batman/Alfred chatbot example with ChatGPT API
gpt4all - gpt4all: run open-source LLMs anywhere
pnpm - Fast, disk space efficient package manager
ollama - Get up and running with Llama 3, Mistral, Gemma, and other large language models.
qadog - QA agent that follows user stories
llama-cpp-python - Python bindings for llama.cpp
window.ai - Use your own AI models on the web
private-gpt - Interact with your documents using the power of GPT, 100% privately, no data leaks
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
FastChat - An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
localGPT - Chat with your documents on your local device using GPT models. No data leaves your device and 100% private.
gpt4free - The official gpt4free repository | various collection of powerful language models