catai
llama.cpp
catai | llama.cpp | |
---|---|---|
7 | 775 | |
408 | 57,463 | |
1.7% | - | |
8.6 | 10.0 | |
3 months ago | 3 days ago | |
TypeScript | C++ | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
catai
-
Are you sure you are focusing on the right things? (venting)
The easiest tool I found is CatAI: https://github.com/ido-pluto/catai You just type 3 npm commands and THATS IT! You have your own Chat Web UI on your computer without hundrets of settings
- How to use CatAI to apologize to your boss
-
Wizard-Vicuna-13B-Uncensored
I am a noob. I saw your comment on github and another post here. I am confused about what has changed and what us users have to do. Do we have to update llama.cpp and redownload all the models(I am using something called catai instead of the webui, i think it also uses llama.cpp)? How do we know which versions of the models are compatible with which vesions of llama.cpp?
-
Google removes the waitlist on Bard today and will be available in 180 more countries
https://github.com/ggerganov/llama.cpp https://github.com/oobabooga/text-generation-webui https://github.com/mlc-ai/mlc-llm https://github.com/cocktailpeanut/dalai https://github.com/ido-pluto/catai (this is super easy to install but it doesnt provide an api or have integration with langchain)
-
GPT For All 13B (/GPT4All-13B-snoozy-GPTQ) is Completely Uncensored, a great model
Pretty simple using catai.
- How to run something like chatgpt, locally?
-
How to install Wizard-Vicuna
You can check out the original GitHub project here
llama.cpp
-
Ask HN: Affordable hardware for running local large language models?
Yes, Metal seems to allow a maximum of 1/2 of the RAM for one process, and 3/4 of the RAM allocated to the GPU overall. There’s a kernel hack to fix it, but that comes with the usual system integrity caveats. https://github.com/ggerganov/llama.cpp/discussions/2182
- Xmake: A modern C/C++ build tool
-
Better and Faster Large Language Models via Multi-Token Prediction
For anyone interested in exploring this, llama.cpp has an example implementation here:
https://github.com/ggerganov/llama.cpp/tree/master/examples/...
- Llama.cpp Bfloat16 Support
-
Fine-tune your first large language model (LLM) with LoRA, llama.cpp, and KitOps in 5 easy steps
Getting started with LLMs can be intimidating. In this tutorial we will show you how to fine-tune a large language model using LoRA, facilitated by tools like llama.cpp and KitOps.
- GGML Flash Attention support merged into llama.cpp
-
Phi-3 Weights Released
well https://github.com/ggerganov/llama.cpp/issues/6849
- Lossless Acceleration of LLM via Adaptive N-Gram Parallel Decoding
- Llama.cpp Working on Support for Llama3
-
Embeddings are a good starting point for the AI curious app developer
Have just done this recently for local chat with pdf feature in https://recurse.chat. (It's a macOS app that has built-in llama.cpp server and local vector database)
Running an embedding server locally is pretty straightforward:
- Get llama.cpp release binary: https://github.com/ggerganov/llama.cpp/releases
What are some alternatives?
alpaca-electron - The simplest way to run Alpaca (and other LLaMA-based local LLMs) on your own computer
ollama - Get up and running with Llama 3, Mistral, Gemma, and other large language models.
HyunGPT - chatbot thing
gpt4all - gpt4all: run open-source LLMs anywhere
AutoGPTQ - An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
dalai - The simplest way to run LLaMA on your local machine
GPTQ-for-LLaMa - 4 bits quantization of LLaMA using GPTQ
mlc-llm - Enable everyone to develop, optimize and deploy AI models natively on everyone's devices.
ggml - Tensor library for machine learning
alpaca.cpp - Locally run an Instruction-Tuned Chat-Style LLM