Our great sponsors
tabby | llama.cpp | |
---|---|---|
24 | 769 | |
17,192 | 56,891 | |
6.2% | - | |
9.9 | 10.0 | |
6 days ago | 2 days ago | |
Rust | C++ | |
GNU General Public License v3.0 or later | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
tabby
- Google CodeGemma: Open Code Models Based on Gemma [pdf]
-
What AI assistants are already bundled for Linux?
NixOS just got tabbyml[1] which is built on llama-cpp. Working on systemsd services the weekend and updating latest tabbyml release which supports rocm in addition to cuda
[1] https://github.com/TabbyML/tabby
[2] https://github.com/NixOS/nixpkgs/pull/291744
- FLaNK Stack Weekly 19 Feb 2024
-
Show HN: Tabby back end in 20 Python lines (self-hosted AI coding assistant)
Nice implementation! It should serve as a great reference for a minimal Tabby's backend API. Thank you for sharing it!
Yeah - ultimately, it won't be as performant or feature-rich compared to https://github.com/TabbyML/tabby, but it's still perfect for educational purposes!
- Stable Code 3B: Coding on the Edge
-
Show HN: I built local copilot alternative using Codellama
Looks interesting! What are the main differences between this and https://github.com/TabbyML/tabby ?
-
Ask HN: Who is hiring? (October 2023)
TabbyML | Software Engineer (Rust) | REMOTE
Self-hosted AI coding assistant. An opensource / on-prem alternative to GitHub Copilot.
Project: https://github.com/TabbyML/tabby
Tabby is seeking a Software Engineer proficient in Rust to join our core engineering team. In this role, you will be responsible for developing the following features:
- Show HN: Tabby – AI Coding Assistant Runs on Apple M1/M2 GPU
-
Meta: Code Llama, an AI Tool for Coding
There are a bunch of VSCode extensions that make use of local models. Tabby seems to be the most friendly right now, but I admittedly haven't tried it myself: https://tabbyml.github.io/tabby/
-
CodeCompose: Meta’s AI Coding Assistant
Check out https://github.com/TabbyML/tabby, which is fully self-hostable and comes with niche features. On M1/M2, it offers a convenient single binary deployment, thanks to Rust. You can find the latest release at https://github.com/TabbyML/tabby/releases/tag/latest.
(Disclaimer: I am the author)
llama.cpp
-
Phi-3 Weights Released
well https://github.com/ggerganov/llama.cpp/issues/6849
- Lossless Acceleration of LLM via Adaptive N-Gram Parallel Decoding
- Llama.cpp Working on Support for Llama3
-
Embeddings are a good starting point for the AI curious app developer
Have just done this recently for local chat with pdf feature in https://recurse.chat. (It's a macOS app that has built-in llama.cpp server and local vector database)
Running an embedding server locally is pretty straightforward:
- Get llama.cpp release binary: https://github.com/ggerganov/llama.cpp/releases
- Mixtral 8x22B
- Llama.cpp: Improve CPU prompt eval speed
-
Ollama 0.1.32: WizardLM 2, Mixtral 8x22B, macOS CPU/GPU model split
Ah, thanks for this! I can't edit my parent comment that you replied to any longer unfortunately.
As I said, I only compared the contributors graphs [0] and checked for overlaps. But those apparently only go back about year and only list at most 100 contributors ranked by number of commits.
[0]: https://github.com/ollama/ollama/graphs/contributors and https://github.com/ggerganov/llama.cpp/graphs/contributors
-
KodiBot - Local Chatbot App for Desktop
KodiBot is a desktop app that enables users to run their own AI chat assistants locally and offline on Windows, Mac, and Linux operating systems. KodiBot is a standalone app and does not require an internet connection or additional dependencies to run local chat assistants. It supports both Llama.cpp compatible models and OpenAI API.
-
Mixture-of-Depths: Dynamically allocating compute in transformers
There are already some implementations out there which attempt to accomplish this!
Here's an example: https://github.com/silphendio/sliced_llama
A gist pertaining to said example: https://gist.github.com/silphendio/535cd9c1821aa1290aa10d587...
Here's a discussion about integrating this capability with ExLlama: https://github.com/turboderp/exllamav2/pull/275
And same as above but for llama.cpp: https://github.com/ggerganov/llama.cpp/issues/4718#issuecomm...
-
The lifecycle of a code AI completion
For those who might not be aware of this, there is also an open source project on GitHub called "Twinny" which is an offline Visual Studio Code plugin equivalent to Copilot: https://github.com/rjmacarthy/twinny
It can be used with a number of local model services. Currently for my setup on a NVIDIA 4090, I'm running both the base and instruct model for deepseek-coder 6.7b using 5_K_M Quantization GGUF files (for performance) through llama.cpp "server" where the base model is for completions and the instruct model for chat interactions.
llama.cpp: https://github.com/ggerganov/llama.cpp/
deepseek-coder 6.7b base GGUF files: https://huggingface.co/TheBloke/deepseek-coder-6.7B-base-GGU...
deepseek-coder 6.7b instruct GGUF files: https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct...
What are some alternatives?
fauxpilot - FauxPilot - an open-source alternative to GitHub Copilot server
ollama - Get up and running with Llama 3, Mistral, Gemma, and other large language models.
turbopilot - Turbopilot is an open source large-language-model based code completion engine that runs locally on CPU
gpt4all - gpt4all: run open-source LLMs anywhere
refact - WebUI for Fine-Tuning and Self-hosting of Open-Source Large Language Models for Coding
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
GPTQ-for-LLaMa - 4 bits quantization of LLaMA using GPTQ
aider - aider is AI pair programming in your terminal
ggml - Tensor library for machine learning
ollama-ui - Simple HTML UI for Ollama
alpaca.cpp - Locally run an Instruction-Tuned Chat-Style LLM