twinny
koboldcpp
| twinny | koboldcpp | |
|---|---|---|
| 7 | 180 | |
| 1,750 | 3,887 | |
| - | - | |
| 9.9 | 10.0 | |
| 4 days ago | 6 days ago | |
| TypeScript | C++ | |
| MIT License | GNU Affero General Public License v3.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
twinny
- Twinny: Locally hosted (or API hosted) AI code completion for Visual Studio Code
-
The lifecycle of a code AI completion
For those who might not be aware of this, there is also an open source project on GitHub called "Twinny" which is an offline Visual Studio Code plugin equivalent to Copilot: https://github.com/rjmacarthy/twinny
It can be used with a number of local model services. Currently for my setup on a NVIDIA 4090, I'm running both the base and instruct model for deepseek-coder 6.7b using 5_K_M Quantization GGUF files (for performance) through llama.cpp "server" where the base model is for completions and the instruct model for chat interactions.
llama.cpp: https://github.com/ggerganov/llama.cpp/
deepseek-coder 6.7b base GGUF files: https://huggingface.co/TheBloke/deepseek-coder-6.7B-base-GGU...
deepseek-coder 6.7b instruct GGUF files: https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct...
- Private Ollama GitHub Copilot Alternative with FIM and Chat
- Ollama AI code completion plugin for VSCode, 100% free and 100% private
- A new locally hosted AI code completion API and vscode extension. Like Copilot but totally free and best of all private.
- Continue with LocalAI: An alternative to GitHub's Copilot that runs locally
-
Locally hosted code completion API and vscode extension. 100% free and 100% private.
https://github.com/rjmacarthy/twinny - vscode extension https://github.com/rjmacarthy/twinny-api - python inference api
koboldcpp
- Any Online Communities on Local/Home AI?
- Koboldcpp-1.62.1 adds support for Command-R+
- Show HN: I made an app to use local AI as daily driver
-
Easiest way to show my model to my mom?
FYI this is the easiest way to host on the horde: https://github.com/LostRuins/koboldcpp
- IT Veteran... why am I struggling with all of this?
- What do you use to run your models?
- ByteDance AI researcher suggests that open source model more powerful than Gemini to be released soon
- i need some help guys
-
[Guide] How install KoboldAI in Android via Termux (Update 04-12-2023)
For more information of Koboldcpp look this guide: https://github.com/LostRuins/koboldcpp/wiki
-
SillyTavern 1.10.10 has been released
Out of curiosity, is there a specific reason for this? The most popular fork KoboldCpp is in active development, and was the first to adopt the Min P sampler, and even distincts itself with the context shift feature. Just wondering what this means for the future. Thanks!
What are some alternatives?
code-llama-for-vscode - Use Code Llama with Visual Studio Code and the Continue extension. A local LLM alternative to GitHub Copilot.
KoboldAI
twinny-api - Locally hosted AI code completion server. Like Github Copilot but 100% free and 100% private.
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
pinferencia - Python + Inference - Model Deployment library in Python. Simplest model inference server ever.
TavernAI - Atmospheric adventure chat for AI language models (KoboldAI, NovelAI, Pygmalion, OpenAI chatgpt, gpt-4)
ollama - Get up and running with Llama 3, Mistral, Gemma, and other large language models.
KoboldAI - KoboldAI is generative AI software optimized for fictional use, but capable of much more!
aichat - All-in-one AI-Powered CLI Chat & Copilot that integrates 10+ AI platforms, including OpenAI, Azure-OpenAI, Gemini, VertexAI, Claude, Mistral, Cohere, Ollama, Ernie, Qianwen...
ChatRWKV - ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.
SillyTavern - LLM Frontend for Power Users. [Moved to: https://github.com/SillyTavern/SillyTavern]
exllama - A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.