Koboldcpp Alternatives

stable-diffusion-webui

Stable Diffusion web UI

llama.cpp

LLM inference in C/C++

text-generation-webui

A Gradio web UI for Large Language Models with support for multiple inference backends.

ollama

Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, Mistral Small 3.1 and other large language models.

KoboldAI-Client

For GGUF support, see KoboldCPP: https://github.com/LostRuins/koboldcpp

gpt4all

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

mlc-llm

Universal LLM Deployment Engine with ML Compilation

FastChat

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

SillyTavern

LLM Frontend for Power Users.

SillyTavern

Discontinued LLM Frontend for Power Users. [Moved to: https://github.com/SillyTavern/SillyTavern] (by Cohee1207)

exllama

A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

KoboldAI

serge

A web interface for chatting with Alpaca through llama.cpp. Fully dockerized, with an easy to use API.

lm-evaluation-harness

A framework for few-shot evaluation of language models.

exllamav2

A fast inference library for running LLMs locally on modern consumer-class GPUs

simple-proxy-for-tavern

Discontinued [GET https://api.github.com/repos/anon998/simple-proxy-for-tavern: 404 - Not Found // See: https://docs.github.com/rest/repos/repos#get-a-repository]

Local-LLM-Comparison-Colab-UI

Compare the performance of different LLM that can be deployed locally on consumer hardware. Run yourself with Colab WebUI.

TavernAI

Discontinued TavernAI for nerds [Moved to: https://github.com/Cohee1207/SillyTavern] (by SillyLossy)

runpodctl

🧰 | RunPod CLI for pod management

KoboldAI

KoboldAI is generative AI software optimized for fictional use, but capable of much more!