Top 19 Rust local-llm Projects
-
deltafin
Run full Kimi K3 on a single device. And an OpenAI-compatible API server for local chat and coding agents.
-
AppSignal
Monitoring that respects your time & budget. APM, error tracking, and dashboards for modern web apps. Ten-minute setup, transparent flat pricing, and support from engineers who actually use the product.
-
prismos-ai
Local-first desktop AI: chat with your files, turn one line into a living 3D scene, investigate logs and harden configs. Runs on your laptop, works offline.
Project mention: Show HN: PrismOS turns one-line prompts into offline 3D scenes with a local LLM | news.ycombinator.com | 2026-10-04 -
pulsar
SSD-streaming inference engine for giant MoE models (Rust + CUDA). GLM 5.2 743B at 2 tok/s and Hy3 295B at 7 tok/s on two consumer 16GB GPUs. Zero-config multi-GPU: measures PCIe bandwidth, places attention and hot experts where they fit. (by giannisanni)
Project mention: Show HN: Pizza Bot – An inbox for AI agents that work in the background | news.ycombinator.com | 2026-09-15 -
claudectl
Auto pilot for Claude Code - connect multiple coding agents to a local LLM brain. 🆕 with a hive mind now
-
Oxide-Lab
Modern desktop application (Rust + Tauri v2 + Svelte 5 + Candle (HF)) for communicating with AI models that runs completely locally on your computer. No subscriptions, no data sent to the internet — just you and your personal AI assistant
Project mention: Oxide Lab: Private Local AI Chat on Your PC - No Internet, No Subscriptions Required | dev.to | 2025-12-04GitHub Repository: https://github.com/FerrisMind/oxide-lab
-
Project mention: Show HN: Wizard – Self-extending Rust terminal AI agent (one-line install) | news.ycombinator.com | 2026-07-11
-
EuLLM
Open-source platform for creating, distributing and running sovereign EU-compliant LLMs. Verticalize any model for your domain, language and brand. AI Act ready.
Project mention: Show HN: Replacing cloud LLM APIs with local, domain-specific models | news.ycombinator.com | 2026-03-25 -
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
Project mention: Hedos – open-source macOS app for running local LLMs | news.ycombinator.com | 2026-07-14
-
Agent-Span
The Web Access Gateway for AI Agents — 52 channels, 92 MCP tools, 9 SDKs, self-healing backends, async Rust
-
nocodo
Coding agent built around small (<10B) models for local development with a Rust (Actix Web, Diesel)/Typescript (Solid, Tailwind) opinionated stack. No coding knowledge needed. Multiple agents and roles - Product Owner, PM, Engineering Manager, Rust Engineer, SolidJS Engineer, etc. work together to build full-stack apps.
It is all binary, all the way down. Code is text that is passes the compiler's checks. Human language is text that has a really ambiguous compiler. And all text is still binary in the computer.
If you think about it, the transformers architecture was created to solve language translation. It works well for human language to code and other way around, already!
What we need is better tooling for this translation on either side. I started working on https://github.com/brainless/nocodo/blob/feature/praxis_agen... for this reason - how can we go from human language to code representing it.
-
ax-engine
One Mac process. Many models. Real speed. Multi-model LLM serving with prefix reuse, MTP acceleration, and OpenAI APIs — built for Apple Silicon, measured against mlx-lm and llama.cpp.
Project mention: Ax-engine: Native Apple Silicon ML inference runtime with a fast Rust core | news.ycombinator.com | 2026-06-09 -
claudette
Air-gapped AI coding agent in one Rust binary + Q56, a hidden-test benchmark for local coding models: 16 configs, 36 full runs, no LLM judge. Runs entirely on your own hardware through Ollama or LM Studio. No cloud brain. No API key. No telemetry. (by mrdushidush)
Every number below was recounted from the committed SCORES-*.tsv files, not transcribed from notes. The raw data - 36 rows, one per run, each with its full failure list - is RESULTS-q56.csv.
-
-
gguf-switchboard
Run GGUF through llama.cpp and SafeTensors through vLLM behind one OpenAI-compatible endpoint. Your coding tools select a model; the switchboard manages the local runtime, process, and resident-model change.
GGUF Switchboard is open source: GitHub — GGUF Switchboard
-
OpenYoke
A private, local-first desktop app for chatting with open-source AI models via Ollama — with branching conversations you explore as a visual graph. No cloud, no accounts. macOS · Windows · Linux.
Project mention: OpenYoke – Branch AI conversations into a graph, with local or cloud models | news.ycombinator.com | 2026-07-14 -
declass
Coding agent for your terminal that keeps private context local: a cloud model writes the code, a local model reads your sensitive files.
Project mention: Declass is a coding agent that keeps sensitive context on your machine | news.ycombinator.com | 2026-10-07 -
docent
Docent: your private AI assistant. Chat with your PDFs and Office files, search the web and connect MCP servers, running on your own machine. (by smartloop-ai)
Project mention: Docent- An Open Source Private AI assistant in your terminal | news.ycombinator.com | 2026-10-04 -
rai
CPU-only LLM inference engine in pure Rust — 4-bit quantized models, hand-written AVX2 kernels, speculative decoding, and a local HTTP/MCP server. No GPU, no Python runtime. (by Classevelabs)
Project mention: Rai: CPU-only LLM inference engine in pure Rust | news.ycombinator.com | 2026-10-02 -
localmind
Run any local LLM with persistent memory and context. CLI agent over Ollama with SQLite-backed hybrid recall. No cloud.
Project mention: Local LLM agent with persistent memory and learnable skills | news.ycombinator.com | 2026-04-17 -
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Rust local-llm discussion
Rust local-llm related posts
Index
What are some of the best open-source local-llm projects in Rust? This list will help you:
| # | Project | Stars |
|---|---|---|
| 1 | deltafin | 828 |
| 2 | prismos-ai | 431 |
| 3 | pulsar | 226 |
| 4 | claudectl | 148 |
| 5 | Oxide-Lab | 116 |
| 6 | wizard | 103 |
| 7 | EuLLM | 58 |
| 8 | hedos | 56 |
| 9 | Agent-Span | 42 |
| 10 | nocodo | 31 |
| 11 | ax-engine | 30 |
| 12 | claudette | 19 |
| 13 | llmterm | 10 |
| 14 | gguf-switchboard | 7 |
| 15 | OpenYoke | 6 |
| 16 | declass | 0 |
| 17 | docent | 0 |
| 18 | rai | 0 |
| 19 | localmind | 0 |
Sponsored