SaaSHub helps you find the best software and product alternatives Learn more →
Top 20 Rust llm Projects
-
Tabby (by TabbyML) is a self-hosted AI coding assistant built in Rust, licensed under Apache 2.0, and available at github.com/TabbyML/tabby with around 33,000 GitHub stars as of mid-2026. The latest stable release is v0.32.0 (January 25, 2026), which added Mistral Embedding API support, generic OAuth, and multi-branch repository indexing.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
plano
Plano is an AI-native proxy and data plane for agentic apps — with built-in orchestration, safety, observability, and smart LLM routing so you stay focused on your agents core logic.
Project mention: Show HN: Signals – finding the most informative agent traces without LLM judges | news.ycombinator.com | 2026-04-04Project where Signals are already implemented: https://github.com/katanemo/plano
Happy to answer questions on the taxonomy, implementation details, or where this breaks down.
-
lance
Open Lakehouse Format for Multimodal AI. Convert from Parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..
Project mention: Accelerating Multimodal Vector DB with Hugging Face + LanceDB | dev.to | 2026-02-17 -
bionic-gpt
Bionic is an on-premise replacement for ChatGPT, offering the advantages of Generative AI while maintaining strict data confidentiality
-
-
Project mention: Make a medical chatbot using GraphRAG with SurrealDB + LangChain | dev.to | 2025-07-21
Rust developers have an option to do the same too, thanks to a crate called langchain_rust which as of last year includes support for SurrealDB as a vector store. This implementation doesn't (yet!) include graph queries, but we can still use classic vector search to find recommendations for treatment for a patient.
-
-
-
Project mention: Weave: Merging based on language structure and not lines | news.ycombinator.com | 2026-06-13
On the point around agent collaboration with this, I think there's a larger opportunity in the agent to agent protocol over git beyond just semantic merge.
https://github.com/h5i-dev/h5i has caught my attention lately and I have been meaning to explore this more, I feel like git workflows are leveling up fast since worktrees become ubiquitous and agentic development exposed more how it can unlock parallelization
-
-
tokuin
CLI tool – estimates LLM tokens/costs and runs provider-aware load tests for OpenAI, Anthropic, OpenRouter, or custom endpoints.
Project mention: Show HN: Tokuin – CLI load tester and token estimator for LLM APIs | news.ycombinator.com | 2025-11-08 -
Project mention: Sqz: Compress LLM context to save tokens and reduce costs | news.ycombinator.com | 2026-04-22
-
Project mention: Memoryport: Add 500M tokens of context space to any LLM with <300ms latency | news.ycombinator.com | 2026-03-30
-
-
sochdb
SochDB is a high-performance embedded, ACID-compliant vector database purpose-built for AI agents and memory
Project mention: Stop Reindexing: How We Built Real-Time Search Directly Into the Database using sochDB | dev.to | 2026-02-07In this post, I’ll explain why reindexing is fundamentally broken for real-time systems, and how we built SochDB to make search a native database capability instead of a separate infrastructure problem.
-
-
cli
🔥 The easiest way to run AI Agents from your terminal. Scale your agents with Steel. (by steel-dev)
This steel-browser skill is designed for autonomous web tasks where basic fetch tools fall short, often failing across many sites due to blocking, complexity, and other limitations.
-
Project mention: Show HN: Ambits – LLM code coverage tooling written in Rust | news.ycombinator.com | 2026-02-08
-
Project mention: Out-of-core LLM inference engine written from scratch in Rust | news.ycombinator.com | 2026-07-04
-
Project mention: Agentwerk: A minimal Rust crate for agentic apps | news.ycombinator.com | 2026-05-09
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Rust llms discussion
Rust llms related posts
-
Sqz: Compress LLM context to save tokens and reduce costs
-
Show HN: Signals – finding the most informative agent traces without LLM judges
-
Generate Realistic Dummy Data for MongoDB
-
What's semantic caching?
-
Building a RAG pipeline with Kreuzberg and LangChain
-
Build a RAG System with Python and a Local LLM (No API Costs)
-
Accelerating Multimodal Vector DB with Hugging Face + LanceDB
-
A note from our sponsor - SaaSHub
www.saashub.com | 20 Jul 2026
Index
What are some of the best open-source llm projects in Rust? This list will help you:
| # | Project | Stars |
|---|---|---|
| 1 | tabby | 33,716 |
| 2 | plano | 6,867 |
| 3 | lance | 6,822 |
| 4 | bionic-gpt | 2,334 |
| 5 | chidori | 1,358 |
| 6 | langchain-rust | 1,331 |
| 7 | motorhead | 917 |
| 8 | Corvid-lang | 258 |
| 9 | h5i | 182 |
| 10 | swarms-rs | 172 |
| 11 | tokuin | 143 |
| 12 | sqz | 128 |
| 13 | memoryport | 113 |
| 14 | offeryn | 37 |
| 15 | sochdb | 36 |
| 16 | clerk | 31 |
| 17 | cli | 29 |
| 18 | ambits | 9 |
| 19 | Kortex | 4 |
| 20 | agentwerk | 0 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com