Rust LLM

Open-source Rust projects categorized as LLM

Large Language Models

Edit details
Rust AI CLI mcp ai-agents
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured

Top 23 Rust LLM Projects

Rust logo
LLM

  1. CodeWhale

    DeepSeek v4 coding agent in terminal

    Project mention: Terminal coding agent for DeepSeek V4 | news.ycombinator.com | 2026-05-26

  2. SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo

  3. llmfit

    Hundreds of models & providers. One command to find what runs on your hardware.

    Project mention: Show HN: Find the best local LLM for your hardware, ranked by benchmarks | news.ycombinator.com | 2026-05-15

    I got really good results when I asked Pi (agent) to install

    https://github.com/AlexsJones/llmfit

    and tell me which ones are good for me. It would organize it per use and selection was solid.

  4. meetily

    Privacy first, AI meeting assistant with 4x faster Parakeet/Whisper live transcription, speaker diarization, and Ollama summarization built on Rust. 100% local processing. no cloud required. Meetily (Meetly Ai - https://meetily.ai) is the #1 Self-hosted, Open-source Ai meeting note taker for macOS & Windows. Understand How to write meeting minutes

    Project mention: Yap – free offline voice dictation for Mac/Windows/Linux (Wispr Flow alt) | news.ycombinator.com | 2026-06-27

    I actually found this one today and will be adding it to Opensource Builders soon:

    https://github.com/Zackriya-Solutions/meetily

  5. screenpipe

    YC (S26) | Record your screen 24/7 and plug into your agents. Local, private, secure. Connect to OpenClaw, Hermes agent and 100+ apps

    Project mention: Dev log #6 Networking Deep Dives and Scaling Docs: My 30k Line Week in OSS | dev.to | 2026-07-07

    On the AI front, I’ve been working on screenpipe. I’m really excited about the Model Context Protocol (MCP) work I’m doing there. In PR #3072, I’m building out a structured output format specifically for AI consumption. The goal is to make the data coming out of screenpipe easily "digestible" for LLMs without a ton of brittle parsing logic. It’s still open, but the foundation is solid.

  6. openfang

    Open-source Agent Operating System

    Project mention: OpenFang: A Proper OS for Agents That Run 24/7 | dev.to | 2026-04-26

    OpenFang is an operating system for autonomous agents. Not a framework. Not a wrapper. An actual OS, written from scratch in Rust by RightNow AI. 137,728 lines of code across 14 crates. Compiles to a single 32MB binary with zero external dependencies. This is not something someone cobbled together over a weekend.

  7. memvid

    Memory layer for AI Agents. Replace complex RAG pipelines with a serverless, single-file memory layer. Give your agents instant retrieval and long-term memory.

    Project mention: Show HN: I accidentally built "SQLite for AI memory" (Memvid) | news.ycombinator.com | 2026-01-05

  8. DeepSeek-TUI

    Coding agent for DeepSeek models that runs in your terminal

    Project mention: DeepSeek TUI on Windows: A Practical Look at a Terminal-Native Coding Agent | dev.to | 2026-05-11

  9. cocoindex

    Incremental engine for long horizon agents 🌟 Star if you like it!

    Project mention: CocoIndex Review: Incremental RAG Engine for AI Agents | dev.to | 2026-05-12

  10. aichat

    All-in-one LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI Tools & Agents, with access to OpenAI, Claude, Gemini, Ollama, Groq, and more.

    Project mention: AIChat: All-in-One LLM CLI Tool | news.ycombinator.com | 2025-12-27

  11. baml

    The programming language for agents

    Project mention: Baml: The Programming Language for Agents | news.ycombinator.com | 2026-07-15

  12. GenieX

    Run frontier LLMs and VLMs locally on Qualcomm devices across NPU, GPU, and CPU with a few lines of code

    Project mention: Qualcomm acquires Nexa AI, open-sources GenAI runtime for Hexagon NPUs | news.ycombinator.com | 2026-07-07

  13. rig

    ⚙️🦀 Build modular and scalable LLM Applications in Rust

    Project mention: Agentic workflows with Aerie | dev.to | 2026-04-07

    Unfortunately, rig the underlying library to connect to AI providers usually halts the execution thread instead of triggering a recoverable error.

  14. mistral.rs

    Fast, flexible LLM inference

    Project mention: Run Agent Skills with mistral.rs v0.8.10: /v1/skills support and more | news.ycombinator.com | 2026-06-18

    mistralrs serve --agent --isq 4 -m google/gemma-4-E4B-it

    Super excited for you to try this out and any feedback! Do you have any suggestions for what you would like to see in the next releases?

    Check out the GitHub: https://github.com/EricLBuehler/mistral.rs

  15. code2prompt

    A CLI tool to convert your codebase into a single LLM prompt with source tree, prompt templating, and token counting.

    Project mention: Code2prompt: Convert any folder into a single LLM prompt | news.ycombinator.com | 2026-05-28

  16. forgecode

    AI enabled pair programmer for Claude, GPT, O Series, Grok, Deepseek, Gemini and 300+ models

    Project mention: Zerostack – A Unix-inspired coding agent written in pure Rust | news.ycombinator.com | 2026-05-16

  17. postgresml

    Postgres with GPUs for ML/AI apps.

  18. deepreasoning

    A high-performance LLM inference API and Chat UI that integrates DeepSeek R1's CoT reasoning traces with Anthropic Claude models.

  19. text-embeddings-inference

    A blazing fast inference solution for text embeddings models

    Project mention: Cut token waste across your entire team with the MCP Optimizer | dev.to | 2026-03-11

    The operator defaults to BAAI/bge-small-en-v1.5 as the model and runs the HuggingFace Text Embeddings Inference server. You can increase the replica count via spec.replicas to match your team's throughput needs. One shared instance serves every vMCP in the namespace. For all available configuration options, see the Optimizer docs.

  20. agentos

    A faster, lighter, cheaper alternative to sandboxes. Run any coding agent inside an isolated Linux VM, with agent orchestration built in. (by rivet-dev)

    Project mention: Your Agent Bills While It Waits. Here's the Fix. | dev.to | 2026-07-18

    Their companion project AgentOS runs coding agents inside isolated Linux VMs with an in-process operating system kernel. Everything runs inside the kernel; nothing executes on the host. The pitch: faster, lighter, cheaper than mainstream sandbox providers.

  21. lsp-ai

    LSP-AI is an open-source language server that serves as a backend for AI-powered functionality, designed to assist and empower software engineers, not replace them.

  22. agent-os

    A portable open-source operating system for agents. ~6 ms coldstarts, 32x cheaper than sandboxes. Powered by WebAssembly and V8 isolates. (by rivet-dev)

    Project mention: AgentOS: A portable open-source operating system for agents | news.ycombinator.com | 2026-05-05

  23. moltis

    A secure persistent personal agent server in Rust. One binary, sandboxed execution, multi-provider LLMs, voice, memory, Telegram, WhatsApp, Discord, Teams, and MCP tools. Secure by design, runs on your hardware.

    Project mention: OpenClaw Is a Security Nightmare Dressed Up as a Daydream | news.ycombinator.com | 2026-03-22

    I'm on my 3rd "claw" variant (currently https://github.com/moltis-org/moltis). I had the same issues you had. Moltis is better (at the moment).

  24. trieve

    All-in-one platform for search, recommendations, RAG, and analytics offered via API

    Project mention: Ask HN: Looking for Headless CMS Recommendation | news.ycombinator.com | 2025-09-03
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Rust LLM discussion

Log in or Post with

Rust LLM related posts

Index

What are some of the best open-source LLM projects in Rust? This list will help you:

# Project Stars
1 CodeWhale 35,037
2 llmfit 29,528
3 meetily 25,407
4 screenpipe 20,295
5 openfang 18,025
6 memvid 15,975
7 DeepSeek-TUI 13,516
8 cocoindex 10,904
9 aichat 10,246
10 baml 8,569
11 GenieX 8,226
12 rig 7,945
13 mistral.rs 7,494
14 code2prompt 7,476
15 forgecode 7,461
16 postgresml 6,807
17 deepreasoning 5,364
18 text-embeddings-inference 4,930
19 agentos 3,907
20 lsp-ai 3,188
21 agent-os 2,779
22 moltis 2,771
23 trieve 2,696

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
Loading...