Stream helps developers build engaging apps that scale to millions with performant and flexible Chat, Feeds, Moderation, and Video APIs and SDKs powered by a global edge network and enterprise-grade infrastructure. Learn more →
Top 23 LLM Open-Source Projects
-
ollama
Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, Mistral Small 3.1 and other large language models.
NOTE : We are using the OllamaLLM and RetrievalQA to connect to our local LLM API but we could use the Ollama API here with the /api/generate end point, but this would be more verbose.
-
InfluxDB
InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.
-
Project mention: The Backend Shift: Leveraging Open Source Powerhouses for Faster, Leaner Apps | dev.to | 2025-05-10
Dify (dify.ai):
-
I replaced it all with a self-hosted setup using Open WebUI, and it is now saving me thousands of dollars a year across sliplane.io and side projects.
-
GitHub: All-Hands-AI/OpenHands
-
ragflow
RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.
⭐️ RAG Flow on GitHub
-
-
-
Stream
Stream - Scalable APIs for Chat, Feeds, Moderation, & Video. Stream helps developers build engaging apps that scale to millions with performant and flexible Chat, Feeds, Moderation, and Video APIs and SDKs powered by a global edge network and enterprise-grade infrastructure.
-
MetaGPT
🌟 The Multi-Agent Framework: First AI Software Company, Towards Natural Language Programming
Project mention: Backlog.md – CLI that auto-generates task files (took my Claude success to 95 %) | news.ycombinator.com | 2025-07-06 -
Basically, LLaMA-Factory is just this totally awesome open-source thing by some great dev dude hiyouga. It's a one-stop-shop for fiddling with data from over 100 different LLMs and even VLMs (those are the ones which get visual). People love this thing. It doesn't surprise me. It takes some serious headache out of fiddling around.
-
-
anything-llm
The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, No-code agent builder, MCP compatibility, and more.
Ok, now to finally point AnythingLLM (my current favorite desktop LLM client) to the MCP server and tell the LLM to do something in the Pulsar editor! AnythingLLM keeps it's mcpServers config in this file on Mac:
-
Resource: LlamaIndex Documentation
-
firecrawl
🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.
Just a few days ago, Eric - CEO of Firecrawl - announced that they were closing down their previous startup, Mendable in this article and Hassan was promoted to the Director of Developer Relations in this post, both of whom post sample applications they build on a daily basis. These recent posts are testament to the prolific impact of sample applications on the adoption of Firecrawl and Together.ai.
-
unsloth
Fine-tuning & Reinforcement Learning for LLMs. 🦥 Train Qwen3, Llama 4, DeepSeek-R1, Gemma 3, TTS 2x faster with 70% less VRAM.
Project mention: When Fine-Tuning Makes Sense: A Developer's Guide | news.ycombinator.com | 2025-06-01Lot's of tools for each of those separately (RAG and fine-tuning). We're working on combining them but it's not ready yet.
You don't need a big GPU cluster. Fine-tuning is quite accessible via both APIs and local tools. Some suggestions:
- getkiln.ai (biased, my tool): let's you try all of the below, and compare/eval the resulting models
- API based tuning for closed models: OpenAI, Google Gemini
- API based tuning for open models: Together.ai, Fireworks.ai
- Local tuning for open models: https://unsloth.ai (can be run on Google Collab instances if you don't have local Nvidia GPUs).
-
quivr
Opiniated RAG for integrating GenAI in your apps 🧠 Focus on your product rather than the RAG. Easy integration in existing products with customisation! Any LLM: GPT4, Groq, Llama. Any Vectorstore: PGVector, Faiss. Any Files. Anyway you want.
-
chatgpt-on-wechat
基于大模型搭建的聊天机器人，同时支持 微信公众号、企业微信应用、飞书、钉钉 等接入，可选择ChatGPT/Claude/DeepSeek/文心一言/讯飞星火/通义千问/ Gemini/GLM-4/Kimi/LinkAI，能处理文本、语音和图片，访问操作系统和互联网，支持基于自有知识库进行定制企业智能客服。
-
-
Project mention: Show HN: How to make your MCP clients more context-aware | news.ycombinator.com | 2025-05-13
-
Milvus
Milvus is a high-performance, cloud-native vector database built for scalable vector ANN search
Project mention: What I Learned About Vector Databases When Building Semantic Search | dev.to | 2025-07-06Kubernetes Operators: Milvus and Zilliz Cloud Helm charts simplified provisioning. Weaviate required manual StatefulSets.
-
LocalAI
:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed, P2P inference
Project mention: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC | news.ycombinator.com | 2025-04-10If you're going to run Ollama in Windows anyway, why not use the native build? And if you want to use WSL, then I'd sugggest using something like LocalAI which gives you a lot more control and support for additional formats (GGML, GGUF, GPTQ, ONNX, etc).
https://github.com/mudler/LocalAI
-
khoj
Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (gpt, claude, gemini, llama, qwen, mistral). Get started - free.
GitHub https://github.com/khoj-ai/khoj GitHub Star 12.4k GitHub Fork 627 GitHub Issue 64 GitHub Pull Request 3 GitHub Contributor 35 Open Source License AGPL-3.0 Official Website https://khoj.dev/ Documentation https://docs.khoj.dev/
-
You can test the RagApi with CherryStudio, a visual tool that supports OpenAI-compatible APIs.
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
LLM discussion
LLM related posts
-
Repomix, a tool that packs your entire repository into a single AI-friendly file
-
LLM Inference Handbook
-
One Input, Multiple AI Minds: Meet the New MultiMindSDK LLM Router
-
Show HN: Code is all you need – Sherlog MCP
-
Agentmark
-
Build a Local AI RAG App with Ollama and Python
-
Building an MCP Server in Umbraco: My Journey into AI-Powered Content Management
-
A note from our sponsor - Stream
getstream.io | 11 Jul 2025
Index
What are some of the best open-source LLM projects? This list will help you:
| # | Project | Stars |
|---|---|---|
| 1 | ollama | 145,951 |
| 2 | dify | 106,042 |
| 3 | open-webui | 101,378 |
| 4 | OpenHands | 60,139 |
| 5 | ragflow | 59,172 |
| 6 | LLMs-from-scratch | 58,259 |
| 7 | llm-course | 57,181 |
| 8 | MetaGPT | 56,980 |
| 9 | LLaMA-Factory | 53,691 |
| 10 | vllm | 51,592 |
| 11 | anything-llm | 46,237 |
| 12 | llama_index | 42,912 |
| 13 | firecrawl | 42,673 |
| 14 | unsloth | 41,827 |
| 15 | quivr | 38,095 |
| 16 | chatgpt-on-wechat | 38,037 |
| 17 | ChatTTS | 37,081 |
| 18 | mem0 | 36,188 |
| 19 | Milvus | 35,892 |
| 20 | LocalAI | 33,768 |
| 21 | khoj | 30,523 |
| 22 | cherry-studio | 29,761 |
| 23 | Mr.-Ranedeer-AI-Tutor | 29,581 |
Sponsored
InfluxDB – Built for High-Performance Time Series Workloads
InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.
www.influxdata.com