Other PDF SDKs promise a lot - then break. Laggy scrolling, poor mobile UX, tons of bugs, and lack of support cost you endless frustrations. Nutrient’s SDK handles billion-page workloads - so you don’t have to debug PDFs. Used by ~1 billion end users in more than 150 different countries. Learn more →
Top 23 Python LLM Projects
-
MetaGPT
🌟 The Multi-Agent Framework: First AI Software Company, Towards Natural Language Programming
-
CodeRabbit
CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.
-
Implementation: ORPO has been integrated into popular fine-tuning libraries like TRL, Axolotl, and LLaMA-Factory.
-
Project mention: Quick tip: Replace MongoDB® Atlas with SingleStore Kai in LlamaIndex | dev.to | 2025-01-21
The notebook is adapted from the LlamaIndex GitHub repo.
-
quivr
Opiniated RAG for integrating GenAI in your apps 🧠 Focus on your product rather than the RAG. Easy integration in existing products with customisation! Any LLM: GPT4, Groq, Llama. Any Vectorstore: PGVector, Faiss. Any Files. Anyway you want.
-
Project mention: Supercharging Deepseek-R1 with Ray + vLLM: A Distributed System Approach | dev.to | 2025-02-02
ray_actor_options are empty because whenever we pass tensor-parallelism or pipeline-parallelism > 1 then it should either be empty to num_gpus set to zero, refer this issue and this sample for further understanding.
-
chatgpt-on-wechat
基于大模型搭建的聊天机器人,同时支持 微信公众号、企业微信应用、飞书、钉钉 等接入,可选择GPT3.5/GPT-4o/GPT-o1/ DeepSeek/Claude/文心一言/讯飞星火/通义千问/ Gemini/GLM-4/Claude/Kimi/LinkAI,能处理文本、语音和图片,访问操作系统和互联网,支持基于自有知识库进行定制企业智能客服。
-
ragflow
RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.
⭐️ RAG Flow on GitHub
-
Nutrient
Nutrient – The #1 PDF SDK Library, trusted by 10K+ developers. Other PDF SDKs promise a lot - then break. Laggy scrolling, poor mobile UX, tons of bugs, and lack of support cost you endless frustrations. Nutrient’s SDK handles billion-page workloads - so you don’t have to debug PDFs. Used by ~1 billion end users in more than 150 different countries.
-
-
khoj
Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (gpt, claude, gemini, llama, qwen, mistral). Get started - free.
GitHub https://github.com/khoj-ai/khoj GitHub Star 12.4k GitHub Fork 627 GitHub Issue 64 GitHub Pull Request 3 GitHub Contributor 35 Open Source License AGPL-3.0 Official Website https://khoj.dev/ Documentation https://docs.khoj.dev/
-
Project mention: Show HN: Claude Memory – Long-term memory for Claude | news.ycombinator.com | 2024-09-05
-
Project mention: A Picture Is Worth 170 Tokens: How Does GPT-4o Encode Images? | news.ycombinator.com | 2024-06-07
Has anyone tried Kosmos [0] ? I came across it the other day and it looked shiny and interesting, but I haven't had a chance to put it to the test much yet.
[0] - https://github.com/microsoft/unilm/tree/master/kosmos-2.5
-
It depends on what you mean by "this." MLC's catch is that you need to define/compile models for it with TVM. Here is the list of supported model architectures: https://github.com/mlc-ai/mlc-llm/blob/main/python/mlc_llm/m...
llama.cpp has a much bigger supported model list, as does vLLM and of course PyTorch/HF transformers covers everything else, all of which work w/ ROCm on RDNA3 w/o too much fuss these days.
For inference, the biggest caveat is that Flash Attention is only an aotriton implementation, which besides being less performant sometimes, also doesn't support SWA. For CDNA there is a better CK-based version of FA, but CK doesn't not have RDNA support. There are a couple people at AMD apparently working on native FlexAttention, os I guess we'll how that turns out.
(Note the recent SemiAccurate piece was on training, which I'd agree is in a much worse state (I have personal experience with it being often broken for even the simplest distributed training runs). Funnily enough, if you're running simple fine tunes on a single RDNA3 card, you'll probably have a better time. OOTB, a 7900 XTX will train at about the same speed as an RTX 3090 (4090s blow both of those away, but you'll probably want more cards and VRAM of just move to H100s).
-
haystack
AI orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.
Project mention: Lists of open-source frameworks for building RAG applications | dev.to | 2025-01-02Ideal For: Building question-answering systems and document-heavy retrieval applications. GitHub Repository
-
-
Project mention: Unit Testing Without Tears: How CodeBeaver Turned Testing from 'pytest run pain' to 'git push joy' 🚀 | dev.to | 2025-02-10
Let's face it, fellow devs - we'd rather debug a production outage at 3 AM than write unit tests. Okay, maybe not that extreme, but you get the point! 😅 Today, I'm going to share how the folks at ScrapegraphAI (18k stars!) solved their testing woes with a solution so smooth, it's like they found a cheat code for the matrix.
-
litellm
Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
Project mention: Build an AI code review assistant with v0.dev, litellm and Agenta | dev.to | 2025-01-13Next, we'll use LiteLLM to handle our interactions with language models. LiteLLM provides a unified interface for working with various LLM providers—making it easy to experiment with different models later:
-
-
Qwen
The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.
Project mention: Running Qwen, Nearly as Powerful as DeepSeek, on a MacBook Pro | dev.to | 2025-02-05Qwen (Qwen GitHub Repository) has been gaining attention recently as a powerful open-source large language model (LLM). I decided to give it a spin on my MacBook Pro using Ollama, a platform designed for running local LLMs. While Qwen2.5-Max boasts the highest performance, my setup could only handle the smaller Qwen2.5 (32B) model. Here's what I found!
-
SuperAGI
<⚡️> SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.
AI agents are, at the most basic level, software. With all software comes the risk of security vulnerabilities. During this research, we looked closely at some popular agent tooling, including LangChain (CVE-2024-21513) and SuperAGI (CVE-2024-21552), with a particular focus on traditional software vulnerabilities that have been around for decades and will continue to be around in the future.
-
-
DB-GPT
AI Native Data App Development framework with AWEL(Agentic Workflow Expression Language) and Agents
-
composio
Composio equip's your AI agents & LLMs with 100+ high-quality integrations via function calling
Project mention: We made an AI SWE that solved 48.60% of issues on the SWE bench, 100% open-source. | dev.to | 2024-12-13Check out the technical report for more: Tool design is all you need for SOTA AI agents Complete code for the SWE agent: SWE-kit Agent
-
Project mention: Show HN: Flow – A Dynamic Task Engine for AI Agents Without DAG | news.ycombinator.com | 2024-12-02
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Python LLM discussion
Python LLM related posts
-
Did Semgrep Just Get a Lot More Interesting?
-
Show HN: Transform Your Codebase into a Single Markdown Doc for Feeding into AI
-
IPEX-LLM Portable Zip for Ollama on Intel GPU
-
Show HN: Open-Source Discord Bot That Answers Questions from Your Developer Docs
-
How to Easily Share OpenLLM API Online
-
Show HN: Sort lines semantically using LLM-sort
-
FlawLess AgenTs – Feedback Welcome
-
A note from our sponsor - Nutrient
www.nutrient.io | 15 Feb 2025
Index
What are some of the best open-source LLM projects in Python? This list will help you:
# | Project | Stars |
---|---|---|
1 | MetaGPT | 46,195 |
2 | LLaMA-Factory | 39,877 |
3 | llama_index | 38,772 |
4 | quivr | 37,264 |
5 | vllm | 37,034 |
6 | chatgpt-on-wechat | 34,199 |
7 | ragflow | 34,068 |
8 | unsloth | 28,887 |
9 | khoj | 26,072 |
10 | mem0 | 24,468 |
11 | unilm | 20,718 |
12 | mlc-llm | 19,900 |
13 | haystack | 19,170 |
14 | Chinese-LLaMA-Alpaca | 18,693 |
15 | Scrapegraph-ai | 17,950 |
16 | litellm | 17,408 |
17 | peft | 17,269 |
18 | Qwen | 16,769 |
19 | SuperAGI | 15,851 |
20 | ChatGLM2-6B | 15,751 |
21 | DB-GPT | 14,553 |
22 | composio | 14,483 |
23 | letta | 14,380 |