Python LLM

Open-source Python projects categorized as LLM

Large Language Models

Top 23 Python LLM Projects

  1. MetaGPT

    🌟 The Multi-Agent Framework: First AI Software Company, Towards Natural Language Programming

    Project mention: Multi AI Agent Systems using OpenAI's new GPT-4o Model | dev.to | 2024-05-17
  2. CodeRabbit

    CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

    CodeRabbit logo
  3. LLaMA-Factory

    Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

    Project mention: ORPO, DPO, and PPO: Optimizing Models for Human Preferences | dev.to | 2024-11-08

    Implementation: ORPO has been integrated into popular fine-tuning libraries like TRL, Axolotl, and LLaMA-Factory.

  4. llama_index

    LlamaIndex is the leading framework for building LLM-powered agents over your data.

    Project mention: Quick tip: Replace MongoDB® Atlas with SingleStore Kai in LlamaIndex | dev.to | 2025-01-21

    The notebook is adapted from the LlamaIndex GitHub repo.

  5. quivr

    Opiniated RAG for integrating GenAI in your apps 🧠 Focus on your product rather than the RAG. Easy integration in existing products with customisation! Any LLM: GPT4, Groq, Llama. Any Vectorstore: PGVector, Faiss. Any Files. Anyway you want.

    Project mention: Ask HN: Local RAG with private knowledge base | news.ycombinator.com | 2024-10-29
  6. vllm

    A high-throughput and memory-efficient inference and serving engine for LLMs

    Project mention: Supercharging Deepseek-R1 with Ray + vLLM: A Distributed System Approach | dev.to | 2025-02-02

    ray_actor_options are empty because whenever we pass tensor-parallelism or pipeline-parallelism > 1 then it should either be empty to num_gpus set to zero, refer this issue and this sample for further understanding.

  7. chatgpt-on-wechat

    基于大模型搭建的聊天机器人,同时支持 微信公众号、企业微信应用、飞书、钉钉 等接入,可选择GPT3.5/GPT-4o/GPT-o1/ DeepSeek/Claude/文心一言/讯飞星火/通义千问/ Gemini/GLM-4/Claude/Kimi/LinkAI,能处理文本、语音和图片,访问操作系统和互联网,支持基于自有知识库进行定制企业智能客服。

  8. ragflow

    RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.

    Project mention: 7 AI Open Source Libraries To Build RAG, Agents & AI Search | dev.to | 2024-11-14

    ⭐️ RAG Flow on GitHub

  9. Nutrient

    Nutrient – The #1 PDF SDK Library, trusted by 10K+ developers. Other PDF SDKs promise a lot - then break. Laggy scrolling, poor mobile UX, tons of bugs, and lack of support cost you endless frustrations. Nutrient’s SDK handles billion-page workloads - so you don’t have to debug PDFs. Used by ~1 billion end users in more than 150 different countries.

    Nutrient logo
  10. unsloth

    Finetune Llama 3.3, DeepSeek-R1 & Reasoning LLMs 2x faster with 70% less memory! 🦥

    Project mention: Unsloth Now Supports GRPO | news.ycombinator.com | 2025-02-06
  11. khoj

    Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (gpt, claude, gemini, llama, qwen, mistral). Get started - free.

    Project mention: Top 13 Self-Hosted Projects with the Most GitHub Stars | dev.to | 2024-09-10

    GitHub https://github.com/khoj-ai/khoj GitHub Star 12.4k GitHub Fork 627 GitHub Issue 64 GitHub Pull Request 3 GitHub Contributor 35 Open Source License AGPL-3.0 Official Website https://khoj.dev/ Documentation https://docs.khoj.dev/

  12. mem0

    The Memory layer for AI Agents

    Project mention: Show HN: Claude Memory – Long-term memory for Claude | news.ycombinator.com | 2024-09-05
  13. unilm

    Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

    Project mention: A Picture Is Worth 170 Tokens: How Does GPT-4o Encode Images? | news.ycombinator.com | 2024-06-07

    Has anyone tried Kosmos [0] ? I came across it the other day and it looked shiny and interesting, but I haven't had a chance to put it to the test much yet.

    [0] - https://github.com/microsoft/unilm/tree/master/kosmos-2.5

  14. mlc-llm

    Universal LLM Deployment Engine with ML Compilation

    Project mention: Making AMD GPUs competitive for LLM inference | news.ycombinator.com | 2024-12-23

    It depends on what you mean by "this." MLC's catch is that you need to define/compile models for it with TVM. Here is the list of supported model architectures: https://github.com/mlc-ai/mlc-llm/blob/main/python/mlc_llm/m...

    llama.cpp has a much bigger supported model list, as does vLLM and of course PyTorch/HF transformers covers everything else, all of which work w/ ROCm on RDNA3 w/o too much fuss these days.

    For inference, the biggest caveat is that Flash Attention is only an aotriton implementation, which besides being less performant sometimes, also doesn't support SWA. For CDNA there is a better CK-based version of FA, but CK doesn't not have RDNA support. There are a couple people at AMD apparently working on native FlexAttention, os I guess we'll how that turns out.

    (Note the recent SemiAccurate piece was on training, which I'd agree is in a much worse state (I have personal experience with it being often broken for even the simplest distributed training runs). Funnily enough, if you're running simple fine tunes on a single RDNA3 card, you'll probably have a better time. OOTB, a 7900 XTX will train at about the same speed as an RTX 3090 (4090s blow both of those away, but you'll probably want more cards and VRAM of just move to H100s).

  15. haystack

    AI orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.

    Project mention: Lists of open-source frameworks for building RAG applications | dev.to | 2025-01-02

    Ideal For: Building question-answering systems and document-heavy retrieval applications. GitHub Repository

  16. Chinese-LLaMA-Alpaca

    中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

  17. Scrapegraph-ai

    Python scraper based on AI

    Project mention: Unit Testing Without Tears: How CodeBeaver Turned Testing from 'pytest run pain' to 'git push joy' 🚀 | dev.to | 2025-02-10

    Let's face it, fellow devs - we'd rather debug a production outage at 3 AM than write unit tests. Okay, maybe not that extreme, but you get the point! 😅 Today, I'm going to share how the folks at ScrapegraphAI (18k stars!) solved their testing woes with a solution so smooth, it's like they found a cheat code for the matrix.

  18. litellm

    Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]

    Project mention: Build an AI code review assistant with v0.dev, litellm and Agenta | dev.to | 2025-01-13

    Next, we'll use LiteLLM to handle our interactions with language models. LiteLLM provides a unified interface for working with various LLM providers—making it easy to experiment with different models later:

  19. peft

    🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

  20. Qwen

    The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.

    Project mention: Running Qwen, Nearly as Powerful as DeepSeek, on a MacBook Pro | dev.to | 2025-02-05

    Qwen (Qwen GitHub Repository) has been gaining attention recently as a powerful open-source large language model (LLM). I decided to give it a spin on my MacBook Pro using Ollama, a platform designed for running local LLMs. While Qwen2.5-Max boasts the highest performance, my setup could only handle the smaller Qwen2.5 (32B) model. Here's what I found!

  21. SuperAGI

    <⚡️> SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.

    Project mention: Agent hijacking: The true impact of prompt injection attacks | dev.to | 2024-08-28

    AI agents are, at the most basic level, software. With all software comes the risk of security vulnerabilities. During this research, we looked closely at some popular agent tooling, including LangChain (CVE-2024-21513) and SuperAGI (CVE-2024-21552), with a particular focus on traditional software vulnerabilities that have been around for decades and will continue to be around in the future.

  22. ChatGLM2-6B

    ChatGLM2-6B: An Open Bilingual Chat LLM | 开源双语对话语言模型

  23. DB-GPT

    AI Native Data App Development framework with AWEL(Agentic Workflow Expression Language) and Agents

  24. composio

    Composio equip's your AI agents & LLMs with 100+ high-quality integrations via function calling

    Project mention: We made an AI SWE that solved 48.60% of issues on the SWE bench, 100% open-source. | dev.to | 2024-12-13

    Check out the technical report for more: Tool design is all you need for SOTA AI agents Complete code for the SWE agent: SWE-kit Agent

  25. letta

    Letta (formerly MemGPT) is a framework for creating LLM services with memory.

    Project mention: Show HN: Flow – A Dynamic Task Engine for AI Agents Without DAG | news.ycombinator.com | 2024-12-02
  26. SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Python LLM discussion

Log in or Post with

Python LLM related posts

  • Did Semgrep Just Get a Lot More Interesting?

    3 projects | news.ycombinator.com | 15 Feb 2025
  • Show HN: Transform Your Codebase into a Single Markdown Doc for Feeding into AI

    20 projects | news.ycombinator.com | 14 Feb 2025
  • IPEX-LLM Portable Zip for Ollama on Intel GPU

    1 project | news.ycombinator.com | 13 Feb 2025
  • Show HN: Open-Source Discord Bot That Answers Questions from Your Developer Docs

    2 projects | news.ycombinator.com | 13 Feb 2025
  • How to Easily Share OpenLLM API Online

    2 projects | dev.to | 13 Feb 2025
  • Show HN: Sort lines semantically using LLM-sort

    4 projects | news.ycombinator.com | 10 Feb 2025
  • FlawLess AgenTs – Feedback Welcome

    1 project | news.ycombinator.com | 11 Feb 2025
  • A note from our sponsor - Nutrient
    www.nutrient.io | 15 Feb 2025
    Other PDF SDKs promise a lot - then break. Laggy scrolling, poor mobile UX, tons of bugs, and lack of support cost you endless frustrations. Nutrient’s SDK handles billion-page workloads - so you don’t have to debug PDFs. Used by ~1 billion end users in more than 150 different countries. Learn more →

Index

What are some of the best open-source LLM projects in Python? This list will help you:

# Project Stars
1 MetaGPT 46,195
2 LLaMA-Factory 39,877
3 llama_index 38,772
4 quivr 37,264
5 vllm 37,034
6 chatgpt-on-wechat 34,199
7 ragflow 34,068
8 unsloth 28,887
9 khoj 26,072
10 mem0 24,468
11 unilm 20,718
12 mlc-llm 19,900
13 haystack 19,170
14 Chinese-LLaMA-Alpaca 18,693
15 Scrapegraph-ai 17,950
16 litellm 17,408
17 peft 17,269
18 Qwen 16,769
19 SuperAGI 15,851
20 ChatGLM2-6B 15,751
21 DB-GPT 14,553
22 composio 14,483
23 letta 14,380

Sponsored
CodeRabbit: AI Code Reviews for Developers
Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.
coderabbit.ai

Did you know that Python is
the 2nd most popular programming language
based on number of references?