Top 23 Python llama Projects

llama

• Add a project

1. vllm

   1 91 82,489 10.0 Python

   A high-throughput and memory-efficient inference and serving engine for LLMs

   

   Project mention: Speculative decoding: when and why it actually speeds up inference | dev.to | 2026-06-04

   Here's a real, runnable config that uses EAGLE for offline batched generation. It's straight from the vLLM repo's eagle.md example:

2. SaaSHub

   www.saashub.com featured

   SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

   

3. LlamaFactory

   2 8 72,081 9.6 Python

   Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

   

   Project mention: Llama-Factory: Unified, Efficient Fine-Tuning for 100 Open LLMs | news.ycombinator.com | 2025-09-18

4. unsloth

   3 34 66,397 10.0 Python

   Unsloth Studio is a web UI for training and running open models like Gemma 4, Qwen3.6, DeepSeek, gpt-oss locally.

   

   Project mention: I Trained an LLM on 75K of My Own Messages So It Would Stop Writing Like a Chatbot | dev.to | 2026-05-08

   Training: unsloth + trl (SFTTrainer). Unsloth handles the 4-bit quantization and gradient checkpointing; trl handles the training loop.

5. aider

   4 157 46,112 9.4 Python

   aider is AI pair programming in your terminal

   

   Project mention: Microsoft pulled internal Claude Code licenses — what their engineers' preference tells you about picking AI tools | dev.to | 2026-06-06

   Aider with a bring-your-own API key has no subscription floor. Pay per token. For 2-3 focused hours per week, costs less than $20/mo. For full-time use, costs more than Max. Best for terminal natives who already live in tmux and do not want a separate IDE.

6. AstrBot

   5 2 34,454 9.9 Python

   AI Agent Assistant & development framework that integrates lots of IM platforms, LLMs, plugins and AI feature, and can be your openclaw alternative. ✨

   

7. fish-speech

   6 8 30,752 8.1 Python

   SOTA Open Source TTS

   

   Project mention: Fish Audio S2-Pro：用自然语言控制语音情感的 TTS 模型 | dev.to | 2026-03-11

   git clone https://github.com/fishaudio/fish-speech.git cd fish-speech pip install uv uv sync

8. sglang

   7 19 28,913 10.0 Python

   SGLang is a high-performance serving framework for large language models and multimodal models.

   

   Project mention: DeepSeek makes the V4 Pro price discount permanent | news.ycombinator.com | 2026-05-22

   There are several things at play:

   Inference stack efficiency: Many of these providers take off the shelf sglang / vllm / trtllm and hope for the best. Meanwhile DeepSeek team is known for pushing the boundary of optimizations.

   Now, sglang and vllm are great pieces of software, but take DeepSeek's Sparse Attention (DSA). Introduced 1.5 years ago (https://arxiv.org/abs/2512.02556), used by DeepSeek 3.2, GLM 5, DeepSeek V4. Only now is it slowly strating to get optimized in the major inference engines: (https://github.com/sgl-project/sglang/issues/19380 https://github.com/sgl-project/sglang/pull/22851 etc.). Of course, DS V4 adds extra optimizations into the model architecture on top of DSA, and those will take more time to be taken full advantage of by the open source inference engines.

   Privacy: Betting that people will pay extra for inference hosted outside China. This is especially true with DeepSeek, because DeepSeek is transparent about using API data for model improvements.

   And few other things (scale (matters a lot for MoEs), reliability, soft enterprise lock in, etc.)

   ---

   There is also, likely, tacit collusion at play here. Look at GLM 5 and GLM 5.1 prices. GLM 5 and 5.1 cost the same to run, but providers decided to charge much more for 5.1 because it is much better model, and because Z.AI raised their price as well.

9. LLaVA

   8 23 24,756 8.6 Python

   [NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

   

   Project mention: OpenAI Open Models | news.ycombinator.com | 2025-08-05

10. free-llm-api-resources

    9 3 23,375 8.7 Python

    A list of free LLM inference resources accessible via API.

    

    Project mention: 7 GitHub Repos I'd Actually Use to Find Free AI APIs and No-Cost AI Tools | dev.to | 2026-04-23

    View on GitHub

11. Chinese-LLaMA-Alpaca

    10 4 18,946 2.3 Python

    中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

    

12. ChuanhuChatGPT

    11 1 15,313 3.3 Python

    GUI for ChatGPT API and many LLMs. Supports agents, file-based QA, GPT finetuning and query with web search. All with a neat UI.

    

13. PaddleNLP

    12 2 12,950 9.6 Python

    Easy-to-use and powerful LLM and SLM library with awesome model zoo.

    

14. OpenLLM

    13 29 12,352 5.3 Python

    Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

    

    Project mention: Your 2025 Roadmap to Becoming an AI Engineer for Free for Vue.js Developers | dev.to | 2025-08-06

    REST APIs to connect AI models to Vue.js apps (example 1, example 2).

15. shell_gpt

    14 46 12,122 5.4 Python

    A command-line productivity tool powered by AI large language models like GPT-5, will help you accomplish your tasks faster and more efficiently.

    

    Project mention: Supercharge Your Terminal: ShellGPT + ChromaDB + LangChain for Context-Aware Automation | dev.to | 2025-09-01

    🗃 To explore ShellGPT in depth, including installation instructions, usage examples, and advanced configuration options, head over to the official ShellGPT GitHub repository.

16. ludwig

    15 3 11,716 9.6 Python

    Low-code framework for building custom LLMs, neural networks, and other AI models

    

17. petals

    16 104 10,171 7.2 Python

    🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

    

    Project mention: Open Source AI Must Win | news.ycombinator.com | 2026-06-12

    there are already projects like Petals https://github.com/bigscience-workshop/petals

18. inference

    17 2 9,348 9.7 Python

    Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

    

19. oumi

    18 5 9,315 9.7 Python

    Easily fine-tune, evaluate and deploy Gemma 4, Qwen3.5, Qwen3.6, gpt-oss, DeepSeek-R1, or any open source LLM / VLM!

    

    Project mention: Open source x 3: GRPO training with OpenEnv, vLLM, and Oumi | news.ycombinator.com | 2025-11-12

20. GPTCache

    19 45 8,066 2.4 Python

    Semantic cache for LLMs. Fully integrated with LangChain and llama_index.

    

    Project mention: What's semantic caching? | dev.to | 2026-03-16

    GPTCache — an open source library built specifically for caching LLM responses. Pretty flexible and worth looking at if you're rolling your own pipeline.

21. lmdeploy

    20 5 7,895 9.7 Python

    LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

    

22. mergekit

    21 6 7,137 7.5 Python

    Tools for merging pretrained large language models.

    

23. Liger-Kernel

    22 3 6,417 9.5 Python

    Efficient Triton Kernels for LLM Training

    

    Project mention: Why ML Needs a New Programming Language | news.ycombinator.com | 2025-09-05

24. YuE

    23 3 6,249 9.2 Python

    YuE: Open Full-song Music Generation Foundation Model, something similar to Suno.ai but open (by multimodal-art-projection)

    

Python llama related posts

• The Chomsky Objection the AI Industry Has Been Quietly Working Around

  4 projects | dev.to | 9 Jun 2026

• 5 Best Local LLM Tools and Models You Should Run in 2026

  5 projects | dev.to | 2 Jun 2026

• DeepSeek-V3: The 671B MoE Model You Can Run Locally in 2026

  1 project | dev.to | 20 May 2026

• Agents 101: Reasoning, Actions & Autonomy

  4 projects | dev.to | 11 May 2026

• I Trained an LLM on 75K of My Own Messages So It Would Stop Writing Like a Chatbot

  1 project | dev.to | 8 May 2026

• Meltdown: LLM Client Made in Python and Tk

  1 project | news.ycombinator.com | 8 May 2026

• Accelerating Gemma 4: faster inference with multi-token prediction drafters

  7 projects | news.ycombinator.com | 5 May 2026
• A note from our sponsor - SaaSHub
  www.saashub.com | 13 Jun 2026

  SaaSHub helps you find the best software and product alternatives Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com

Do not miss the trending Python projects with our weekly report!