Top 23 Python llmops Projects

llmops

• Add a project

1. litellm

   1 123 58,121 10.0 Python

   The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

   

   Project mention: CISA Adds Seven Known Exploited Vulnerabilities to Catalog: AI Infrastructure, Python Web Frameworks, SonicWall, VoIP, and Artifactory | dev.to | 2026-09-03

   Related Sources: CISA KEV catalog, LiteLLM GHSA-7488-6r32-c95q, Starlette GHSA-86qp-5c8j-p5mr, Kestra GHSA-5vc5-wxxq-3fjx, Horizon3.ai Switchvox analysis, SonicWall SNWLID-2026-0016

2. Kargo

   akuity.io sponsored

   Stop Scripting Promotions. Start Shipping with Kargo. Kargo automates promotion across dev, staging, and prod with approval gates and verification. Open source, built by the team behind Argo CD. Download now.

   

3. MLflow

   2 85 27,795 10.0 Python

   The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

   

   Project mention: MLflow CVE-2026-64849: Cloud Credential Theft via Webhook SSRF | dev.to | 2026-08-20

   Original Source: MLflow Security Advisory / CISA

4. serve

   3 126 21,858 8.8 Python

   ☁️ Build multimodal AI applications with cloud-native stack

   

5. opik

   4 21 21,797 10.0 Python

   Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

   

   Project mention: AcruxCore alternatives - langfuse, phoenix-on-azure, phoenix, opik, and MLflow | libhunt.com/r/AcruxCore | 2026-08-14

6. SuperAGI

   5 83 17,629 4.2 Python

   <⚡️> SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.

   

7. ragas

   6 14 15,621 9.6 Python

   Supercharge Your LLM Application Evaluations 🚀 (by vibrantlabsai)

   

   Project mention: From 62% to 94% RAG Accuracy: The 5 Architecture Changes That Actually Moved the Needle | dev.to | 2026-04-26

   Reference answers written by domain experts. Automated grading via RAGAS across four dimensions: faithfulness, answer relevance, context precision, context recall. Manual spot-check on the ~15% of answers that fell in scoring edge cases.

8. OpenLLM

   7 29 12,524 2.4 Python

   Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

   

9. AppSignal

   www.appsignal.com sponsored

   AppSignal knows why the f*#k it crashed. Stop vibe-debugging. Every exception, every backtrace, grouped so you see patterns, not noise.

   

10. phoenix

    8 10 11,318 10.0 Python

    AI Observability & Evaluation (by Arize-ai)

    

    Project mention: AcruxCore alternatives - langfuse, phoenix-on-azure, phoenix, opik, and MLflow | libhunt.com/r/AcruxCore | 2026-08-14

11. metaflow

    9 30 10,255 9.3 Python

    Build, Manage and Deploy AI/ML Systems

    

12. BentoML

    10 18 8,821 8.3 Python

    The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

    

13. openllmetry

    11 10 7,413 9.2 Python

    Open-source observability for your GenAI or LLM application, based on OpenTelemetry

    

    Project mention: Semantic Observability: Engineering Reliability for Production RAG | dev.to | 2026-07-01

    Injecting these into standard logs is a recipe for storage-cost disasters and PII leaks. Instead, we use OpenLLMetry, an open-source extension of OTel that introduces semantic conventions for AI. It auto-instruments calls to providers like OpenAI or Anthropic, capturing the input and output as attributes within a span.

14. agent-starter-pack

    12 8 6,552 9.4 Python

    Ship AI Agents to Google Cloud in minutes, not months. Production-ready templates with built-in CI/CD, evaluation, and observability.

    

    Project mention: Building "Sweets Vault" - a multimodal Gemini Agent with physical hardware integration | dev.to | 2026-05-15

    To kick-start the agent development, I leveraged the agent-starter-pack templates. It provides a production-ready foundation with FastAPI, frontend UI integration, and built-in observability.

15. giskard-oss

    13 9 5,802 9.7 Python

    🐢 Open-Source Evaluation & Testing library for LLM Agents

    

16. zenml

    14 35 5,575 9.8 Python

    ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.

    

17. superduper

    15 25 5,317 9.6 Python

    Superduper: End-to-end framework for building custom AI applications and agents.

    

18. Soup

    16 3 5,759 9.9 Python

    Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU. (by MakazhanAlpamys)

    

    Project mention: Show HN: Fine-tune an 8B model on a 4 GB laptop GPU | news.ycombinator.com | 2026-08-04

    There are some samples of training data in this folder - https://github.com/MakazhanAlpamys/Soup/tree/main/examples/d...

    They're all very short though. Anyone got a good rule for how much data of this nature is needed to successfully fine-tune a model of this size?

19. lorax

    17 4 3,829 2.9 Python

    Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

    

20. trulens

    18 20 3,536 9.3 Python

    Evaluation and Tracking for LLM Experiments and AI Agents

    

    Project mention: All Data and AI Weekly #224-12 Jan 2026 | dev.to | 2026-01-12

    TruLens: Evaluating Agent Workflows

21. AGiXT

    19 26 3,214 9.8 Python

    AGiXT is a dynamic AI Agent Automation Platform that seamlessly orchestrates instruction management and complex task execution across diverse AI providers. Combining adaptive memory, smart features, and a versatile plugin system, AGiXT delivers efficient and comprehensive AI solutions.

    

22. burr

    20 12 2,539 9.1 Python

    Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastructure.

    

    Project mention: Apache Burr: Build reliable AI agents and applications | news.ycombinator.com | 2026-06-10

    Right -- possible it's slightly out of date https://github.com/apache/burr#-comparison-against-common-fr.... Good point on differentiating.

23. uptrain

    21 37 2,357 9.3 Python

    UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.

    

24. LLMStack

    22 23 2,311 9.9 Python

    No-code multi-agent framework to build LLM Agents, workflows and applications with your data

    

25. intellagent

    23 2 1,257 4.5 Python

    A framework for comprehensive diagnosis and optimization of agents using simulated, realistic synthetic interactions

    

26. SaaSHub

    www.saashub.com sponsored

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    

Python llmops related posts

• 3 Candidate Content Policy Checks Before Review — Why LLM Moderation False Positives Happen

  1 project | dev.to | 23 Aug 2026

• Text-to-JSON Extraction Explained: Long-Document Timeouts, Token Limits, and Chunking

  1 project | dev.to | 20 Aug 2026

• A Quality Gate for Node.js SaaS Text Summarization Chat APIs

  1 project | dev.to | 20 Aug 2026

• Long-Text Summarization Explained: Cheap Node.js API Token Counts for SaaS

  1 project | dev.to | 18 Aug 2026

• How to Benchmark a Unified LLM API: Node.js Invoice Extraction Under Latency Limits

  1 project | dev.to | 18 Aug 2026

• Your agent ignored a failed tool call. Here's how to catch that in CI.

  2 projects | dev.to | 17 Aug 2026

• AcruxCore alternatives - langfuse, phoenix-on-azure, phoenix, opik, and MLflow

  6 projects | 14 Aug 2026
• A note from our sponsor - AppSignal
  www.appsignal.com | 8 Sep 2026

  Stop vibe-debugging. Every exception, every backtrace, grouped so you see patterns, not noise. Learn more →

Index

Sponsored

Stop Scripting Promotions. Start Shipping with Kargo

Kargo automates promotion across dev, staging, and prod with approval gates and verification. Open source, built by the team behind Argo CD. Download now.

akuity.io

Do not miss the trending Python projects with our weekly report!