SaaSHub helps you find the best software and product alternatives Learn more →
Top 23 Python llmops Projects
-
No, vLLM is a thing for serving language models: https://github.com/vllm-project/vllm
-
InfluxDB
InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.
-
autogen
A programming framework for agentic AI 🤖 PyPi: autogen-agentchat Discord: https://aka.ms/autogen-discord Office Hour: https://aka.ms/autogen-officehour
Project mention: 🧠 AI Agents Are the New DevOps: How Autonomous Agents Are Redefining Software Engineering | dev.to | 2025-05-23Microsoft AutoGen – Multi-agent framework for building collaborative AI agents.
-
composio
Composio equips your AI agents & LLMs with 100+ high-quality integrations via function calling
Star the Composio Repo⭐
-
litellm
Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
Substitute your own embeddings database to change the knowledge base. txtai supports running local LLMs via transformers or llama.cpp. It also supports a wide variety of LLMs via LiteLLM. For example, setting the 2nd RAG pipeline parameter below to gpt-4o along with the appropriate environment variables with access keys switches to a hosted LLM. See this documentation page for more on this.
-
-
SuperAGI
<⚡️> SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.
AI agents are, at the most basic level, software. With all software comes the risk of security vulnerabilities. During this research, we looked closely at some popular agent tooling, including LangChain (CVE-2024-21513) and SuperAGI (CVE-2024-21552), with a particular focus on traditional software vulnerabilities that have been around for decades and will continue to be around in the future.
-
OpenLLM
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
Project mention: Fast and secure ingress to remote AIs with ngrok, Deepseek, and Ollama | dev.to | 2025-02-13Hosted: Hosted platforms work like a SaaS—they launch your LLM on their infrastructure, and you get a simple web app or API to explore. There are plenty of hosted AI/LLM platforms already, like RunPod, Mystic, BentoML, and others. Major cloud providers also have platforms for training and deploying LLMs, like Google’s Vertex AI or AWS’ AI Services. Hosted platforms win out on simplicity but don’t come with privacy and compliance guarantees, and trend on the expensive side.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
Project mention: Open Source Frameworks for Building Generative AI Applications | dev.to | 2024-10-02
RAGAS
-
Project mention: Show HN: Flow – A Dynamic Task Engine for AI Agents Without DAG | news.ycombinator.com | 2024-12-02
Interesting! I feel like this is a cross between https://github.com/dagworks-inc/burr (switch state for context) and https://github.com/Netflix/metaflow because the output of the "task" declares its next hop...
-
BentoML
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
Project mention: My personal favorite MCP server which has became part of my life | dev.to | 2025-05-27 -
Traceloop is also an open-source project that provides end-to-end tracing for LLM applications. It uses OpenTelemetry standards to offer visibility into the request flow through code, especially in agent-based and multi-step workflows. Traceloop focuses exclusively on tracing and requires an existing OpenTelemetry setup to unlock its full potential.
-
Project mention: Build fully portable AI applications on top of Snowflake with SuperDuperDB | dev.to | 2024-06-26
Customize how AI and databases work together. Scale your AI projects to handle more data and users. Move AI projects between different environments easily. Extend the system with new AI features and database functionality. Check it out: Blog: https://blog.superduperdb.com/version-02 Github: https://github.com/SuperDuperDB/superduperdb (leave us a star ⭐️🥳)
-
Seamless integration: Works with OCI-compliant registries (e.g., Docker Hub and Jozu Hub) and integrates with popular tools like HuggingFace, ZenML, and Git.
-
When originally designing the chatbot, we opted to build it in Python, despite being a heavily JavaScript-oriented shop. This decision was driven by the availability of more mature analytic tools for objectively testing chatbot hallucination and accuracy in Python. So far, we've been evaluating answers qualitatively, but we plan to incorporate a tool like Giskard to bring a more quantitative approach to our evaluations. This step is crucial and one that, anecdotally, is often overlooked in many production chatbots.
-
cognita
RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry
Project mention: Lists of open-source frameworks for building RAG applications | dev.to | 2025-01-02Ideal For: Enterprises seeking a robust framework for large-scale AI applications. GitHub Repository
-
AGiXT
AGiXT is a dynamic AI Agent Automation Platform that seamlessly orchestrates instruction management and complex task execution across diverse AI providers. Combining adaptive memory, smart features, and a versatile plugin system, AGiXT delivers efficient and comprehensive AI solutions.
-
-
agenta
The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.
Project mention: Compare All LLMs in one Playground: Introducing Agenta's AI Model Hub | dev.to | 2025-04-15Agenta is an open-source platform built for AI engineers and teams working with large language models. Our playground, evaluation tools, and deployment solutions help streamline the entire LLM application development process. As an open-source tool, Agenta gives developers complete control over their AI infrastructure while providing enterprise-grade features for teams of all sizes.
-
-
uptrain
UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.
Project mention: UpTrain is an open-source unified platform to evaluate and improve Gen AI apps | news.ycombinator.com | 2024-09-10 -
LLMStack
No-code multi-agent framework to build LLM Agents, workflows and applications with your data
Project mention: Show HN: Langrocks – tools like computer access, browser etc., for LLM agents | news.ycombinator.com | 2024-11-20We built tools like web browser, code interpreter etc., needed for LLM agents as part of our LLMStack project. We have now moved them into a single collection as langrocks. We're using this in LLMStack and thought others might find it useful. https://github.com/trypromptly/LLMStack/blob/main/llmstack/p... shows how we use langrocks with Anthropic's Claude with computer use to automate web browsing.
The coolest part is watching an LLM actually use a computer - you get a unique URL to view the virtual display, so you can see exactly what it's doing with tools like computer access and web browser. We've used this to automate complex workflows where the LLM needs to research across multiple sites, interact with web apps, or perform system operations.
-
dstack
dstack is an open-source alternative to Kubernetes and Slurm, designed to simplify GPU allocation and AI workload orchestration for ML teams across top clouds, on-prem clusters, and accelerators.
Project mention: Orchestrating GPUs in data centers and private clouds | news.ycombinator.com | 2025-02-18Super excited to hear any feedback.
[1] https://github.com/dstackai/dstack/issues/2184
-
Project mention: Securing and enhancing LLM prompts & outputs: A guide using Amazon Bedrock and open-source solutions | dev.to | 2024-10-01
Image source:https://llm-guard.com/
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Python llmops discussion
Python llmops related posts
-
Nano-Vllm: Nano VLLM
-
Code Review: Deep Dive into vLLM's Architecture and Implementation Analysis of OpenAI-Compatible Serving (1/2)
-
Your Desktop AI: Seamlessly Running Granite with LiteLLM and Ollama
-
Introdução ao LangGraph: Orquestrando agentes com fluxos baseados em grafos
-
Top Open Source Tools for LLM Observability in 2025
-
Compare All LLMs in one Playground: Introducing Agenta's AI Model Hub
-
DeepSeek Open Source FlashMLA – MLA Decoding Kernel for Hopper GPUs
-
A note from our sponsor - SaaSHub
www.saashub.com | 24 Jun 2025
Index
What are some of the best open-source llmops projects in Python? This list will help you:
| # | Project | Stars |
|---|---|---|
| 1 | vllm | 50,438 |
| 2 | autogen | 46,103 |
| 3 | composio | 25,529 |
| 4 | litellm | 24,275 |
| 5 | serve | 21,621 |
| 6 | SuperAGI | 16,451 |
| 7 | OpenLLM | 11,424 |
| 8 | ragas | 9,634 |
| 9 | metaflow | 8,897 |
| 10 | BentoML | 7,801 |
| 11 | openllmetry | 5,971 |
| 12 | superduper | 5,085 |
| 13 | zenml | 4,638 |
| 14 | giskard | 4,639 |
| 15 | cognita | 4,114 |
| 16 | AGiXT | 3,031 |
| 17 | lorax | 3,023 |
| 18 | agenta | 2,843 |
| 19 | trulens | 2,571 |
| 20 | uptrain | 2,278 |
| 21 | LLMStack | 1,967 |
| 22 | dstack | 1,803 |
| 23 | llm-guard | 1,768 |
Sponsored
InfluxDB – Built for High-Performance Time Series Workloads
InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.
www.influxdata.com