-
tensorzero
Discontinued TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation.
There was a high severity advisory last week https://github.com/tensorzero/tensorzero/security/advisories... though these days can't even tell if this is related or just routine
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
plexus
Unified API gateway for multiple AI providers (OpenAI, Anthropic, Gemini, etc.). Switch models and providers without changing client code. Features OAuth authentication, quota tracking, embeddings, transcriptions, and OpenAI-compatible endpoints. (by mcowger)
Just use Plexus [1]. The maintainer is not trying to be a hero or raise seed dollars or even really trying to promote it. He's just making an excellent, useful product. (Unaffiliated, just a happy user). It's not a full-on "LLMOps" platform (whatever that is), it's just a proxy that works very well and has some nice features.
[1] https://github.com/mcowger/plexus
-
litellm
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
-
I've forked this and will be updating and fixing issues going forward.
https://github.com/agentify-sh/gateway
For now I will just open this up to see if there is any interest if so I would be spending whatever free time I get to fix issues and open it up to other contributors so we can keep going.
I think its important to have an LLM gateway tool like this to remain open source.
Related posts
-
Long-Text Summarization Explained: Cheap Node.js API Token Counts for SaaS
-
How to Benchmark a Unified LLM API: Node.js Invoice Extraction Under Latency Limits
-
Tenant Cost Attribution Explained — 3 Fallback Models Behind One Chatbot API
-
Per-Tenant Media Scheduling Explained: Node.js LLM JSON Schema for Ticket Tags
-
Speech-to-Text API 429 Triage: Backoff, Queue, and Batch Transcription