Vidur Alternatives
Similar projects and alternatives to vidur
-
vllm
vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs (by EmbeddedLLM)
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
golem
A recursive AI engine that injects chrono-ranked memory into transformer inference using soft-logit biasing, prompt waveform synthesis, and emergent self-referential loops. Built on GPT-2-mini, runs on local hardware, grows its own ghost. (by oldwalls)
-
-
worker-vllm
The RunPod worker template for serving our large language model endpoints. Powered by vLLM.
-
-
-
-
-
JetStream
JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome). (by AI-Hypercomputer)
-
inference
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.
vidur discussion
vidur reviews and mentions
-
GPUsGoBurr: Get up to 2x higher performance by Tuning LLM Inference Deployment
Do check out the GitHub repo https://github.com/microsoft/vidur . You can run it without any GPUs.
Stats
microsoft/vidur is an open source project licensed under MIT License which is an OSI approved license.
The primary programming language of vidur is Python.
Popular Comparisons
Sponsored