budgetml VS tritony

What are some alternatives?

pinferencia - Python + Inference - Model Deployment library in Python. Simplest model inference server ever.

vllm - A high-throughput and memory-efficient inference and serving engine for LLMs

zenml - ZenML 🙏: Build portable, production-ready MLOps pipelines. https://zenml.io.

DeepSpeed - DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

onnxruntime - ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

quick-deploy - Optimize, convert and deploy machine learning models as fast inference API using Triton and ORT. Currently support Hugging Face transformers, PyToch, Tensorflow, SKLearn and XGBoost models.

ck - Collective Mind (CM) is a simple, modular, cross-platform and decentralized workflow automation framework with a human-friendly interface and reusable automation recipes to make it easier to compose, run, benchmark and optimize AI, ML and other applications and systems across diverse and continuously changing models, data, software and hardware

serving-compare-middleware - FastAPI middleware for comparing different ML model serving approaches

fastapi-template - Completely Scalable FastAPI based template for Machine Learning, Deep Learning and any other software project which wants to use Fast API as an API framework.

ColossalAI - Making large AI models cheaper, faster and more accessible

experta - Expert Systems for Python

FastAPI-template - Feature rich robust FastAPI template.