tritony VS budgetml

What are some alternatives?

vllm - A high-throughput and memory-efficient inference and serving engine for LLMs

pinferencia - Python + Inference - Model Deployment library in Python. Simplest model inference server ever.

DeepSpeed - DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

zenml - ZenML 🙏: Build portable, production-ready MLOps pipelines. https://zenml.io.

quick-deploy - Optimize, convert and deploy machine learning models as fast inference API using Triton and ORT. Currently support Hugging Face transformers, PyToch, Tensorflow, SKLearn and XGBoost models.

onnxruntime - ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

serving-compare-middleware - FastAPI middleware for comparing different ML model serving approaches

ck - Collective Mind (CM) is a small, modular, cross-platform and decentralized workflow automation framework with a human-friendly interface and reusable automation recipes to make it easier to build, run, benchmark and optimize AI, ML and other applications and systems across diverse and continuously changing models, data, software and hardware

ColossalAI - Making large AI models cheaper, faster and more accessible

fastapi-template - Completely Scalable FastAPI based template for Machine Learning, Deep Learning and any other software project which wants to use Fast API as an API framework.

experta - Expert Systems for Python

FastAPI-template - Feature rich robust FastAPI template.