Tritony Alternatives

Similar projects and alternatives to tritony based on common topics and language

• vllm

  30 18,571 9.9 Python tritony VS vllm

  A high-throughput and memory-efficient inference and serving engine for LLMs

  

• budgetml

  4 1,332 0.0 Python tritony VS budgetml

  Deploy a ML inference service on a budget in less than 10 lines of code.

  

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• DeepSpeed

  51 32,739 9.8 Python tritony VS DeepSpeed

  DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

  

• quick-deploy

  1 6 0.0 Python tritony VS quick-deploy

  Optimize, convert and deploy machine learning models as fast inference API using Triton and ORT. Currently support Hugging Face transformers, PyToch, Tensorflow, SKLearn and XGBoost models.

• serving-compare-middleware

  1 14 0.0 Python tritony VS serving-compare-middleware

  FastAPI middleware for comparing different ML model serving approaches

  

• ColossalAI

  42 37,911 9.7 Python tritony VS ColossalAI

  Making large AI models cheaper, faster and more accessible